Bug: N^2 chart builds when using multiple charts from the same repo

[hwo411]

Currently helm-git always builds all charts from the repo for every installed chart (so we have N^2 chart builds).

Helm version: 3.17.0
helm-git version: 1.3.0
Git version: 2.48.1
OS: MacOS 14

Steps To Reproduce

1. Set HELM_GIT_REPO_CACHE and HELM_GIT_CHART_CACHE to existing folders
2. Install 1 chart from some git repo
3. Install another chart from the same git repo

Link to chart repository:

The current behavior

Can't post logs, but helm-git generates multiple same folders in cache

ls ~/.cache/helm-git/charts/
3377947c6711542441a2df575b846d2d 4fd4b5d7de2b1cf1cfa6c9d8477631d4 5b1d9360b595f7e9b0252cff6692bc68

The contents are absolutely the same (except for timestamps in index.yaml), since they come from the same branch. And it builds all the charts in the repo (so there are many gzipped charts + index.yaml).

The behavior reproduces without caching too.

The expected behavior

It has to build a single chart or at most the chart + deps, but not all charts.

Alternatively, it might reuse the folder (which is unlikely, since it's called chart cache), since all the charts in it are the same. But that could also be an option if the first one doesn't work.

Some notes

One of the reasons why it happens is here:

  chart_search=$(find "$chart_search_root" -maxdepth 2 -name "Chart.yaml" -print)
  chart_search_count=$(echo "$chart_search" | wc -l)

Since it finds all charts to build, but there might be a reason to do so.

The problem happens for us since we have a chart from oci repository as sub-chart for one of the charts. As the result it's fetched every time for every chart, which makes all the process very slow.

Update: even after excluding OCI charts the process remains quite slow

Another source of problem is that cache is built from full path to chart, but not from repo + branch/ref. If builds become cached on the level of git repo, the problem won't also exist.

[hwo411]

I tried to investigate the code more and one of the fixes on cache level could be to make request hash from helm_repo_uri instead of _raw_uri.

I wonder if it can be applied in all cases or only if package is enabled (so for raw source). Or maybe there are some other cases. I'm going to provide an MR soon with such fixes being enabled as options and I believe in MR we can decide what has to be the default.