Helm Chart - not a valid chart repository or cannot be reached

[dinbtechit]

Bug Report

Unable to add helm chart repo - https://aot-technologies.github.io/helm-charts

Steps to Reproduce

1. Add helm chart

    helm repo add formsflow https://aot-technologies.github.io/helm-charts

   Output:

    Error: looks like "https://aot-technologies.github.io/helm-charts" is not a valid chart repository or cannot be reached: failed to 
    fetch https://aot-technologies.github.io/helm-charts/index.yaml : 404 Not Found

[sumesh-aot]

@dinbtechit We had renamed the repo which caused this issue. Updated the doc to reflect this https://aot-technologies.github.io/forms-flow-ai-doc/#kubernetes_individual_services

[dinbtechit]

Thank you so much for the quick response. I have couple of questions.

1. I want to use our existing keycloak instance. I have already imported the form-flow-ai realm configuration into our keycloak. How do I tell helm charts to use my keycloak instance?

2. Similarly can I use our existing postgress, redis, MongoDB with helm charts?

[sumesh-aot]

@dinbtechit as in that documentation, you can pass a custom keycloak details. The example below is for just updating only keycloak host URL, but you can override any values here https://github.com/AOT-Technologies/forms-flow-ai-charts/blob/master/charts/forms-flow-ai/values.yaml by using helm --set command.

And for disabling default DB setup, you may provide enabled=false for mongo and postgres. Ref : https://github.com/AOT-Technologies/forms-flow-ai-charts/blob/master/charts/forms-flow-ai/values.yaml#L38 and https://github.com/AOT-Technologies/forms-flow-ai-charts/blob/master/charts/forms-flow-ai/values.yaml#L53.
The current redis setup we have is for only analytics : https://github.com/AOT-Technologies/forms-flow-ai-charts/blob/master/charts/forms-flow-analytics/values.yaml#L16, which unfortunately doesn't have an option to disable.

If you have more customization, I would suggest to clone https://github.com/AOT-Technologies/forms-flow-ai-charts, then execute helm commands from the local version than the published version. Also would be great if you can share your feedback on the chart as a PR or as a ticket.

[dinbtechit]

This was very helpful... thank you so much.

[dinbtechit]

Are you able to send me the steps to update the helm charts locally? I would like to reduce the MongoDB and Postgres storage size from the default 8Gi to 2Gi

Unable to install the helm chart from my local directory not sure what I am missing.

╭─Dineshs-MBP in $ ~/code/formsflow/forms-flow-ai-charts/charts on master ✘ (origin/master +1)
╰$ helm install forms-flow-ai forms-flow-ai --set Domain=$DOMAIN_NAME  --set forms-flow-idm.keycloak.ingress.hostname=$KEYCLOAK.$DOMAIN_NAME  --namespace $NAMESPACE 
Error: INSTALLATION FAILED: An error occurred while checking for chart dependencies. You may need to run `helm dependency build` to fetch missing dependencies: found in Chart.yaml, but missing in charts/ directory: postgresql, mongodb

[sumesh-aot]

@dinbtechit It's complaining about the dependency.
So you would need go inside forms-flow-ai then run helm dependency build.
Also for changing the storage; you would need to add

persistence:
    size: 2Gi

Here https://github.com/AOT-Technologies/forms-flow-ai-charts/blob/master/charts/forms-flow-ai/values.yaml#L47
FYI : the chart is using bitnami chart for mongo; https://github.com/bitnami/charts/tree/main/bitnami/mongodb so any customization would need to follow it's structure.

[dinbtechit]

ty ty! Yes I updated few things in values.yml for my infrastructure:

In charts/forms-flow-ai/values.yaml updated the following:

forms-flow-bpm:
  clientid: "forms-flow-bpm"
  clientsecret: "786001d6-68a8-4519-903c-bc5b5a870d68".     // <--------<< to my keycloak clientSecret
...
mongodb:
  persistence:
    size: 2Gi                    // <--------<< to my keycloak secret

PostgreSQL:
   . . .
    persistence:
     . . .
      size: 2Gi                // <--------<< to my keycloak secret

. . .

And did the following commands to install the updates

1. cd into forms-flow-ai
2. built helm dependencies.

charts/forms-flow-ai$ helm dependency build

output:

Getting updates for unmanaged Helm repositories...
...Unable to get an update from the "https://charts.bitnami.com/bitnami/mongodb" chart repository:
        failed to fetch https://charts.bitnami.com/bitnami/mongodb/index.yaml : 403 Forbidden
...Unable to get an update from the "https://charts.bitnami.com/bitnami/postgresql" chart repository:
        failed to fetch https://charts.bitnami.com/bitnami/postgresql/index.yaml : 403 Forbidden
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "formsflow" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
Error: no cached repository for helm-manager-849e65aefd4347379fd5e9f0d931487abd34b10467f1a9e73238db3d5726a395 found. (try 'helm repo update'): open /Users/diensh/Library/Caches/helm/repository/helm-manager-849e65aefd4347379fd5e9f0d931487abd34b10467f1a9e73238db3d5726a395-index.yaml: no such file or directory

3. And installed the helm chart again. (this time from local directory)

⚠️ Note sure if I am doing something wrong with the command.

╭─Dineshs-MBP in ~/code/formsflow/forms-flow-ai-charts/charts/forms-flow-ai on master ✘ (origin/master +1)
╰$ cd ..         # going one level up
╭─Dineshs-MBP in ~/code/formsflow/forms-flow-ai-charts/charts on master ✘ (origin/master +1)
╰$ helm install forms-flow-ai forms-flow-ai --set Domain=$DOMAIN_NAME  --set forms-flow-idm.keycloak.ingress.hostname=$KEYCLOAK.$DOMAIN_NAME  --namespace $NAMESPACE
Error: INSTALLATION FAILED: An error occurred while checking for chart dependencies. You may need to run `helm dependency build` to fetch missing dependencies: found in Chart.yaml, but missing in charts/ directory: postgresql, mongodb

[sumesh-aot]

@cberg-aot Any idea on this error ?

[cberg-aot]

@dinbtechit @sumesh-aot I don't think helm dependency build is the correct command here. Try helm dependency update.

[dinbtechit]

On a side note - I managed to get it up and running by increasing our quota limits. There is a bunch of issues we identified with the helm charts. Maybe we will submit a PR for that.. Also it's taking a significant amount of CPU & memory.
roughly around - CPU request 16Gi, Memory Requests up to 32Gi and Storage up to 64Gi.

So many different databases - postgres, MongoDB. There is no easy way to update the forms-flow-ai - content security policy so we couldn't use an external keycloak instance as we are getting CORS error.

However, I am still not sure how to get past the below error. I am also new to Helm charts so there is a bit of a learning curve for me and I don't understand why I am getting a 403 forbidden error for locally updating the values of bitnami helm charts.

@cberg-aot - That makes sense. I am following this script here.

export DIRECTORY="../charts"
$ helm dependency update $DIRECTORY/forms-flow-ai/

output:

Note there are some 403 Forbidden errors and it did not create the .tgz file.

Getting updates for unmanaged Helm repositories...
...Unable to get an update from the "https://charts.bitnami.com/bitnami/mongodb" chart repository:
        failed to fetch https://charts.bitnami.com/bitnami/mongodb/index.yaml : 403 Forbidden
...Unable to get an update from the "https://charts.bitnami.com/bitnami/postgresql" chart repository:
        failed to fetch https://charts.bitnami.com/bitnami/postgresql/index.yaml : 403 Forbidden
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "formsflow" chart repository
Update Complete. ⎈Happy Helming!⎈
Error: no cached repository for helm-manager-849e65aefd4347379fd5e9f0d931487abd34b10467f1a9e73238db3d5726a395 found. (try 'helm repo update'): open /Users/srinivad/Library/Caches/helm/repository/helm-manager-849e65aefd4347379fd5e9f0d931487abd34b10467f1a9e73238db3d5726a395-index.yaml: no such file or directory

2. And then tried to install the chart are specified in this line41. but with an upgrade as I already have a version of the chart already installed.

$ helm upgrade forms-flow-ai $DIRECTORY/forms-flow-ai --set Domain=$DOMAIN_NAME --set forms-flow-idm.keycloak.ingress.hostname=forms-flow-idm-$NAMESPACE.$DOMAIN_NAME --namespace $NAMESPACE

output

Error: An error occurred while checking for chart dependencies. You may need to run `helm dependency build` to fetch missing dependencies: found in Chart.yaml, but missing in charts/ directory: postgresql, mongodb

[cberg-aot]

Turns out there is something up with the bitnami chart registry! There is source code for the postgres and mongodb charts available here: https://github.com/bitnami/charts/tree/main/bitnami

My suggestion as a work around:

• Download postgresql/mongodb charts from bitnami github.
• Create a charts dir -> forms-flow-ai/charts/
• move postgres/mongo into the forms-flow-ai/charts directory
• Modify forms-flow-ai/Chart.yaml to refrence the local charts.

I will investigate how long this outage will be and decide if we need to keep a fork of postgres/mongo to decouple from bitnami.

[cberg-aot]

@dinbtechit @sumesh-aot forms-flow-idm/keycloak will also require some modification since it is set to pull from the bitnami registry by default.

[cberg-aot]

@dinbtechit @sumesh-aot bitnami/charts#13309 Active thread - 2/hrs

[cberg-aot]

@dinbtechit @sumesh-aot I was thinking about this. You should not need to run helm repo update at all, and the issue you are having is avoidable.

https://artifacthub.io/packages/helm/bitnami/postgresql

The forms-flow-ai Chart is able to modify all values of its subcharts. Above is the api for postgres.

To set resourses on databases you can modify the resources on install.

For example:
helm install forms-flow-ai formsflow/forms-flow-ai --set Domain=$DOMAIN --set postgresql.primary.resources.requests.memory=$MEM_REQ

[dinbtechit]

I tried to do that but got an error saying values mismatch or something like that. Let me try that again. Thanks for the info!

[cberg-aot]

tested