diff --git a/deployment/Chart.yaml b/deployment/Chart.yaml index af715dc..0260d08 100644 --- a/deployment/Chart.yaml +++ b/deployment/Chart.yaml @@ -2,5 +2,5 @@ apiVersion: v2 name: prowes description: A proWES Helm chart for Kubernetes type: application -version: 0.1.0 -appVersion: 1.16.0 +version: 2.0.0 +appVersion: 2.0.0 diff --git a/deployment/README.md b/deployment/README.md index 5ba6509..11a7667 100644 --- a/deployment/README.md +++ b/deployment/README.md @@ -3,8 +3,6 @@ - [Kubernetes deployment for proWES](#kubernetes-deployment-for-wes) - [Usage](#usage) - [Updates](#updates) - - [Using with an external MongoDB](#using-with-an-external-mongodb) - - [Setting up RabbitMQ for testing on OpenShift](#setting-up-rabbitmq-for-testing-on-openshift) - [Technical details](#technical-details) - [MongoDB](#mongodb) - [RabbitMQ](#rabbitmq) @@ -53,37 +51,23 @@ Create a Kubernetes Secret from the `.netrc` file: kubectl create secret generic netrc --from-file .netrc ``` -After this you can deploy proWES using `kubectl`: +You need to edit the `values.yaml` file to specify your `applicationDomain` and the `clusterType` -```bash -cd deployment/common/wes -ls wes-* | xargs -L1 kubectl create -f -``` - -Once proWES is deployed, you can expose it with the YAML files found under -`deployment/ingress`. Which file to use depends on your cluster and how ingress -is configured there. - -Creating an OpenShift Route: +After this you can deploy proWES using `helm`: ```bash -cd deployment/ingress -oc create -f wes-route.yaml +helm install prowes . -f values.yaml ``` ### Updates -If you make changes to any of the Deployments, you can update them with -`kubectl`. For example, this is how you would update the Celery worker Deployment: +If you want to edit any of the Deployments, you can update them with +`helm` and the `values.yaml` file. Once edited, you can run this command: ```bash -kubectl replace -f wes-celery-deployment.yaml +helm upgrade prowes . -f values.yaml ``` -The OpenShift specific objects need to be updated using the `oc` tool instead. -Also, if you update the Route you must use the `--force` flag. This removes and -recreates the Route. - If you want to point to a different FTP server or change the login credentials for the current FTP server, you can update the `.netrc` secret like so: @@ -91,57 +75,22 @@ for the current FTP server, you can update the `.netrc` secret like so: kubectl create secret generic netrc --from-file .netrc --dry-run -o yaml | kubectl apply -f - ``` -If you want to update the configuration, you can update the ConfigMap or use a -different ConfigMap with the same name. The Deployments expect to find the -`app_config.yaml` ConfigMap with the name `wes-config`. You can update the -ConfigMap like so: - -```bash -kubectl replace -f wes-configmap.yaml -``` - -### Using with an external MongoDB - -In certain situations, you may want to run an external MongoDB. There is an -example headless service file for this case under `deployment/mongodb`. - -Make a copy of the example Service: - -```bash -cd deployment/mongodb -cp mongodb-service-external.yaml.example mongodb-service-external.yaml -``` - -Edit the `mongodb-service-external.yaml` file and under `externalName` add your -external MongoDB's host name. After this, create the Service: - -```bash -kubectl create -f mongodb-service-external.yaml -``` +## Technical details -It is assumed that the external MongoDB already has a database called -`prowes-db` created and a user with read-write access available. Next you -will need to configure the MongoDB user and password in a secret (replace -`` and ``): +### MongoDB -```bash -kubectl create secret generic mongodb --from-literal=database-user= --from-literal=database-password= -``` +The MongoDB database is deployed using: -Depending on your MongoDB provider, the port may differ from 27017. In this -case, you will need to modify the Deployment files accordingly to use the -correct port (replace ``): +- `templates/mongodb/mongodb-deployment.yaml` -```bash -cd deployment/common -find . -name *.yaml | xargs -L1 sed -i -e 's/27017//g' -``` +### RabbitMQ -After this you can deploy proWES (see above). +The message broker RabbitMQ that allows the app to communicate with the worker +is deployed using: -## Technical details +- `templates/rabbitmq/rabbitmq-deployment.yaml` -### ApplicaWES +### WES proWES consists of five deployments: a Flask server and a Celery worker. These are deployed using: @@ -150,22 +99,14 @@ are deployed using: - `templates/prowes/celery-deployment.yaml` These deployments depend on setting up a shared ReadWriteMany volume between -Flask and Celery (`wes-volume.yaml`) and a shared ConfigMap (`wes-configmap.yaml`). -### MongoDB -The MongoDB database is deployed using: - -- `templates/mongodb/mongodb-deployment.yaml` - -### RabbitMQ +## Destroy -The message broker RabbitMQ that allows the app to communicate with the worker -is deployed using: +Simply run: -- `templates/rabbitmq/rabbitmq-deployment.yaml` +```bash +helm uninstall prowes +``` -The disadvantage with this template is that it only works with OpenShift because -it uses objects that are only available in OpenShift. For plain Kubernetes a -different approach needs to be used. diff --git a/deployment/templates/flower/flower-deployment.yaml b/deployment/templates/flower/flower-deployment.yaml index 44ada45..4e1cb64 100644 --- a/deployment/templates/flower/flower-deployment.yaml +++ b/deployment/templates/flower/flower-deployment.yaml @@ -19,3 +19,4 @@ spec: command: ['flower'] args: ['--broker=amqp://guest:guest@rabbitmq:5672//', '--port=5555', '--basic_auth={{ .Values.flower.basicAuth }}'] name: flower + resources: {{- toYaml .Values.flower.resources | nindent 10 }} diff --git a/deployment/templates/mongodb/mongo-init-script.yaml b/deployment/templates/mongodb/mongo-init-script.yaml new file mode 100644 index 0000000..937f66c --- /dev/null +++ b/deployment/templates/mongodb/mongo-init-script.yaml @@ -0,0 +1,41 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: mongo-init-script +data: + init-script.js: | + db = db.getSiblingDB('runStore'); + dbproWES = db.getSiblingDB('{{ tpl .Values.mongodb.secret.databaseName . }}') + + dbproWES.createUser({ + user: "{{ tpl .Values.mongodb.secret.databaseUser . }}", + pwd: "{{ tpl .Values.mongodb.secret.databasePassword . }}", + roles: [ + { + role: "readWrite", + db: "{{ tpl .Values.mongodb.secret.databaseName . }}" + } + ] + }); + + // Create the 'runs' and 'service_info' collections + // Database configuration from https://github.com/elixir-cloud-aai/proWES/blob/59b53a4bdd06dd83ff41754b27d97309073ed237/pro_wes/config.yaml#L30 + db.createCollection('runs'); + db.runs.createIndex( + { run_id: 1, task_id: 1 }, + { unique: true, sparse: true } + ); + db.createCollection('service_info'); + db.service_info.createIndex( + { id: 1 } + ); + + dbproWES.createCollection('runs'); + dbproWES.runs.createIndex( + { run_id: 1, task_id: 1 }, + { unique: true, sparse: true } + ); + dbproWES.createCollection('service_info'); + dbproWES.service_info.createIndex( + { id: 1} + ); diff --git a/deployment/templates/mongodb/mongodb-deployment.yaml b/deployment/templates/mongodb/mongodb-deployment.yaml index 22261d7..3cfd44f 100644 --- a/deployment/templates/mongodb/mongodb-deployment.yaml +++ b/deployment/templates/mongodb/mongodb-deployment.yaml @@ -16,25 +16,30 @@ spec: spec: containers: - env: - - name: MONGODB_USER + - name: MONGO_INITDB_ROOT_USERNAME valueFrom: secretKeyRef: - key: database-user + key: databaseRootUsername name: {{ .Values.mongodb.appName }} - - name: MONGODB_PASSWORD + - name: MONGO_INITDB_ROOT_PASSWORD valueFrom: secretKeyRef: - key: database-password + key: databaseRootPassword name: {{ .Values.mongodb.appName }} - - name: MONGODB_ADMIN_PASSWORD + - name: MONGO_INITDB_DATABASE valueFrom: secretKeyRef: - key: database-admin-password + key: databaseName name: {{ .Values.mongodb.appName }} - - name: MONGODB_DATABASE + - name: MONGO_APP_USERNAME valueFrom: secretKeyRef: - key: database-name + key: databaseUser + name: {{ .Values.mongodb.appName }} + - name: MONGO_APP_PASSWORD + valueFrom: + secretKeyRef: + key: databasePassword name: {{ .Values.mongodb.appName }} image: {{ .Values.mongodb.image }} imagePullPolicy: IfNotPresent @@ -57,20 +62,23 @@ spec: - '-i' - '-c' - >- - mongo 127.0.0.1:27017/$MONGODB_DATABASE -u $MONGODB_USER -p - $MONGODB_PASSWORD --eval="quit()" + mongosh --host 127.0.0.1:27017 -u $MONGO_INITDB_ROOT_USERNAME -p $MONGO_INITDB_ROOT_PASSWORD --authenticationDatabase admin $MONGO_INITDB_DATABASE --eval="quit()" failureThreshold: 3 - initialDelaySeconds: 3 + initialDelaySeconds: 30 periodSeconds: 10 successThreshold: 1 - timeoutSeconds: 1 - resources: - limits: - memory: 512Mi + timeoutSeconds: 50 + resources: {{- toYaml .Values.mongodb.resources | nindent 12 }} volumeMounts: - - mountPath: /var/lib/mongodb/data + - mountPath: /data/db name: mongodb-data + - name: init-script + mountPath: /docker-entrypoint-initdb.d/init-script.js + subPath: init-script.js volumes: - name: mongodb-data persistentVolumeClaim: claimName: {{ .Values.mongodb.appName }}-volume + - name: init-script + configMap: + name: mongo-init-script diff --git a/deployment/templates/mongodb/mongodb-pvc.yaml b/deployment/templates/mongodb/mongodb-pvc.yaml index 70fc970..dede4b5 100644 --- a/deployment/templates/mongodb/mongodb-pvc.yaml +++ b/deployment/templates/mongodb/mongodb-pvc.yaml @@ -4,7 +4,7 @@ metadata: name: {{ .Values.mongodb.appName }}-volume spec: accessModes: - - ReadWriteMany + - {{ .Values.storageAccessMode }} resources: requests: storage: {{ .Values.mongodb.volumeSize }} \ No newline at end of file diff --git a/deployment/templates/mongodb/mongodb-secret.yaml b/deployment/templates/mongodb/mongodb-secret.yaml index 57949b7..e634e49 100644 --- a/deployment/templates/mongodb/mongodb-secret.yaml +++ b/deployment/templates/mongodb/mongodb-secret.yaml @@ -4,7 +4,7 @@ type: Opaque metadata: name: {{ .Values.mongodb.appName }} data: - database-admin-password: {{ .Values.mongodb.databaseAdminPassword | b64enc }} - database-name: {{ .Values.mongodb.databaseName | b64enc }} - database-password: {{ .Values.mongodb.databasePassword | b64enc }} - database-user: {{ .Values.mongodb.databaseUser | b64enc }} + {{- range $key, $val := .Values.mongodb.secret }} + "{{ $key }}": "{{ tpl $val $ | b64enc }}" + {{- end }} + diff --git a/deployment/templates/prowes/celery-deployment.yaml b/deployment/templates/prowes/celery-deployment.yaml index c8d05d1..3739a8c 100644 --- a/deployment/templates/prowes/celery-deployment.yaml +++ b/deployment/templates/prowes/celery-deployment.yaml @@ -16,6 +16,7 @@ spec: image: busybox command: [ 'mkdir' ] args: [ '-p', '/data/db', '/data/output', '/data/tmp' ] + resources: {{- toYaml .Values.celeryWorker.initResources | nindent 10 }} volumeMounts: - mountPath: /data name: prowes-volume @@ -34,29 +35,23 @@ spec: - name: MONGO_USERNAME valueFrom: secretKeyRef: - key: database-user + key: databaseUser name: {{ .Values.mongodb.appName }} - name: MONGO_PASSWORD valueFrom: secretKeyRef: - key: database-password + key: databasePassword name: {{ .Values.mongodb.appName }} - name: MONGO_DBNAME valueFrom: secretKeyRef: - key: database-name + key: databaseName name: {{ .Values.mongodb.appName }} - name: RABBIT_HOST value: {{ .Values.rabbitmq.appName }} - name: RABBIT_PORT value: "5672" - resources: - requests: - memory: "512Mi" - cpu: "300m" - limits: - memory: "8Gi" - cpu: "1" + resources: {{- toYaml .Values.celeryWorker.resources | nindent 10 }} volumeMounts: - mountPath: /data name: prowes-volume diff --git a/deployment/templates/prowes/prowes-deployment.yaml b/deployment/templates/prowes/prowes-deployment.yaml index 7b54166..bab88f2 100644 --- a/deployment/templates/prowes/prowes-deployment.yaml +++ b/deployment/templates/prowes/prowes-deployment.yaml @@ -12,11 +12,24 @@ spec: labels: app: {{ .Values.prowes.appName }} spec: + {{- if eq .Values.storageAccessMode "ReadWriteOnce" }} + affinity: + podAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchExpressions: + - key: app + operator: In + values: + - {{ .Values.celeryWorker.appName }} + topologyKey: "kubernetes.io/hostname" + {{- end }} initContainers: - name: vol-init image: busybox command: [ 'mkdir' ] args: [ '-p', '/data/db', '/data/specs' ] + resources: {{- toYaml .Values.prowes.initResources | nindent 10 }} volumeMounts: - mountPath: /data name: prowes-volume @@ -35,22 +48,23 @@ spec: - name: MONGO_USERNAME valueFrom: secretKeyRef: - key: database-user + key: databaseUser name: {{ .Values.mongodb.appName }} - name: MONGO_PASSWORD valueFrom: secretKeyRef: - key: database-password + key: databasePassword name: {{ .Values.mongodb.appName }} - name: MONGO_DBNAME valueFrom: secretKeyRef: - key: database-name + key: databaseName name: {{ .Values.mongodb.appName }} - name: RABBIT_HOST value: {{ .Values.rabbitmq.appName }} - name: RABBIT_PORT value: "5672" + resources: {{- toYaml .Values.prowes.resources | nindent 10 }} livenessProbe: tcpSocket: port: prowes-port diff --git a/deployment/templates/prowes/prowes-volume.yaml b/deployment/templates/prowes/prowes-volume.yaml index fc07e34..2e7e6cd 100644 --- a/deployment/templates/prowes/prowes-volume.yaml +++ b/deployment/templates/prowes/prowes-volume.yaml @@ -5,7 +5,7 @@ metadata: name: {{ .Values.prowes.appName}}-volume spec: accessModes: - - ReadWriteMany + - {{ .Values.storageAccessMode }} resources: requests: storage: '1Gi' diff --git a/deployment/templates/rabbitmq/rabbitmq-deployment.yaml b/deployment/templates/rabbitmq/rabbitmq-deployment.yaml index 7b8926f..212cfbe 100644 --- a/deployment/templates/rabbitmq/rabbitmq-deployment.yaml +++ b/deployment/templates/rabbitmq/rabbitmq-deployment.yaml @@ -17,10 +17,11 @@ spec: containers: - name: rabbitmq image: {{ .Values.rabbitmq.image }} + resources: {{- toYaml .Values.rabbitmq.resources | nindent 10 }} volumeMounts: - mountPath: /var/lib/rabbitmq name: rabbitmq-volume volumes: - name: rabbitmq-volume persistentVolumeClaim: - claimName: {{ .Values.rabbitmq.appName }}-volume \ No newline at end of file + claimName: {{ .Values.rabbitmq.appName }}-volume diff --git a/deployment/templates/rabbitmq/rabbitmq-pvc.yaml b/deployment/templates/rabbitmq/rabbitmq-pvc.yaml index 544e239..9d36f77 100644 --- a/deployment/templates/rabbitmq/rabbitmq-pvc.yaml +++ b/deployment/templates/rabbitmq/rabbitmq-pvc.yaml @@ -5,7 +5,7 @@ metadata: name: {{ .Values.rabbitmq.appName }}-volume spec: accessModes: - - ReadWriteMany + - {{ .Values.storageAccessMode }} resources: requests: - storage: {{ .Values.rabbitmq.volumeSize }} \ No newline at end of file + storage: {{ .Values.rabbitmq.volumeSize }} diff --git a/deployment/values.yaml b/deployment/values.yaml index d821db7..109c2c4 100644 --- a/deployment/values.yaml +++ b/deployment/values.yaml @@ -2,35 +2,89 @@ # This is a YAML-formatted file. # Declare variables to be passed into your templates. -applicationDomain: rahtiapp.fi +applicationDomain: "" # which cluster type proWES is going to be deployed on # it can be either 'kubernetes' or 'openshift' clusterType: kubernetes +# mongodb-pvc.yaml/rabbitmq-pvc.yaml, change to ReadWriteMany if storageClass can do RWX +storageAccessMode: ReadWriteOnce + flower: appName: prowes-flower basicAuth: admin:admin image: endocode/flower - + resources: + limits: + cpu: 200m + memory: 1Gi + requests: + cpu: 200m + memory: 500Mi prowes: appName: prowes image: elixircloud/prowes:latest + initResources: + limits: + memory: 16Mi + cpu: 50m + requests: + memory: 16Mi + cpu: 50m + resources: + limits: + memory: 256Mi + cpu: 100m + requests: + memory: 256Mi + cpu: 100m celeryWorker: appName: celery-worker image: elixircloud/prowes:latest + initResources: + limits: + memory: 16Mi + cpu: 50m + requests: + memory: 16Mi + cpu: 50m + resources: + limits: + cpu: 200m + memory: 256Mi + requests: + cpu: 100m + memory: 256Mi mongodb: appName: mongodb - databaseAdminPassword: adminpasswd - databaseName: prowes-db - databasePassword: prowes-db-passwd - databaseUser: prowes-user + secret: + databaseRootUsername: "" + databaseRootPassword: "" + databaseUser: "" + databasePassword: "" + databaseName: "" volumeSize: 1Gi - image: centos/mongodb-36-centos7 + image: docker.io/library/mongo:noble + resources: + limits: + cpu: 200m + memory: 512Mi + requests: + cpu: 200m + memory: 512Mi rabbitmq: appName: rabbitmq volumeSize: 1Gi - image: rabbitmq:3-management + image: rabbitmq:4.1.4-management + resources: + limits: + cpu: 200m + memory: 1Gi + requests: + cpu: 200m + memory: 256Mi +