Kubernetes YAML config

[OliverEvans96]

So I think I almost have it working, but I'm stuck.

All of the volumes seem to be mounted appropriately, and the containers can talk to one another. I'm able to log in manually to the MongoDB server via pymongo from an IPython container, but there seems to be an authentication issue somewhere.

In the mongo server, I'm seeing the following error throughout /var/log/mongod/current:

2018-04-18_09:57:29.63438 2018-04-18T09:57:29.634+0000 [conn9199] assertion 13 not authorized for query on local.oplog.rs ns:local.oplog.rs query:{ orderby: { $natural: -1 }, $query: {} }

And from the other three containers (worker, peerdb, web), /var/log/meteor shows:

2018-04-18_10:01:10.93522 /bundle/programs/server/node_modules/fibers/future.js:313
2018-04-18_10:01:10.93523                                               throw(ex);
2018-04-18_10:01:10.93524                                               ^
2018-04-18_10:01:10.93525 MongoError: not authorized for query on local.oplog.rs
2018-04-18_10:01:10.93526     at Function.MongoError.create (/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/error.js:31:11)
2018-04-18_10:01:10.93527     at queryCallback (/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/cursor.js:197:34)
2018-04-18_10:01:10.93527     at /bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:469:18
2018-04-18_10:01:10.93528     at _combinedTickCallback (internal/process/next_tick.js:131:7)
2018-04-18_10:01:10.93529     at process._tickCallback (internal/process/next_tick.js:180:9)

So clearly, the logger is not authorized properly. As a result, port 80 on the web container is closed and from the browser, all we see is Bad Gateway.

Here's my super-secure run.config secret:

MONGODB_ADMIN_PWD='password'
MONGODB_CREATE_PWD='password'
MONGODB_OPLOGGER_PWD='password'

export MONGO_URL="mongodb://meteor:${MONGODB_CREATE_PWD}@mongodb/meteor"
export MONGO_OPLOG_URL="mongodb://oplogger:${MONGODB_OPLOGGER_PWD}@mongodb/local?authSource=admin"

And here's what I have so far for the kubernetes YAML.
(<ip-of-nfs-vol> is replaced with the actual ip of the NFS volume)

---

peermind-kubernetes.yaml

# NOTE: You must create a k8s secret called `mongo-config` 
# containing a file called run.config with the following format:
# MONGODB_ADMIN_PWD='<pass>'
# MONGODB_CREATE_PWD='<pass>'
# MONGODB_OPLOGGER_PWD='<pass>'
# 
# export MONGO_URL="mongodb://meteor:${MONGODB_CREATE_PWD}@mongodb/meteor"
# export MONGO_OPLOG_URL="mongodb://oplogger:${MONGODB_OPLOGGER_PWD}@mongodb/local?authSource=admin"
# To do so, you can create this file locally and run:
# kubectl create secret generic mongo-config --from-file=run.config

# NOTE: You have to put your own URL in this configMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: peermind-config
data:
  root-url: "https://peermind.nautilus.optiputer.net"
  mail-url: "smtp://mail.tnode.com"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: peermind-mongodb-claim
spec:
  storageClassName: rook-block
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: peermind-mongodb
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: peermind-mongodb
    spec:
      containers:
      - name: mongodb
        image: tozd/meteor-mongodb:2.6
        stdin: true
        tty: true
        volumeMounts:
        - name: nfs-vol
          mountPath: /var/lib/mongodb
          subPath: mongodb/data
        - name: nfs-vol
          mountPath: /var/log/mongod
          subPath: mongodb/log
        - name: mongo-config-vol
          mountPath: /etc/service/mongod/run.config
          subPath: run.config
      volumes:
      - name: nfs-vol
        nfs:
          server: <ip-of-nfs-vol>
          path: /peermind
      - name: mongo-config-vol
        secret:
          secretName: mongo-config
---
apiVersion: v1
kind: Service
metadata:
  name: mongodb
  labels:
    app: peermind-mongodb
spec:
  selector:
    app: peermind-mongodb
  ports:
  - port: 27017
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: peermind-peerdb
spec:
  template:
    metadata:
      name: peermind-peerdb
      labels:
        app: peermind-peerdb
    spec:
      containers:
      - image: peermind/peermind
        name: peerdb
        env:
        - name: WORKER_INSTANCES
          value: "0"
        - name: PEERDB_MIGRATIONS_DISABLED
          value: "1"
        - name: PEERDB_INSTANCES
          value: ""
        - name: PEERDB_INSTANCE
          # Use pod name for peerDB instance
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ROOT_URL
          valueFrom:
            configMapKeyRef:
              name: peermind-config
              key: root-url
        - name: MAIL_URL
          valueFrom:
            configMapKeyRef:
              name: peermind-config
              key: mail-url
        - name: STORAGE_DIRECTORY
          value: /storage
        volumeMounts:
        - name: mongo-config-vol
          mountPath: /etc/service/mongod/run.config
          subPath: run.config
        - name: nfs-vol
          mountPath: /storage
          subPath: meteor/storage
      volumes:
      - name: nfs-vol
        nfs:
          server: <ip-of-nfs-vol>
          path: /peermind
      - name: mongo-config-vol
        secret:
          secretName: mongo-config
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: peermind-worker
  labels:
    app: peermind-worker
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: peermind-worker
    spec:
      containers:
      - image: peermind/peermind
        name: worker
        env:
        - name: WORKER_INSTANCES
          value: ""
        - name: PEERDB_MIGRATIONS_DISABLED
          value: "1"
        - name: PEERDB_INSTANCES
          value: "0"
        - name: ROOT_URL
          valueFrom:
            configMapKeyRef:
              name: peermind-config
              key: root-url
        - name: MAIL_URL
          valueFrom:
            configMapKeyRef:
              name: peermind-config
              key: mail-url
        - name: STORAGE_DIRECTORY
          value: /storage
        volumeMounts:
        - name: mongo-config-vol
          mountPath: /etc/service/mongod/run.config
          subPath: run.config
        - name: nfs-vol
          mountPath: /storage
          subPath: meteor/storage
      volumes:
      - name: nfs-vol
        nfs:
          server: <ip-of-nfs-vol>
          path: /peermind
      - name: mongo-config-vol
        secret:
          secretName: mongo-config
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: peermind-web
  labels:
    app: peermind-web
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: peermind-web
    spec:
      containers:
      - image: peermind/peermind
        name: peermind
        volumeMounts:
        - name: mongo-config-vol
          mountPath: /etc/service/mongod/run.config
          subPath: run.config
        - mountPath: /var/log/meteor
          name: nfs-vol
          subPath: meteor/log
        - mountPath: /storage
          name: nfs-vol
          subPath: meteor/storage
      volumes:
      - name: nfs-vol
        nfs:
          server: <ip-of-nfs-vol>
          path: /peermind
      - name: mongo-config-vol
        secret:
          secretName: mongo-config
---
apiVersion: v1
kind: Service
metadata:
  name: peermind-web-service
  labels:
    app: peermind-web
spec:
  selector:
    app: peermind-web
  ports:
  - port: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: peermind-ingress
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: peermind.nautilus.optiputer.net
    http:
      paths:
      - backend:
          serviceName: peermind-web-service
          servicePort: 80

---

Let me know if anything sticks out to you!

Thanks,
Oliver

[OliverEvans96]

After trying again, I'm finding that I'm unable to authenticate into the admin DB manually from the mongodb container. I'm not sure what I changed.

I was trying to run /etc/service/mongod/run.initialization manually, but I couldn't figure out how to stop the running mongo service. None of service mongod stop, stop mongod, or /etc/init.d/mongod seemed to do the trick.