Running a live OFN Instance

Preparation

When running a live instance you should consider the following:

Backup sysadmins

Add ssh keys for your core sysadmins, and give access to some backup sysadmins in case your core team is unavailable during an emergency. This should be done by making a PR to ofn-install to add any new keys in /files/keys and naming the sysadmins in your group_vars file, e.g: /inventory/group_vars/uk.yml

Server Monitoring

Set up Server Monitoring for receiving alerts if the server goes down. If you add an integration with Slack, you can have alerts sent to a private channel for your instance's alerts, and invite your core and backup sysadmins to that channel so they receive your alerts too.

Tips and tricks

There are various issues you might encounter whilst running a live instance.

• To see check on a running process like delayed_job, use: ps faux | grep process_name

• If there are issues with missing templates after a deployment, you can try recompiling the assets by switching to the openfoodnetwork user with: sudo su openfoodnetwork and using this: bash -lc "bundle exec rake assets:precompile RAILS_ENV=production"

• When dealing with issues on a server, sometimes you will need to restart either unicorn or nginx, or both. You can do that with: sudo systemctl restart nginx.service and sudo systemctl restart unicorn_openfoodnetwork.service

Hard drive full

When your hard drive gets full, use a tool like ncdu to locate problem areas. Common causes:

• Check for large temp files in places like ncdu /home/openfoodnetwork
• Remove old Rubies and Gems: (todo: automate this)
  • /home/openfoodnetwork/.rbenv/versions
  • /home/openfoodnetwork/.gem/ruby
• /tmp/v8-compile-cache-1001 - This seems to be stored each and every deployment. Remove them all!

Otherwise you may simply need more space. Check your hosting provider console and increase the hard drive.

Also consider using an external file storage method if not already.