Supervisord

Running Oddslingers with Supervisord

Note: Remember to Manual Setup the project before running the programs with supervisor

Install

apt install supervisor
# OR on mac:
brew install supervisor

Documentation

Supervisor

Supervisor is a process control system that spawns multiple services and handles directory mangement, restarting, and logs.

Reference: Supervisor

Supervisord uses supervisor.conf files to define services, located in /etd/supervisor/conf.d/.

Reference: supervisor.conf

Supervisord program options

A supervisor configuration file is composed of a set of sections.

A kind of section define the programs. A program, at this case the postgres service. In oddslingers the different services used have a program section defined each one and most of them has the same options setted.

Let's check every option for the postgres service:

• [program:postgres]: It is the header to define a new monitored program. In this case it will be called "postgres".
• user: This option define who is the user that the process is run as. Postgres create a user called postgres in its installation and we can use it for running the process.
• command: It is the command that starts the monitored process. Depending on oddslingers installation method a postgres database can be started by using different commands but this is usually the common way. You can check Manual setup to review the steps.
• autorestart: This option enables service autorestart if something exits or stops it unexpectedly.
• startretries: A process is considered failed when the number of attempts for restarting is more than this option.
• stopwaitsecs: This is the time period that supervisor wait to stop a service in a aggressive way when it has tried to stop before.
• stopasgroup: This option enable supervisor sends a stop signal to the whole process group. It prevents orphaned processes.
• stderr_logfile: This file store the error outputs for the program. Oddslingers store one file for every program in the /opt/oddslingers.poker/data/logs folder.
• stdout_logfile: This file store the normal outputs for the program. Oddslingers store one file for every program in the /opt/oddslingers.poker/data/logs folder.

[program:postgres]
user=postgres
priority=2
command=/usr/lib/postgresql/12/bin/postgres -D /opt/oddslingers.poker/data/postgres
autorestart=true
startretries=3
stopwaitsecs=10
stopasgroup=true
stderr_logfile=/opt/oddslingers.poker/data/logs/postgres.err.log
stdout_logfile=/opt/oddslingers.poker/data/logs/postgres.out.log

This is another program that runs a django process. It is similar to the last example but this includes another option called environment.

• environment: ENV variables to pass to the program separated by commas.

[program:django]
priority=6
command=/opt/oddslingers.poker/core/.venv/bin/python manage.py runworker
directory=/opt/oddslingers.poker/core
autostart=false
autorestart=true
startretries=3
stopasgroup=true
stderr_logfile=/opt/oddslingers.poker/data/logs/http-worker.err.log
stdout_logfile=/opt/oddslingers.poker/data/logs/http-worker.out.log
environment=DJANGO_SETTINGS_MODULE="oddslingers.settings",ODDSLINGERS_ENV='DEV',PATH="/opt/oddslingers.poker/core/.venv/bin:%(ENV_PATH)s",

For django program at this case is important pass oddslingers module settings (DJANGO_SETTINGS_MODULE), the kind of environment (ODDSLINGERS_ENV) and finally the python virtual env path (PATH).

Supervisor config file has another section to define groups. A group could be defined as a set of homogeneus programs. For oddslingers, the programs are grouped in 3 sets according to Layers of the Stack.

• oddslingers.poker group are services always are running.

[group:oddslingers.poker]
programs=nginx,postgres,redis

• oddslingers-bg are task workers.

[group:oddslingers-bg]
programs=yacron,dramatiq,botbeat

• oddslingers-django are django workers.

[group:oddslingers-django]
programs=daphne,django

Commands

Run specific services using supervisorctl

Reference: supervisorctl

Run any individual, or combination services:

supervisorctl start <service>
supervisorctl stop <service>
supervisorctl restart <service>
supervisorctl restart all

In oddslingers you can use these commands for managing the services individually or according to groups exposed previously.

For example:

# start nginx
supervisorctl start oddslingers.poker:nginx
# stop nginx
supervisorctl stop oddslingers.poker:nginx
# restart postgres
supervisorctl restart oddslingers.poker:postgres
# stop oddslingers.poker group
supervisorctl restart oddslingers.poker:*
# start oddslingers-bg group
supervisorctl start oddslingers-bg:*
# restart all the services
supervisorctl restart all