Tasks that run Django queries mostly fail with "connection already closed"

[yatahaze]

I have a working Django 4.0 project. My Django 4.2 project will error out on almost any basic query that uwsgi-tasks runs.

With this code it simply is failing to run obj, created = Task.objects.get_or_create(id=task_id).

Any tips as to what could cause this? Is it simply me running a newer version of Django or is something else going on?

Here is the error output.

mark_running start
[2023-09-27 21:06:11] Background task 'process_async_events' is starting...
Traceback (most recent call last):
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/backends/base/base.py", line 308, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/backends/postgresql/base.py", line 330, in create_cursor
    cursor = self.connection.cursor()
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/psycopg/connection.py", line 853, in cursor
    self._check_connection_ok()
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/psycopg/connection.py", line 489, in _check_connection_ok
    raise e.OperationalError("the connection is closed")
psycopg.OperationalError: the connection is closed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/uwsgi_tasks/tasks.py", line 539, in signal_handler
    return self.execute_now()
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/uwsgi_tasks/tasks.py", line 228, in execute_now
    return self.function(*self.args, **self.kwargs)
  File "/home/ryan/apps/orderdirect/content/./tasks/tasks.py", line 131, in process_async_events_thread2
    process_async_events(timing, 2)
  File "/home/ryan/apps/orderdirect/content/./tasks/utils.py", line 141, in process_async_events
    rid = mark_running("process_async_events", timing)
  File "/home/ryan/apps/orderdirect/content/./tasks/utils.py", line 327, in mark_running
    obj, created = Task.objects.get_or_create(id=task_id)
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/models/query.py", line 916, in get_or_create
    return self.get(**kwargs), False
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/models/query.py", line 633, in get
    num = len(clone)
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/models/query.py", line 380, in __len__
    self._fetch_all()
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/models/sql/compiler.py", line 1558, in execute_sql
    cursor = self.connection.cursor()
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/backends/base/base.py", line 330, in cursor
    return self._cursor()
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/backends/base/base.py", line 308, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/backends/base/base.py", line 308, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/django/db/backends/postgresql/base.py", line 330, in create_cursor
    cursor = self.connection.cursor()
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/psycopg/connection.py", line 853, in cursor
    self._check_connection_ok()
  File "/home/ryan/apps/orderdirect/venv/lib64/python3.9/site-packages/psycopg/connection.py", line 489, in _check_connection_ok
    raise e.OperationalError("the connection is closed")
django.db.utils.OperationalError: the connection is closed
Wed Sep 27 14:06:11 2023 - error managing signal 1 on worker 2

[yatahaze]

Oops I think this was just caused by another uwsgi running the same project in the background.

[yatahaze]

Alright I'm officially stumped. I found that other copy of uwsgi running, stopped it and my problems disappeared. I then updated all my packages (Django 4.2.1 to 4.2.5) and the problem has reappeared. I've confirmed I don't have multiple UWSGI's running. I have no idea where to look here...

I have a similar project running similar code on 4.2.5 as well without issue...

[Bahus]

Hi, it looks like the issue related to db connection management, a single connection to a db is shared between your main application process and spooler process(es) and when your main application closes the connection - spooler fails since the connection is closed.

Here are couple workarounds which could be considered:

1. Set MAX_CONNECTION_AGE to None explicitly (https://docs.djangoproject.com/en/4.2/ref/databases/#persistent-connections), so the shared connection will never be closed - I would not recommend this for a serious project because of possible performance degradation.
2. Add lazy-apps: true to your uwsgi configuration file (https://uwsgi-docs.readthedocs.io/en/latest/Options.html?#lazy-apps). This should force a spooler to start in a separate process with a separate db connection.