Skip to content

Add async exporter streams with ASGI-safe admin response handling#44

Merged
TimurKady merged 2 commits intomainfrom
convert-stream-methods-to-async-interface
Mar 3, 2026
Merged

Add async exporter streams with ASGI-safe admin response handling#44
TimurKady merged 2 commits intomainfrom
convert-stream-methods-to-async-interface

Conversation

@TimurKady
Copy link
Copy Markdown
Owner

Motivation

  • Сделать экспорт потоковых данных неблокирующим в ASGI окружении, чтобы StreamingHttpResponse для ASGI не блокировал event loop при итерации QuerySet и при сериализации/сборке XLSX.

Description

  • Добавлены async-версии стрим-генераторов в TreeNodeExporter: async_csv_stream_data, async_json_stream_data, async_yaml_stream_data, async_xlsx_stream_data, оставлены sync-методы для обратной совместимости с WSGI (файл treenode/admin/exporter.py).
  • Блокирующие операции (итерация QuerySet, получение сериализуемой строки, построение XLSX) обёрнуты через sync_to_async(..., thread_sensitive=True) или вынесены в sync-адаптер (_get_sync_iterator_item, _build_xlsx_payload) чтобы избежать sync-операций в event loop. (treenode/admin/exporter.py).
  • process_record() теперь принимает опционально request и выбирает sync или async итератор в зависимости от контекста (ASGI/WSGI), при этом сохраняется поведение для WSGI; export_view() передаёт request в process_record() (treenode/admin/mixin.py).
  • Добавлен регрессионный тест AdminExportAsyncTests.test_export_endpoint_uses_async_stream_without_warning, который вызывает endpoint через AsyncClient, собирает потоковый ответ и проверяет корректность содержимого и отсутствие предупреждений про синхронные итераторы (treenode/tests.py).

Testing

  • Запущен тест на экспорт: python -m pytest treenode/tests.py -k export -q, тест прошёл успешно.
  • Запущен полный набор тестов для модуля: python -m pytest treenode/tests.py -q, все тесты прошли (8 passed, 1 skipped).

Codex Task

@TimurKady TimurKady added bug Something isn't working and removed codex labels Mar 2, 2026
@TimurKady
Copy link
Copy Markdown
Owner Author

@codex Check it pls. Look at the tests specifically.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e6d3089449

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +244 to +245
class AdminExportAsyncTests(TestCase):
"""Regression tests for async export streaming in ASGI mode."""
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve importer upsert regression coverage

This change replaces the previous TreeNodeImporterTests block with AdminExportAsyncTests, which removes the only test that verified TreeNodeImporter.import_tree() updates an existing row when the incoming payload contains an existing primary key. Losing that assertion means regressions in importer upsert behavior (e.g., duplicate-PK create attempts) can now ship unnoticed through CI; the async export test should be added in addition to, not instead of, the importer regression test.

Useful? React with 👍 / 👎.

@TimurKady TimurKady merged commit 18e33fb into main Mar 3, 2026
5 checks passed
@TimurKady TimurKady deleted the convert-stream-methods-to-async-interface branch March 3, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant