Skip to content

Support ASGI async export streams, refactor XLSX payload, add importer bulk-update and tests#46

Closed
TimurKady wants to merge 2 commits intomainfrom
convert-stream-methods-to-async-interface
Closed

Support ASGI async export streams, refactor XLSX payload, add importer bulk-update and tests#46
TimurKady wants to merge 2 commits intomainfrom
convert-stream-methods-to-async-interface

Conversation

@TimurKady
Copy link
Copy Markdown
Owner

Motivation

  • Enable non-blocking exports under ASGI by providing asynchronous stream generators to avoid Django warnings about synchronous iterators.
  • Provide a thread-safe way to build XLSX payloads so large-file exports can be produced for both sync and async handlers.
  • Allow the importer to update existing rows (by primary key) during import instead of only creating new records.
  • Add regression tests to validate async export streaming and importer update behavior.

Description

  • Added asgiref.sync.sync_to_async usage and helper TreeNodeExporter._get_sync_iterator_item and async stream variants async_csv_stream_data, async_json_stream_data, async_yaml_stream_data, and async_xlsx_stream_data to treenode/admin/exporter.py.
  • Refactored XLSX generation into _build_xlsx_payload and adjusted xlsx_stream_data to yield the built payload, and added _resolve_stream_data to select async or sync stream plus process_record(request=None) to detect ASGI requests.
  • Updated treenode/admin/mixin.py to pass the request into process_record so the exporter can choose async streams for ASGI.
  • Enhanced treenode/admin/importer.py to detect existing objects per level via _get_existing_objects_map, normalize parent references with _normalize_parent_reference, collect to_update items, and perform bulk_update using _get_update_fields, while tracking result['updated'].
  • Extended treenode/tests.py with AdminExportAsyncTests (async export streaming) and TreeNodeImporterTests (import updates existing row), and added necessary imports and fixtures.

Testing

  • Added unit tests in treenode/tests.py including test_export_endpoint_uses_async_stream_without_warning and test_import_tree_updates_existing_row_by_pk.
  • Ran the Django test suite (test cases under treenode.tests) exercising async export and importer update paths.
  • All automated tests completed successfully.

Codex Task

@TimurKady TimurKady closed this Mar 3, 2026
@TimurKady TimurKady reopened this Mar 3, 2026
@TimurKady TimurKady closed this Mar 3, 2026
@TimurKady TimurKady deleted the convert-stream-methods-to-async-interface branch March 3, 2026 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant