Skip to content

Conversation

MadLittleMods
Copy link
Contributor

@MadLittleMods MadLittleMods commented Oct 13, 2025

Remove logcontext problems caused by raw deferLater. This is a normal problem where we await a deferred without wrapping it in make_deferred_yieldable(...). But I've opted to replace the usage of deferLater with something more standard for the Synapse codebase.

Part of #18905

It's unclear why we're only now seeing these failures happen with the changes from #19057

Example failures seen in https://github.com/element-hq/synapse/actions/runs/18477454390/job/52645183606?pr=19057

builtins.AssertionError: Expected `looping_call` callback from the reactor to start with the sentinel logcontext but saw task-_resumable_task-0-IBzAmHUoepQfLnEA. In other words, another task shouldn't have leaked their logcontext to us.

Dev notes

Our logcontext docs: docs/log_contexts.md

Pull Request Checklist

  • Pull request is based on the develop branch
  • Pull request includes a changelog file. The entry should:
    • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
    • Use markdown where necessary, mostly for code blocks.
    • End with either a period (.) or an exclamation mark (!).
    • Start with a capital letter.
    • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
  • Code style is correct (run the linters)

```
builtins.AssertionError: Expected `looping_call` callback from the reactor to start with the sentinel logcontext but saw task-_resumable_task-0-IBzAmHUoepQfLnEA. In other words, another task shouldn't have leaked their logcontext to us.
```
Comment on lines +174 to +177
# Create a deferred which we will never complete
incomplete_d: Deferred = Deferred()
# Await forever to simulate an aborted task because of a restart
await deferLater(self.reactor, 2**16, lambda: None)
await make_deferred_yieldable(incomplete_d)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the goal is to wait forever, let's actually wait forever instead of 2**16 seconds.

self, task: ScheduledTask
) -> Tuple[TaskStatus, Optional[JsonMapping], Optional[str]]:
# Sleep for a second
await deferLater(self.reactor, 1, lambda: None)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could have also fixed this by wrapping the deferred with make_deferred_yieldable(...). (see our logcontext docs

ex. await make_deferred_yieldable(deferLater(self.reactor, 1, lambda: None))

Instead, I've opted to replace all of our deferLater usage with more proper clock utilities that handle logcontext rules already.

@MadLittleMods MadLittleMods changed the title Remove logcontext problems caused by raw deferLater Remove logcontext problems caused by awaiting raw deferLater(...) Oct 13, 2025
@MadLittleMods MadLittleMods marked this pull request as ready for review October 13, 2025 22:45
@MadLittleMods MadLittleMods requested a review from a team as a code owner October 13, 2025 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant