[reivilibre]

Part of: MSC4354 (Sticky Events)

Our locally-sent Sticky Events need to be reliably (within bounds of possibility) sent to remote homeservers.

This PR adds the mechanism to make this happen.

Notable omissions:

• catching up historical sticky events to homeservers that only just join the room

---

Review Guidance

The following commit is refactoring and hopefully stands alone, with no behavioural changes. I suggest considering it on its own.

• Pull out user_is_local_like_pattern

After that, I recommend looking at the database schema (delta introducing a table) to understand the data model
for tracking backlogged sticky events. That's more or less the central part of this PR.

It may be helpful to remember that Sticky Events are events that are temporarily 'promoted' such that we want to ensure that all users and all homeservers receive them,
without letting them fall down 'gaps' of unretrieved history. Essentially, it's a temporary delivery reliability increase.

There are then roughly three logical pieces to review:

• how, when sending federation transactions, we retrieve events from the sticky event backlog and record that
  we have succeeded in sending them
• how we become aware of a new backlog of sticky events. (Bear in mind that, unless the remote homeserver goes down, the intention
  is for sticky events to not have any special processing)
  The docstring on mark_backlogged_sticky_events_after_catchup_transaction contains a diagrammatical explanation of this
  mechanism and how it works.
• how we round-robin between sending out sticky event backlog transactions and 'main queue' (normal) transactions containing
  real-time PDUs and EDUs

Or if you prefer, it may be easiest to just start looking at the storage changes
to get an idea of the storage contract exposed by the database layer, since
the federation transmission logic itself is (I think) fairly understandable,
as long as you trust that contract.

Happy to pair, explain, add comments, anything; just let me know!

---

1. Replace _TransactionQueueManager with prepare/complete transaction methods
   Split out at Refactor the federation transmission code to delineate transaction preparation and completion. #20166

2. Pull out user_is_local_like_pattern

3. unwrap utility

4. Add destination_room_sticky_events_backlog table

5. Introduce a StickyEventStreamPosition NewType

6. Add store methods

7. Wire sticky event backlog transactions into the per-destination queue

8. Add mark of backlog after catch-up transaction

9. Add tests

[github-actions]

Schema Diff

Please check that this looks as expected!

common (sqlite)

Unchanged

common (postgres)

Unchanged

main (sqlite)

--- /tmp/schema_diff_vc7yy24k/before/main/full_schemas/9999/full.sql.sqlite	2026-10-02 10:39:20.322790882 +0000
+++ /tmp/schema_diff_vc7yy24k/after/main/full_schemas/9999/full.sql.sqlite	2026-10-02 10:39:09.954713358 +0000
@@ -1194,20 +1194,45 @@
 
   -- The full user ID of the local user that should receive the profile update.
   user_id TEXT NOT NULL,
 
   -- Unix timestamp (milliseconds). Used to determine when to prune rows (to prevent the table
   -- from growing indefinitely).
   inserted_ts BIGINT NOT NULL,
 
   PRIMARY KEY (user_id, stream_id)
 );
+CREATE TABLE destination_rooms_sticky_events_backlog (
+    -- Server name of the remote homeserver.
+    destination TEXT NOT NULL,
+
+    -- Room ID in which sticky events have been missed.
+    room_id TEXT NOT NULL
+        -- Only track this information for rooms we know about;
+        -- if we delete a room locally then also delete the tracking info.
+        REFERENCES rooms(room_id) ON DELETE CASCADE,
+
+    -- Position in the sticky events stream, corresponding to the
+    -- `sticky_events.stream_id` of the first sticky event that
+    -- has yet to be sent.
+    --
+    -- Because sticky events must be sent in order (as per MSC4354),
+    -- all subsequent sticky events for the same room with higher
+    -- `stream_id`s are also unsent.
+    --
+    -- Not a foreign key because we must still support expiration of sticky events.
+    sticky_events_stream_position INTEGER NOT NULL,
+
+    -- It's enough to track one position per (destination, room_id) pair.
+    PRIMARY KEY (destination, room_id)
+);
+CREATE INDEX destination_rooms_sticky_events_backlog_room_id ON destination_rooms_sticky_events_backlog (room_id);
 CREATE UNIQUE INDEX receipts_graph_unique_index ON receipts_graph (room_id, receipt_type, user_id) WHERE thread_id IS NULL;
 CREATE UNIQUE INDEX receipts_linearized_unique_index ON receipts_linearized (room_id, receipt_type, user_id) WHERE thread_id IS NULL;
 CREATE INDEX device_lists_changes_in_room_by_room_idx ON device_lists_changes_in_room (room_id, stream_id) ;
 CREATE INDEX events_jump_to_date_idx ON events (room_id, origin_server_ts) WHERE NOT outlier;
 CREATE INDEX room_membership_user_room_idx ON room_memberships (user_id, room_id) ;
 CREATE UNIQUE INDEX profiles_full_user_id_key ON profiles (full_user_id) ;
 CREATE UNIQUE INDEX full_users_unique_idx ON user_filters (full_user_id, filter_id) ;
 CREATE INDEX event_push_actions_stream_highlight_index ON event_push_actions (highlight, stream_ordering) WHERE highlight=0;
 CREATE INDEX current_state_events_stream_ordering_idx ON current_state_events (event_stream_ordering) ;
 CREATE INDEX local_current_membership_stream_ordering_idx ON local_current_membership (event_stream_ordering) ;

main (postgres)

--- /tmp/schema_diff_vc7yy24k/before/main/full_schemas/9999/full.sql.postgres	2026-10-02 10:39:20.791793642 +0000
+++ /tmp/schema_diff_vc7yy24k/after/main/full_schemas/9999/full.sql.postgres	2026-10-02 10:39:10.462717334 +0000
@@ -203,20 +203,25 @@
 CREATE TABLE destination_rooms (
     destination text NOT NULL,
     room_id text NOT NULL,
     stream_ordering bigint NOT NULL
 );
 COMMENT ON TABLE destination_rooms IS 'Information about transmission of PDUs in a given room to a given remote homeserver.';
 COMMENT ON COLUMN destination_rooms.destination IS 'server name of remote homeserver in question';
 COMMENT ON COLUMN destination_rooms.room_id IS 'room ID in question';
 COMMENT ON COLUMN destination_rooms.stream_ordering IS '`stream_ordering` of the most recent PDU in this room that needs to be sent (by us) to this homeserver.
 This can only be pointing to our own PDU because we are only responsible for sending our own PDUs.';
+CREATE TABLE destination_rooms_sticky_events_backlog (
+    destination text NOT NULL,
+    room_id text NOT NULL,
+    sticky_events_stream_position integer NOT NULL
+);
 CREATE TABLE destinations (
     destination text NOT NULL,
     retry_last_ts bigint,
     retry_interval bigint,
     failure_ts bigint,
     last_successful_stream_ordering bigint
 );
 COMMENT ON TABLE destinations IS 'Information about remote homeservers and the health of our connection to them.';
 COMMENT ON COLUMN destinations.destination IS 'server name of remote homeserver in question';
 COMMENT ON COLUMN destinations.retry_last_ts IS 'The last time we tried and failed to reach the remote server, in ms.
@@ -1433,20 +1438,22 @@
 ALTER TABLE ONLY current_state_events
     ADD CONSTRAINT current_state_events_room_id_type_state_key_key UNIQUE (room_id, type, state_key);
 ALTER TABLE ONLY dehydrated_devices
     ADD CONSTRAINT dehydrated_devices_pkey PRIMARY KEY (user_id);
 ALTER TABLE ONLY delayed_events
     ADD CONSTRAINT delayed_events_pkey PRIMARY KEY (user_localpart, delay_id);
 ALTER TABLE ONLY delayed_events_stream_pos
     ADD CONSTRAINT delayed_events_stream_pos_lock_key UNIQUE (lock);
 ALTER TABLE ONLY destination_rooms
     ADD CONSTRAINT destination_rooms_pkey PRIMARY KEY (destination, room_id);
+ALTER TABLE ONLY destination_rooms_sticky_events_backlog
+    ADD CONSTRAINT destination_rooms_sticky_events_backlog_pkey PRIMARY KEY (destination, room_id);
 ALTER TABLE ONLY destinations
     ADD CONSTRAINT destinations_pkey PRIMARY KEY (destination);
 ALTER TABLE ONLY device_lists_changes_converted_stream_position
     ADD CONSTRAINT device_lists_changes_converted_stream_position_lock_key UNIQUE (lock);
 ALTER TABLE ONLY device_lists_changes_in_room_max_pruned_stream_id
     ADD CONSTRAINT device_lists_changes_in_room_max_pruned_stream_id_lock_key UNIQUE (lock);
 ALTER TABLE ONLY device_lists_remote_pending
     ADD CONSTRAINT device_lists_remote_pending_pkey PRIMARY KEY (stream_id);
 ALTER TABLE ONLY devices
     ADD CONSTRAINT device_uniqueness UNIQUE (user_id, device_id);
@@ -1644,20 +1651,21 @@
 CREATE INDEX current_state_delta_stream_room_idx ON current_state_delta_stream USING btree (room_id, stream_id);
 CREATE INDEX current_state_events_member_index ON current_state_events USING btree (state_key) WHERE (type = 'm.room.member'::text);
 CREATE INDEX current_state_events_members_room_index ON current_state_events USING btree (room_id, membership) WHERE (type = 'm.room.member'::text);
 CREATE INDEX current_state_events_stream_ordering_idx ON current_state_events USING btree (event_stream_ordering);
 CREATE UNIQUE INDEX delayed_events_idx ON delayed_events USING btree (delay_id);
 CREATE INDEX delayed_events_is_processed ON delayed_events USING btree (is_processed);
 CREATE INDEX delayed_events_room_state_event_idx ON delayed_events USING btree (room_id, event_type, state_key) WHERE (state_key IS NOT NULL);
 CREATE INDEX delayed_events_send_ts ON delayed_events USING btree (send_ts);
 CREATE INDEX deleted_pushers_stream_id ON deleted_pushers USING btree (stream_id);
 CREATE INDEX destination_rooms_room_id ON destination_rooms USING btree (room_id);
+CREATE INDEX destination_rooms_sticky_events_backlog_room_id ON destination_rooms_sticky_events_backlog USING btree (room_id);
 CREATE INDEX device_auth_providers_devices ON device_auth_providers USING btree (user_id, device_id);
 CREATE INDEX device_auth_providers_sessions ON device_auth_providers USING btree (auth_provider_id, auth_provider_session_id);
 CREATE INDEX device_federation_inbox_received_ts_index ON device_federation_inbox USING btree (received_ts);
 CREATE INDEX device_federation_inbox_sender_id ON device_federation_inbox USING btree (origin, message_id);
 CREATE INDEX device_federation_outbox_destination_id ON device_federation_outbox USING btree (destination, stream_id);
 CREATE INDEX device_federation_outbox_id ON device_federation_outbox USING btree (stream_id);
 CREATE INDEX device_inbox_stream_id_user_id ON device_inbox USING btree (stream_id, user_id);
 CREATE INDEX device_inbox_user_stream_id ON device_inbox USING btree (user_id, device_id, stream_id);
 CREATE INDEX device_lists_changes_in_room_by_room_idx ON device_lists_changes_in_room USING btree (room_id, stream_id);
 CREATE INDEX device_lists_changes_in_room_inserted_ts_idx ON device_lists_changes_in_room USING btree (inserted_ts) WHERE (inserted_ts IS NOT NULL);
@@ -1850,20 +1858,22 @@
 CREATE TRIGGER check_event_stream_ordering BEFORE INSERT OR UPDATE ON room_memberships FOR EACH ROW EXECUTE FUNCTION check_event_stream_ordering();
 CREATE TRIGGER check_partial_state_events BEFORE INSERT OR UPDATE ON partial_state_events FOR EACH ROW EXECUTE FUNCTION check_partial_state_events();
 CREATE TRIGGER delete_read_write_lock_parent_trigger AFTER DELETE ON worker_read_write_locks FOR EACH ROW EXECUTE FUNCTION delete_read_write_lock_parent();
 CREATE TRIGGER upsert_read_write_lock_parent_trigger BEFORE INSERT ON worker_read_write_locks FOR EACH ROW EXECUTE FUNCTION upsert_read_write_lock_parent();
 ALTER TABLE ONLY access_tokens
     ADD CONSTRAINT access_tokens_refresh_token_id_fkey FOREIGN KEY (refresh_token_id) REFERENCES refresh_tokens(id) ON DELETE CASCADE;
 ALTER TABLE ONLY destination_rooms
     ADD CONSTRAINT destination_rooms_destination_fkey FOREIGN KEY (destination) REFERENCES destinations(destination);
 ALTER TABLE ONLY destination_rooms
     ADD CONSTRAINT destination_rooms_room_id_fkey FOREIGN KEY (room_id) REFERENCES rooms(room_id);
+ALTER TABLE ONLY destination_rooms_sticky_events_backlog
+    ADD CONSTRAINT destination_rooms_sticky_events_backlog_room_id_fkey FOREIGN KEY (room_id) REFERENCES rooms(room_id) ON DELETE CASCADE;
 ALTER TABLE ONLY event_edges
     ADD CONSTRAINT event_edges_event_id_fkey FOREIGN KEY (event_id) REFERENCES events(event_id);
 ALTER TABLE ONLY event_failed_pull_attempts
     ADD CONSTRAINT event_failed_pull_attempts_room_id_fkey FOREIGN KEY (room_id) REFERENCES rooms(room_id);
 ALTER TABLE ONLY event_forward_extremities
     ADD CONSTRAINT event_forward_extremities_event_id FOREIGN KEY (event_id) REFERENCES events(event_id) DEFERRABLE INITIALLY DEFERRED;
 ALTER TABLE ONLY current_state_events
     ADD CONSTRAINT event_stream_ordering_fkey FOREIGN KEY (event_stream_ordering) REFERENCES events(stream_ordering) NOT VALID;
 ALTER TABLE ONLY local_current_membership
     ADD CONSTRAINT event_stream_ordering_fkey FOREIGN KEY (event_stream_ordering) REFERENCES events(stream_ordering) NOT VALID;

state (sqlite)

Unchanged

state (postgres)

Unchanged