Add experimental support for MSC4308: Thread Subscriptions extension to Sliding Sync when MSC4306 and MSC4186 are enabled. #18695

reivilibre · 2025-07-18T10:50:51Z

Implements: matrix-org/matrix-spec-proposals#4308

Follows: #18674

This pull request is intended for commit-by-commit review.

This pull request adds an extension to Sliding Sync and a companion endpoint needed for backpaginating missed thread subscription changes,
as described in MSC4308

MadLittleMods

(I have looked things over but haven't given everything a proper review-level scrutiny)

synapse/handlers/sliding_sync/extensions.py

…_for_user`

Works around #18712

Put MSC4308 behind the MSC4306 feature flag

Signed-off-by: Olivier 'reivilibre <[email protected]>

reivilibre · 2025-08-21T16:29:14Z

previously-reviewed draft was very different, so putting back on the main queue

anoadragon453

Looks good overall! A few minor things below.

Also curious that the CI was cancelled. I assume that was on purpose?

synapse/types/handlers/sliding_sync.py

synapse/streams/events.py

anoadragon453 · 2025-08-27T09:43:29Z

synapse/storage/util/id_generators.py

+        When creating a new sequence for a new stream, it will be necessary to advance it
+        so that position 1 is consumed.
+        DO NOT USE `START WITH 2` FOR THIS PURPOSE:
+            see https://github.com/element-hq/synapse/issues/18712
+        Instead, use `SELECT nextval('sequence_name');` immediately after the
+        `CREATE SEQUENCE` statement.


Should we close #18712 once this PR is merged?
Unless we plan to fix it in some other fashion down the line.

Hrmrmrmr. It feels like a design footgun that we could ideally resolve rather than having a hidden comment somewhere about what to do. But not sure if that's worth keeping an issue in the infinite backlog.

IMO there's not too much benefit gained from properly fixing it vs. other things we could work on. So in a purely triage sense, I'd be in favour of closing the issue and leaving the comment as a helpful touch stone (one that may prompt folks to fix it in future if they really wanted to).

Somewhat agreed, but the comment is not necessarily in a place you will notice (I'm not sure where you could do that, actually). The risk is that next time we come to adding a new one, we lose another couple of hours trying to track this issue down. But mrm I suppose this is probably the best we can do atm

synapse/handlers/sliding_sync/extensions.py

synapse/types/__init__.py

tests/rest/client/sliding_sync/test_extension_thread_subscriptions.py

anoadragon453 · 2025-08-27T11:11:55Z

synapse/rest/client/thread_subscriptions.py

+        if limit <= 0:
+            raise SynapseError(
+                HTTPStatus.BAD_REQUEST,
+                "limit must be greater than 0",
+                errcode=Codes.INVALID_PARAM,
+            )


negative=False on parse_integer will already do this for us.

negative still allows 0 :(

Ah, that's annoying.

Could you add a quick comment noting that this conditional is needed because of that?

synapse/rest/client/thread_subscriptions.py

tests/rest/client/sliding_sync/test_extension_thread_subscriptions.py

…ions.py Co-authored-by: Andrew Morgan <[email protected]>

Co-authored-by: Andrew Morgan <[email protected]>

anoadragon453 · 2025-09-08T13:36:54Z

synapse/rest/client/thread_subscriptions.py

@@ -100,6 +118,129 @@ async def on_DELETE(
        return HTTPStatus.OK, {}


+class ThreadSubscriptionsPaginationRestServlet(RestServlet):


What do you think about adding rate-limiting to this handler?

hmm... do we rate-limit any read operations already?

Just looking at https://spec.matrix.org/v1.15/client-server-api/#get_matrixclientv3roomsroomidmessages and sync and I'm not seeing them be rate limited.

Not that it's a terrible idea, but not sure how to balance with consistency (and the fact you could rate limit general requests with a load balancer e.g. nginx for instance)

I mostly asked the question to check that it had been considered (which we tend to forget to do when adding new endpoints).

I'll accept the argument that the endpoint can be rate-limited by IP or access token in the load balancer for now, as I agree that we don't need anything more granular. We also have a MAX_LIMIT value defined, which prevents any one request from drawing too many resources.

anoadragon453

This LGTM!

anoadragon453 · 2025-09-09T10:35:15Z

(I assume Complement CI will fail until #18819 is merged?)

reivilibre force-pushed the rei/ssext_threadsubs branch 3 times, most recently from 7634587 to ba41526 Compare July 21, 2025 17:32

reivilibre marked this pull request as ready for review July 21, 2025 20:13

reivilibre requested a review from a team as a code owner July 21, 2025 20:13

MadLittleMods added A-Sync A-Threads labels Jul 21, 2025

MadLittleMods reviewed Jul 22, 2025

View reviewed changes

synapse/handlers/sliding_sync/extensions.py Outdated Show resolved Hide resolved

reivilibre requested a review from MadLittleMods July 23, 2025 10:25

MadLittleMods removed their request for review July 23, 2025 15:50

MadLittleMods mentioned this pull request Aug 11, 2025

MSC4306: Thread Subscriptions matrix-org/matrix-spec-proposals#4306

Open

reivilibre force-pushed the rei/ssext_threadsubs branch from ba41526 to 6165651 Compare August 20, 2025 14:41

reivilibre added 12 commits August 21, 2025 12:39

Update simplified sliding sync docstring

2961006

spelling

875dbf7

Add models for Thread Subscriptions extension to Sliding Sync

09f8633

Add overload for gather_optional_coroutines/6

f1f5657

Add thread subscriptions position to StreamToken

748316c

Add subscribed and automatic to `get_updated_thread_subscriptions…

4dcd12b

…_for_user`

Fix thread_subscriptions stream sequence

0ce5dce

Works around #18712

Add comment to MultiWriterIdGenerator about cursed sequence semantics

0c310b9

Add overload for parse_integer_from_args

18881b1

Implement sliding sync extension part of MSC4308

4a34641

Put MSC4308 behind the MSC4306 feature flag

Add companion endpoint for backpagination of thread subscriptions

e72d6cd

Newsfile

f4cd180

Signed-off-by: Olivier 'reivilibre <[email protected]>

reivilibre force-pushed the rei/ssext_threadsubs branch from 6165651 to f4cd180 Compare August 21, 2025 16:28

reivilibre requested a review from a team August 21, 2025 16:28

anoadragon453 requested changes Aug 27, 2025

View reviewed changes

bnjbvr mentioned this pull request Sep 1, 2025

client-api: add support for msc4308 sliding sync extension ruma/ruma#2206

Merged

bnjbvr mentioned this pull request Sep 1, 2025

feat: implement msc4308 + automatic background catchup of thread subscriptions matrix-org/matrix-rust-sdk#5604

Merged

reivilibre and others added 4 commits September 2, 2025 10:23

Update tests/rest/client/sliding_sync/test_extension_thread_subscript…

921cd53

…ions.py Co-authored-by: Andrew Morgan <[email protected]>

Update synapse/handlers/sliding_sync/extensions.py

168b67b

Co-authored-by: Andrew Morgan <[email protected]>

Update synapse/handlers/sliding_sync/extensions.py

1374895

Co-authored-by: Andrew Morgan <[email protected]>

Add notifier hooks for sliding sync

0cf178a

bnjbvr mentioned this pull request Sep 3, 2025

fix(room service): enable the thread subscriptions extension based on server support matrix-org/matrix-rust-sdk#5618

Merged

reivilibre added 2 commits September 3, 2025 13:48

Use copy_and_replace in get_current_token_for_pagination

924c1bf

Simplify if

fa8e3b6

reivilibre requested a review from anoadragon453 September 3, 2025 13:06

anoadragon453 reviewed Sep 8, 2025

View reviewed changes

reivilibre requested a review from anoadragon453 September 9, 2025 08:26

reivilibre linked an issue Sep 9, 2025 that may be closed by this pull request

MultiWriterStreamIdGenerator: wrong behaviour when using sequences initialised with START WITH #18712

Closed

Comment on why we still check limit

80679a7

This was referenced Sep 9, 2025

[Story] Threads: Subscribing to a Thread element-hq/element-meta#2766

Open

[Prep] Threads: Design a participation model API in Matrix element-hq/element-meta#2777

Open

anoadragon453 approved these changes Sep 9, 2025

View reviewed changes

Merge branch 'develop' into rei/ssext_threadsubs

00cb14e

reivilibre merged commit ada3a3b into develop Sep 11, 2025
104 of 112 checks passed

reivilibre deleted the rei/ssext_threadsubs branch September 11, 2025 13:45

		@@ -100,6 +118,129 @@ async def on_DELETE(
		return HTTPStatus.OK, {}


		class ThreadSubscriptionsPaginationRestServlet(RestServlet):

Add experimental support for MSC4308: Thread Subscriptions extension to Sliding Sync when MSC4306 and MSC4186 are enabled. #18695

Add experimental support for MSC4308: Thread Subscriptions extension to Sliding Sync when MSC4306 and MSC4186 are enabled. #18695

Uh oh!

Conversation

reivilibre commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MadLittleMods left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

reivilibre commented Aug 21, 2025

Uh oh!

anoadragon453 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anoadragon453 Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anoadragon453 left a comment

Choose a reason for hiding this comment

Uh oh!

anoadragon453 commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

reivilibre commented Jul 18, 2025 •

edited

Loading

anoadragon453 left a comment •

edited

Loading

anoadragon453 Sep 8, 2025 •

edited

Loading

anoadragon453 commented Sep 9, 2025 •

edited

Loading