macltok

public

Star0

Fork 0

Watch0

65 commits 3 branches 0 tags 0 releases3.0 MiB

Commit `7c7832147a2e`

tarball

atagen <boss@atagen.co>2026-10-25 09:12 UTC

1 file changed+69-44

docs: bring mentions discovery in line with the inbox

The reply rule accepts an explicit mention without a highlight, and the watch snapshot is replaced before the hook acks. Also records the split checkpoint record, the failure gaps and British spellings.

Modified file: docs/mentions-discovery.md+69 additions−44 deletions@@ -48,8 +48,9 @@ The default one-event initial window cannot recover all earlier pings, including
 messages sent while the client was offline. Raising
 `SyncServiceBuilder::with_room_list_timeline_limit` increases the initial
 window but does not establish continuous history. The response hook processes
-joined-room events and records limited timelines, initial room coverage,
-checkpoint discontinuities, and recovery gaps.
+joined-room events and records limited timelines of rooms it had already
+observed, checkpoint discontinuities, and recovery gaps. A room seen for the
+first time adds no gap, since every room starts that way.
 
 ## Observation and durable commit
 
@@ -84,14 +85,27 @@ a handler and placing work on a queue without awaiting its commit does not
 close the crash window.
 
 macltok uses shared `pos` through its SDK service and commits each response
-batch through one serialized mentions actor before the hook acknowledges it.
-The actor records joined-room events, raw encrypted candidates, limited
-timeline gaps, and the connection checkpoint in the same inbox revision.
-Stable room and event keys make replay idempotent. Context capture, mirror
-publication, and event-stream delivery continue after the commit. The classic
+batch through one serialised mentions actor before the hook acknowledges it.
+The actor records events from joined rooms and from rooms the same response
+leaves, raw encrypted candidates, limited timeline gaps, and the connection
+checkpoint in one store commit. Stable room and event keys make replay
+idempotent. The mentions watch snapshot is replaced before the ack, while
+mirror and event-stream delivery and context capture run after it. The classic
 sync `next_batch` ordering differs and does not establish this guarantee.
 Nor does this ordering make arbitrary concurrent sync loops safe.
 
+The checkpoint keeps the position the committed response started from as well
+as the one it ended at. A crash after the commit but before the SDK saves `pos`
+replays from that earlier position, and the replay is a superset of what was
+committed, so it is not treated as a discontinuity. Any other starting position
+records a `SyncDiscontinuity` gap and marks every observed room unresolved.
+
+The hook never fails sync. A failed write keeps the inbox in memory, records a
+`StoreFailed` gap, acks, and retries the write a minute later, so a restart
+before a write lands shows up as a discontinuity. The hook also stops waiting
+after ten seconds and records `HookTimedOut`, so a stuck actor cannot hold the
+position lock.
+
 Sources are `crates/matrix-sdk/src/event_handler/mod.rs`,
 `crates/matrix-sdk/src/event_handler/static_events.rs`,
 `crates/matrix-sdk/src/sliding_sync/client.rs`,
@@ -114,13 +128,13 @@ acknowledgement returns a sync error without advancing the position.
 
 The hook runs outside SDK store locks and supports local futures on wasm and
 sendable futures natively. The default absent hook preserves existing SDK
-behavior. Detached response processing observes a dropped store
+behaviour. Detached response processing observes a dropped store
 acknowledgement after its caller is cancelled. Checkpoint replay works after
 SDK room state was saved, and stable event keys deduplicate repeated commits.
 
 The SDK audit and focused validation covered commit ordering, failed and
 dropped acknowledgements, caller cancellation, replay after room-state
-persistence, restoring a previous checkpoint, absent-hook behavior, builder
+persistence, restoring a previous checkpoint, absent-hook behaviour, builder
 forwarding, and connection isolation. The SDK UI library also received a
 focused wasm compilation check with its `js` feature. The SDK changes are
 published to the user's fork under `mentions-commit-hook`, and the manifest
@@ -153,9 +167,11 @@ store-free client uses memory. Even with persistence, the asynchronous cache
 consumer is not awaited before Sliding Sync saves its position.
 
 The inbox retains unresolved encrypted candidates as encrypted raw events
-within a configurable bound. It retries due candidates at startup and resets
-their retry time when decryption reports arrive. Candidate eviction sets the
-coverage truncation flag. Imported keys and backups cannot reconstruct events
+within a configurable bound. It retries due candidates at startup. A report of
+resolved events retries the encrypted candidates of that room, and lag or a
+new backup retries all of them. Candidates still unresolved a week after they
+were first seen expire behind a `CandidateExpired` gap. Over the bound the
+oldest events go first, and eviction sets the coverage truncation flag. Imported keys and backups cannot reconstruct events
 already discarded by every cache.
 
 Sources are `crates/matrix-sdk/src/event_cache/mod.rs`,
@@ -168,19 +184,19 @@ Sources are `crates/matrix-sdk/src/event_cache/mod.rs`,
 Structured `m.mentions.user_ids` and `m.mentions.room` supply semantic
 candidates. The classifier does not infer pings from rendered bodies or
 unread counts. A reply relationship alone does not prove that a reply notifies
-the user, so reply reasons require notification metadata and confirmation that
-the referenced event belongs to the signed-in user.
+the user, so reply reasons require confirmation that the referenced event
+belongs to the signed-in user plus either an explicit user mention or a
+notifying highlight.
 
 `Room::event_push_actions` evaluates push rules but does not independently
-filter ignored senders. The classifier calls `Client::is_user_ignored` and
-rechecks entries when the ignore list changes.
+filter ignored senders. The service reads the ignore list once per response
+and again when it changes, and rechecks entries then.
 
 A room mute can suppress evaluated notification actions while structured
 mention metadata remains present. The classifier retains explicit user
-mentions in muted rooms and marks notification delivery as suppressed. Reply
-reasons require both a reply to the user and a notifying highlight. Room-wide
+mentions in muted rooms and marks notification delivery as suppressed. Room-wide
 reasons require the relevant push conditions and sender power level. Missing
-authorization context remains unresolved for reevaluation. Keyword-only
+authorisation context remains unresolved for reevaluation. Keyword-only
 notifications are outside the inbox. The classifier excludes the signed-in
 user's own events even when they contain explicit mention metadata.
 
@@ -197,7 +213,7 @@ Ruma exposes
 It accepts opaque `from`, optional `limit`, and optional `only` fields.
 Responses contain notifications and an optional `next_token`. macltok requests
 50 notifications per page. Entries include raw events, room IDs, timestamps,
-actions, and server notification read state, which does not initialize or
+actions, and server notification read state, which does not initialise or
 replace local inbox read state.
 
 The service uses this endpoint as a bounded candidate source, validates
@@ -207,7 +223,7 @@ including unresolved encrypted candidates. A failed commit leaves the prior
 cursor available for replay.
 
 An exhausted notification list does not mean complete room history. Muted-room
-pings, encrypted mentions not recognized by the server, expired notifications,
+pings, encrypted mentions not recognised by the server, expired notifications,
 and server-specific retention can be absent. Recovery inspects at most 20 pages
 of 50 notifications per run and resumes from committed cursors later. The
 older-history cursor is separate from the cursor for scans from the newest
@@ -224,8 +240,7 @@ remain visible when continuity cannot be established.
 Coverage exposes the earliest observed timestamp, history-endpoint support,
 recovery activity, known room gaps, unresolved candidate count, and retention
 truncation. The earliest timestamp is a summary, not proof that every room is
-covered after that time. The inbox keeps its incomplete-history flag even
-after notification pagination ends.
+covered after that time. Each gap carries a typed `MentionGapReason`.
 
 Complete offline mention recovery across arbitrary muted and encrypted rooms
 is not supplied by this endpoint or by the current sync configuration. It
@@ -242,10 +257,12 @@ inbox commits and discovery checkpoints.
 
 The passphrase wraps a random SDK `StoreCipher`. Logical keys are hashed, and
 authenticated encrypted records include schema, account identity, and logical
-key. Restore rejects mismatched accounts, unsupported schemas, corrupt records,
-and inconsistent record counts or byte totals. Atomic transactions compare the
-previous encrypted header before publishing a new revision, so stale writers
-cannot overwrite a newer checkpoint.
+key. Restore treats mismatched accounts, unsupported schemas, corrupt records,
+and inconsistent record counts or byte totals as unreadable, empties the store
+and records a `StoreReset` gap rather than failing `Account::build`. A store
+that cannot be opened at all leaves the inbox in memory behind `StoreFailed`.
+Atomic transactions compare the previous encrypted header before publishing a
+new revision, so stale writers cannot overwrite a newer checkpoint.
 
 The default budget is 64 MiB of encrypted records, hashed keys, header, and
 wrapped cipher, excluding database page overhead. The store record cap is
@@ -255,22 +272,29 @@ limit requires eviction, the store drops bounded data and marks coverage
 truncated.
 
 The storage tests cover atomic restore of records and checkpoints, stale
-writers, capacity failure, encrypted physical records, wrong credentials or
-account, corruption, unsupported schemas, and injected SQLite write failure.
-The inbox tests cover batch failure, notification recovery, live context
-updates, and capture persistence across session resume.
-
-
-The inbox is serialized into one `inbox` value and encrypted as a single
-record on every revision. Each update reserializes and re-encrypts the retained
-inbox, so write cost scales with retained data. Decrypted previews, saved
-context, raw candidates, pending capture state, media source registrations,
-read state, coverage, and sync checkpoints stay inside the authenticated
-encrypted payload. SDK SQLite custom values remain plaintext and are not used
+writers, capacity failure, encrypted physical records, resets on wrong
+credentials or account, corruption and unsupported schemas, and injected
+SQLite write failure. The inbox tests cover failed writes, checkpoint-only
+writes, replay from the previous checkpoint, reset of an unreadable inbox,
+notification recovery, live context updates, and capture persistence across
+session resume.
+
+The store holds two records. `inbox` carries entries, saved events, context
+windows, candidates, edits, redactions, coverage and recovery cursors. `sync`
+carries the connection checkpoints, observed rooms, per-room repair coverage
+and the inbox revision it belongs to. Every response writes `sync`, and
+`inbox` only when the response changed it, both in one commit. A quiet
+response therefore costs one small record, while a changed inbox is still
+reserialised and re-encrypted whole. Background completions are applied in
+memory and written together at most once a second. When the budget is short,
+eviction runs against a size estimate and the inbox is written once.
+Decrypted previews, saved context, raw candidates, pending capture state,
+media source registrations, read state, coverage, and sync checkpoints stay
+inside the authenticated encrypted payloads. SDK SQLite custom values remain plaintext and are not used
 for decrypted mentions.
 
 Context capture in `src/engine/context.rs` sizes the before and after requests
-independently and retains the media registrations produced by normalization.
+independently and retains the media registrations produced by normalisation.
 Each capture requests up to 10 events on each side. At most two capture jobs
 run concurrently, with up to 128 pending or failed entries considered for
 scheduling at a time. Capture uses independent `/context` and `/messages`
@@ -278,7 +302,7 @@ requests, so it does not paginate the live cache. Media source registrations
 are encrypted with saved events and registered again when context is read.
 Media bodies and key material do not enter the mentions public model.
 
-One account-scoped mentions actor serializes store commits for entries,
+One account-scoped mentions actor serialises store commits for entries,
 unresolved candidates, shared context events, read state, revisions, discovery
 checkpoints, and retention decisions.
 
@@ -308,7 +332,7 @@ that boundary, including older messages resolved by late decryption.
 ## Executable probes
 
 `tests/mentions_discovery.rs` exercises unopened-room update and cache access,
-push-rule behavior for ignored senders and muted rooms, and typed
+push-rule behaviour for ignored senders and muted rooms, and typed
 notification-history pagination.
 
 `tests/mentions_decryption.rs` exchanges a real encrypted event and a delayed
@@ -327,11 +351,12 @@ demonstrates that returning a handler error still advances the position.
 with revision boundaries, independent context limits, and persistence across
 session resume. `tests/mentions_late_decryption.rs` verifies that a delayed room key
 updates the production inbox once. `src/mentions/service_tests.rs` covers
-checkpoint rollback, notification-history recovery, and context updates.
+failed and checkpoint-only writes, checkpoint replay, store reset,
+notification-history recovery, and context updates.
 `src/mentions/store/tests.rs` exercises encrypted record restore, capacity,
 stale writers, and corruption handling.
 
-These probes establish SDK behavior against mocked responses. They do not
+These probes establish SDK behaviour against mocked responses. They do not
 establish homeserver notification retention, room-list completeness, or
 complete history on a deployed server.

macltok

Commit 7c7832147a2e

Commit `7c7832147a2e`