# Persistent mentions discovery This records the SDK audit and current persistent mentions implementation. Persistent sessions now commit discovery with Sliding Sync responses, retain an encrypted inbox, and expose it through `ClientHandle` and the mirror. Historical coverage remains incomplete because room-list sync and notification history cannot prove that every past mention was observed. The audited dependency is Matrix Rust SDK 0.18.0 at fork revision `cc2b13ea185c0f694523a3a48d8aadfe366dd767`. macltok now pins fork revision `7aace883fbe03d4c3c118b55188fe9519ba8ce48`, which adds the response commit hook described below. SDK paths below are relative to the audited revision. ## Findings The SDK can deliver and decrypt events without opening a UI timeline. macltok records observed events and recovers candidates through notification history, but neither source establishes complete historical coverage. The SDK audit found that an awaited event handler can hold back the Sliding Sync position until a macltok transaction commits, while a background broadcast consumer cannot provide that guarantee. Handlers cannot propagate commit errors, and response processing outlives cancellation of the calling sync task. These limits motivated the fallible response-commit hook now implemented in the pinned SDK fork. ## Sync coverage macltok constructs `SyncService` in `src/engine/mod.rs` and uses the SDK room list configuration. The list starts with the selective range `0..=19`. After setup it grows by 100 rooms per response, without an SDK-side maximum, until it reaches the server's room count. Recovery and sufficiently long idle periods can return it to the initial selective range before it grows again. The default list timeline limit is one event per room on initial delivery. Subsequent changes for covered rooms continue arriving through sync. Opening a room separately subscribes it with a timeline limit of 20. macltok's `update_room_subscriptions` and `watchers::room_subscriptions` manage those explicit subscriptions for active room actors only. The mentions response hook filters entries to joined rooms. The built-in list still includes invited rooms. Sources are `crates/matrix-sdk-ui/src/room_list_service/mod.rs`, `crates/matrix-sdk-ui/src/room_list_service/state.rs`, and `crates/matrix-sdk/src/sliding_sync/list/request_generator.rs`. The default one-event initial window cannot recover all earlier pings, including messages sent while the client was offline. Raising `SyncServiceBuilder::with_room_list_timeline_limit` increases the initial window but does not establish continuous history. The response hook processes joined-room events and records limited timelines, initial room coverage, checkpoint discontinuities, and recovery gaps. ## Observation and durable commit `Client::subscribe_to_all_room_updates` returns a bounded broadcast receiver of `RoomUpdates`. Its `joined` map includes processed timeline events with raw decrypted content when available and optional evaluated push actions. It does not require a UI timeline. Lag is an error that requires recovery, not an invitation to skip to the next update. `Client::observe_events` retains only the latest observation and can miss intermediate events. It is unsuitable for durable discovery. An earlier probe registers `Client::add_event_handler` for `Raw` with `Room` context before sync starts. Its awaited handler demonstrates event ordering, but it cannot report commit failures reliably and is not the production ingestion path. Production uses the fallible Sliding Sync response hook described below. For Sliding Sync v5, the response processor awaits event handlers before updating and persisting `pos`. The response hook commits the inbox first, so a crash permits replay without skipping committed candidates. Stable event keys make replay idempotent. The hook acknowledgement does not wait for context capture, a UI subscriber, or the engine event stream to drain. For ordinary event handlers, the SDK logs and discards handler errors, so returning an error after a failed write still permits `pos` to advance. A pending handler alone also does not provide a reliable shutdown path. The SDK runs response handling in a detached task that outlives cancellation of `sync_once` or its caller. Aborting the engine's sync task cannot prevent a suspended handler from later returning and advancing the position. Registering a handler and placing work on a queue without awaiting its commit does not close the crash window. macltok uses shared `pos` through its SDK service and commits each response batch through one serialized mentions actor before the hook acknowledges it. The actor records joined-room events, raw encrypted candidates, limited timeline gaps, and the connection checkpoint in the same inbox revision. Stable room and event keys make replay idempotent. Context capture, mirror publication, and event-stream delivery continue after the commit. The classic sync `next_batch` ordering differs and does not establish this guarantee. Nor does this ordering make arbitrary concurrent sync loops safe. Sources are `crates/matrix-sdk/src/event_handler/mod.rs`, `crates/matrix-sdk/src/event_handler/static_events.rs`, `crates/matrix-sdk/src/sliding_sync/client.rs`, `crates/matrix-sdk/src/sliding_sync/mod.rs`, and `crates/matrix-sdk/src/sliding_sync/cache.rs`. ### SDK commit hook implementation The local SDK workspace at `../matrix-rust-sdk-mentions` implements the fallible asynchronous response-commit hook through `ResponseCommitHook` and `ResponseCommitBatch`. `SyncServiceBuilder` forwards it only to its room-list connection, so independent notification lookups are unaffected. The hook receives the processed room events, original room payloads, limited flags, pagination tokens, connection identity, and previous and next position identifiers. It runs before response handlers and observers and before `pos` is updated or saved. SDK room state may already have been saved. A successful acknowledgement follows the macltok inbox transaction. A failed or dropped acknowledgement returns a sync error without advancing the position. The hook runs outside SDK store locks and supports local futures on wasm and sendable futures natively. The default absent hook preserves existing SDK behavior. Detached response processing observes a dropped store acknowledgement after its caller is cancelled. Checkpoint replay works after SDK room state was saved, and stable event keys deduplicate repeated commits. The SDK audit and focused validation covered commit ordering, failed and dropped acknowledgements, caller cancellation, replay after room-state persistence, restoring a previous checkpoint, absent-hook behavior, builder forwarding, and connection isolation. The SDK UI library also received a focused wasm compilation check with its `js` feature. The SDK changes are published to the user's fork under `mentions-commit-hook`, and the manifest and both macltok lockfiles pin revision `7aace883fbe03d4c3c118b55188fe9519ba8ce48`. ## Event cache and delayed decryption `EventCache::subscribe` starts global room-update ingestion. The SDK room list service already calls it during construction. Its generic update subscription identifies the affected room, then `Room::event_cache` and `RoomEventCache::find_event` can retrieve an event. This does not require a `matrix-sdk-ui::Timeline` or a retained subscriber per room. `EventCache::subscribe_to_decryption_reports` reports resolved event IDs, lag, and backup availability. `EventCache::request_decryption` accepts room and Megolm session IDs for explicit retries. `Room::decrypt_event` is also public and can decrypt a retained raw event independently of a UI timeline. The late-key probe confirms automatic decryption and retrieval of a structured mention through the global cache without a UI timeline. Cached push actions are optional. Re-evaluate them with `Room::event_push_actions` when needed, and treat missing push context as unresolved rather than a negative result. The room's own membership state is needed to build that push context. The cache is not a durable ingestion journal. Its room broadcasts and generic updates can lag, and cache retention can remove events. SQLite and IndexedDB builders select persistent SDK event-cache stores, while the default store-free client uses memory. Even with persistence, the asynchronous cache consumer is not awaited before Sliding Sync saves its position. The inbox retains unresolved encrypted candidates as encrypted raw events within a configurable bound. It retries due candidates at startup and resets their retry time when decryption reports arrive. Candidate eviction sets the coverage truncation flag. Imported keys and backups cannot reconstruct events already discarded by every cache. Sources are `crates/matrix-sdk/src/event_cache/mod.rs`, `crates/matrix-sdk/src/event_cache/redecryptor.rs`, `crates/matrix-sdk/src/event_cache/caches/room/mod.rs`, and `crates/matrix-sdk/src/room/mod.rs`. ## Mention policy Structured `m.mentions.user_ids` and `m.mentions.room` supply semantic candidates. The classifier does not infer pings from rendered bodies or unread counts. A reply relationship alone does not prove that a reply notifies the user, so reply reasons require notification metadata and confirmation that the referenced event belongs to the signed-in user. `Room::event_push_actions` evaluates push rules but does not independently filter ignored senders. The classifier calls `Client::is_user_ignored` and rechecks entries when the ignore list changes. A room mute can suppress evaluated notification actions while structured mention metadata remains present. The classifier retains explicit user mentions in muted rooms and marks notification delivery as suppressed. Reply reasons require both a reply to the user and a notifying highlight. Room-wide reasons require the relevant push conditions and sender power level. Missing authorization context remains unresolved for reevaluation. Keyword-only notifications are outside the inbox. The classifier excludes the signed-in user's own events even when they contain explicit mention metadata. The existing `NotificationClient` combines notification evaluation with ignored-sender filtering. Its public lookup takes known room and event IDs. It is useful for resolving a candidate, but filtered results erase the distinction between mute policy and an absent semantic mention. It is not the sole classifier for this inbox. ## Historical candidates and gaps Ruma exposes `api::client::push::get_notifications::v3::Request`, sent with `Client::send`. It accepts opaque `from`, optional `limit`, and optional `only` fields. Responses contain notifications and an optional `next_token`. macltok requests 50 notifications per page. Entries include raw events, room IDs, timestamps, actions, and server notification read state, which does not initialize or replace local inbox read state. The service uses this endpoint as a bounded candidate source, validates events after decryption, and applies the live mention policy. Each page's observed candidates and cursor are committed together in the encrypted inbox, including unresolved encrypted candidates. A failed commit leaves the prior cursor available for replay. An exhausted notification list does not mean complete room history. Muted-room pings, encrypted mentions not recognized by the server, expired notifications, and server-specific retention can be absent. Recovery inspects at most 20 pages of 50 notifications per run and resumes from committed cursors later. The older-history cursor is separate from the cursor for scans from the newest notifications. A newest scan records an anchor and runs at startup and every 60 seconds. It follows the committed cursor until it reaches that anchor, resuming on later runs when needed to find newly added notifications. SDK cache gaps retain pagination tokens and support room pagination, but running that pagination mutates the live cache. Mention context capture uses independent `/context` and `/messages` requests. Tokens may expire. Context can be retried around a known event while the room remains joined, and known gaps remain visible when continuity cannot be established. Coverage exposes the earliest observed timestamp, history-endpoint support, recovery activity, known room gaps, unresolved candidate count, and retention truncation. The earliest timestamp is a summary, not proof that every room is covered after that time. The inbox keeps its incomplete-history flag even after notification pagination ends. Complete offline mention recovery across arbitrary muted and encrypted rooms is not supplied by this endpoint or by the current sync configuration. It would require a retained, replayable event source or bounded per-room history recovery with proof of continuity. The notification endpoint does not provide that guarantee. ## Storage, context, and API `src/mentions/store` implements encrypted SQLite and IndexedDB record stores. Persistent session construction opens the account store, and local logout deletes it alongside the SDK stores. The mentions actor owns the store for inbox commits and discovery checkpoints. The passphrase wraps a random SDK `StoreCipher`. Logical keys are hashed, and authenticated encrypted records include schema, account identity, and logical key. Restore rejects mismatched accounts, unsupported schemas, corrupt records, and inconsistent record counts or byte totals. Atomic transactions compare the previous encrypted header before publishing a new revision, so stale writers cannot overwrite a newer checkpoint. The default budget is 64 MiB of encrypted records, hashed keys, header, and wrapped cipher, excluding database page overhead. The store record cap is 35,128. The inbox defaults to 5,000 entries, 25,000 shared context events, and 5,000 unresolved candidates within that byte budget. When a size or record limit requires eviction, the store drops bounded data and marks coverage truncated. The storage tests cover atomic restore of records and checkpoints, stale writers, capacity failure, encrypted physical records, wrong credentials or account, corruption, unsupported schemas, and injected SQLite write failure. The inbox tests cover batch failure, notification recovery, live context updates, and capture persistence across session resume. The inbox is serialized into one `inbox` value and encrypted as a single record on every revision. Each update reserializes and re-encrypts the retained inbox, so write cost scales with retained data. Decrypted previews, saved context, raw candidates, pending capture state, media source registrations, read state, coverage, and sync checkpoints stay inside the authenticated encrypted payload. SDK SQLite custom values remain plaintext and are not used for decrypted mentions. Context capture in `src/engine/context.rs` sizes the before and after requests independently and retains the media registrations produced by normalization. Each capture requests up to 10 events on each side. At most two capture jobs run concurrently, with up to 128 pending or failed entries considered for scheduling at a time. Capture uses independent `/context` and `/messages` requests, so it does not paginate the live cache. Media source registrations are encrypted with saved events and registered again when context is read. Media bodies and key material do not enter the mentions public model. One account-scoped mentions actor serializes store commits for entries, unresolved candidates, shared context events, read state, revisions, discovery checkpoints, and retention decisions. Pending capture state is durable with the inbox, so interrupted capture can resume after startup. When a room leaves the joined state, saved context remains readable and its capture state becomes unavailable. Refetch is disabled after departure. Store publication follows commit. `ClientHandle` exposes page, context, retry, read, read-through, and recovery methods. The public `Command`, mention models, mirror state, and updates support serde when the `serde` feature is enabled, so native and browser consumers can transport them. Browser worker integration is a consumer phase after SDK and macltok revision publication. Malice's worker still needs mention variants in its explicit `Request` dispatch and TypeScript methods. Mirror snapshots contain at most 50 entry previews, coverage, unread count, and a committed revision boundary, with no saved context bodies. The watcher coalesces updates and computes upserts and removals relative to the previous bounded snapshot it observed. A missed update therefore leaves the latest snapshot authoritative. A removal can mean an entry fell out of the top 50, and does not imply that it was deleted from the inbox. Pagination uses timestamp and stable event key within a discovery revision boundary. Read-through boundaries exclude mentions first discovered or reactivated after that boundary, including older messages resolved by late decryption. ## Executable probes `tests/mentions_discovery.rs` exercises unopened-room update and cache access, push-rule behavior for ignored senders and muted rooms, and typed notification-history pagination. `tests/mentions_decryption.rs` exchanges a real encrypted event and a delayed room key between two mock clients, then retrieves the decrypted mention without opening a UI timeline. `tests/mentions_checkpoint.rs` holds an event handler pending and verifies that the SDK has not persisted its position. It cancels the calling task, then acknowledges the handler and verifies that detached response processing still saves the position. A new Sliding Sync instance restores that position. This is the historical event-handler ordering probe. It does not exercise the production response-commit hook or the durable inbox transaction. It also demonstrates that returning a handler error still advances the position. `tests/mentions_inbox.rs` covers live mention classification, read-through with revision boundaries, independent context limits, and persistence across session resume. `tests/mentions_late_decryption.rs` verifies that a delayed room key updates the production inbox once. `src/mentions/service_tests.rs` covers checkpoint rollback, notification-history recovery, and context updates. `src/mentions/store/tests.rs` exercises encrypted record restore, capacity, stale writers, and corruption handling. These probes establish SDK behavior against mocked responses. They do not establish homeserver notification retention, room-list completeness, or complete history on a deployed server. Focused validation passed 63 native tests and 12 shared inbox and IndexedDB tests in headless Firefox. Compilation checks also pass without storage features and for wasm with IndexedDB and mirror support. The full flake gate was skipped by request.