# macltok design decisions

the contract the working tree actually implements. if the code disagrees with
this, one of them is lying and it should be fixed

## two ways in, one way out

state changes are commands. `ClientHandle::send` queues one and hands back a
`Pending`, `execute` queues and waits, and there's an awaited convenience method
for every command. each command answers exactly once with
`Result<CommandOutput, Error>`. drop the `Pending` if you don't care, the
command still runs

pure reads skip the engine and ask the sdk directly through the handle. rooms,
members, power levels, account data, devices, push notification resolution,
and media bytes all live here

the event stream carries state and only state. timelines, history, typing,
rooms, connection, encryption, verification, upload progress, session. no
command results ride on it. drain it from its own task, it's bounded and the
engine waits for you

one error type for everything, `Error { kind, message, challenge, server }`.
match on `kind`. `Interrupted` means the engine stopped, the room closed, or the
task doing the work died before it could answer. `challenge` turns up with
`InteractiveAuthRequired`

`message` is ours and says what we were doing. `server` is the homeserver's
errcode and message, dug out of the source chain when the error is built, so a
host can show "invalid password" without walking the chain. `AccountError`
carries the same, plus a `kind`. neither folds it into `Display`, since anything
printing the chain would say it twice

## engine

the run loop routes and does nothing slow. every task it spawns is registered
with a context. a command task that panics costs its caller an `Interrupted`
and gets logged. a watcher or a room actor that panics takes the engine down,
emitting `Failed` then `Stopped`. natively, anyway. on wasm a panic traps the
task mid poll and nothing is caught. the rest keeps running in an undefined
state and goes silent, no `Failed` and no `Stopped`, so the host hears of it
through `on_panic`, a free function since a panic hook belongs to the whole
instance rather than to one engine

sync restarts itself with backoff after an error or an unexpected stop, and
`ConnectionChanged` shows `Offline` meanwhile. a run that drops again inside
thirty seconds counts as a failure too, or a server that answers `/versions`
but refuses sync would bounce us between online and offline forever. every return to `Online` re-arms
the send queues, since the sdk parks a room's queue on any send failure and
never unparks it. a recoverable failure while online gets the room another go
after a few seconds

logout and a rejected token both stop the engine. `run` resolves to a
`Termination` naming which of those it was, or a shutdown, or every handle
dropping, so the host never works it out from the events. `Shutdown` answers once
`Stopped` has gone out, every one of them if several were queued

`Engine::run` hands back a boxed future. the real one nests deep enough that
every crate spawning it would trip the compiler's query depth

## rooms

one actor per open room. the engine holds a slot per room, `Active` with a
sender or `Closing` with a fifo of opens and closes that arrived while it wound
down. those replay in order once the actor exits. a room you leave, or get
kicked or banned from, closes itself. it tells the engine first, and anything
already queued to it comes back to be answered or replayed, so an open racing a
leave still gets a fresh actor

however an actor ends, short of the engine stopping, its typing goes out empty
and each of its timelines sends `TimelineClosed`. so does a closed thread. a
timeline that never sent a reset never sends a close either

opening sends `TimelineReset` on the stream before it answers. opening an
already open room sends a fresh reset, so a second frontend gets a snapshot

sends, edits and replies run on the actor in order, so they reach the send
queue in the order they were asked for. redactions, reactions and read
receipts are network calls, so they run beside the actor and a slow one
doesn't queue the room behind it. closing lets them finish rather than lie

threads belong to their room's actor. commands for a thread that's still
opening wait behind the open, up to the actor's queue capacity

pagination runs one request at a time per timeline and emits `HistoryChanged`
around each. the sdk remembers where history starts, we don't, because a gappy
sync can unload it again

the actor's watchers are supervised, and a panic in any of them panics the
actor. its command work isn't, a panicking redaction just answers `Interrupted`
and gets logged

## timelines

`TimelineDiff::try_apply` refuses a diff whose indices don't fit your snapshot
and leaves the snapshot alone. `try_apply_all` does the same for a whole batch,
checking every diff against lengths alone first, since that's all a diff's fit
depends on. snapshots are `imbl::Vector`s so keeping one is a pointer copy.
stop applying, `resync`, and carry on from the next `TimelineReset`. resync
kills the old watcher before the new reset goes out, so no stale diff can land
after it

`TimelineId` tells a room from a thread by its root event

every item is an event, a day divider, the read marker or the start. an event
carries its sender with avatar and name ambiguity, and a `TimelineContent`
saying what it actually is. messages keep their msgtype, membership and profile
changes and the common room state changes get their own variants, and anything
else is `OtherState` or `Other` with its type. nothing is flattened into english
in a body field

thread receipts for a thread that isn't open go straight to the room, so
marking a thread read works the same whether its room is on screen or not

replies carry the replied-to event once it's loaded. the watcher asks the sdk
for any it hasn't got, once per event, since the sdk won't on its own

edit, redact and react take an `EventRef`, so your own message can be edited or
reacted to before the server has echoed it back. redacting an unsent one just
cancels it

a custom emote reaction keys on the image's mxc uri and carries its shortcode
(msc4027), so clients without the pack can still label it. `Reaction` says
which image it is and the shortcode any sender gave, and `toggle_reaction`
takes the shortcode to send

## thread lists

a room's thread list is its own thing, not a timeline and not the room actor's.
it needs only the room, so a room needn't be open for its list. one worker holds
every open list. `Open` sends an empty `ThreadListReset`, `Paginate` asks
`/rooms/{id}/threads` through the sdk's `Room::list_threads`, and the list
follows the room's event cache for as long as it's open. changes go out as
upserts keyed by root, which `ThreadListEntry::upsert` puts in order, newest
activity first and ties by root, so a list can't diverge the way an index diff
can

the sdk's `ThreadListService` was passed over. it never adds a thread that wasn't
on a page already loaded, never reorders, and adds one to the count for every
event the cache hands it, replays and back pagination included. here a reply
counts once, by id, when sync just brought it and it's newer than a latest
whose time we know, by the sdk's corrected timestamp. the reply a thread already
ends on only dates it, since a summary from the event cache names its latest
reply without the event. replays, reloads from the store and back pagination
never count. a reply sync brings to a thread the list hasn't got fetches its
root, four at a time, and adds it. one from anywhere else doesn't, since that
says nothing about the thread being active, and a root the server doesn't have
isn't asked for again. a redacted reply doesn't uncount, an edit doesn't change
the latest reply's preview, and a preview's own reply target comes back
`Unavailable` rather than fetched

a lagged listener can't know what it missed, so the list fetches its first page
again and sends a `ThreadListReset` with it. leaving the room, or being kicked
or banned, closes the list with `ThreadListClosed` naming why, the same as
`Close` does. a list's media belongs to its own scope, bounded like an event
context's so a long open list sheds the previews its threads moved past, and
released on close

## warming up to send

the sdk does all its key work on the send path, in the room's send queue before
the `/send` itself. members synced, a `/keys/query` for anyone untracked or
dirty, then a `/keys/claim` and the room key sent to every device. over a slow
federation that's a minute of nothing. `prepare_to_send` runs all of it ahead
of time through the fork's `Room::prepare_encryption`, the same function the
send queue calls. the share sits behind the sdk's per-room dedup, so a send
landing mid-warm-up waits for it rather than sharing again, and a send after it
finds nothing left to do but the `/send`. a room whose encryption isn't known
yet gets asked, which the send would have done anyway. one warm-up per room at
a time, a second ask joins it. opening a room or starting to type runs one too,
unless `EngineConfig::prepare_to_send` is off. it's on because malice asked,
not because anyone decided

the session rotates after 100 messages or a week, and sharing isn't a message,
so a warm-up is only wasted on a room left idle about that long

## mutations

membership, moderation, power levels, room creation and account data writes go
through one keyed fifo scheduler. same room, same alias, same dm partner or
same account data type run in order, everything else runs at once.
`create_direct` hands back the existing dm when there is one

notifications and encryption each get one serial worker. the encryption worker
owns the verification controller outright

power level changes build one state event from the local mapping, since the
sdk's own patcher is private

## media

media is a read. `ClientHandle::media` takes a `MediaId`, from a retained
timeline or an avatar, through one bounded
download path with the byte cap the sdk's own download lacks, an hour's total
deadline instead of the sdk client's thirty seconds, and a minute's patience
for a stalled stream. concurrent asks for the same bytes share one flight.
a thumbnail the server won't make falls back to the file itself.
`MediaId` parses from a string, since a web client has to round trip it

the cache holds plaintext, keyed like the sdk keys it, so the sdk and macltok
can read each other's entries. that's how your own encrypted uploads come back.
the sdk keys plain and encrypted sources alike, so a plain event pointing at
someone's encrypted file can leave ciphertext in the cache. an encrypted read
checks the cached bytes against the file's hash and decrypts them if they match

media ids are registered as timelines render and forgotten when their timeline
closes, least recently seen first once the cap is hit

avatars are `MediaId`s too, so a host never handles an mxc uri. they aren't
registered, since they live in the room list and member lists for the whole
session and there can be thousands, and the cap would evict ones still on
screen. a plain avatar carries no keys, so its id is its mxc uri and resolves
without the registry. it never expires, and an mxc uri a host got some other way
parses into one. `set_avatar` still takes an mxc uri

streaming decryption inside macltok was declined. in a browser the service
worker answering range requests is a separate context from the worker running
macltok, and streaming through macltok would carry every chunk through wasm
memory, the very thing the host streams to avoid. `media_source` hands over the
url and keys `media` would have used, picked the same way, and `access_token`
the rest. the byte cap doesn't apply, nothing is buffered

uploads are commands that answer once the send queue has them. the fork puts
upload progress on the client-wide send queue channel too, which upstream keeps
for each room's own, so one watcher there follows every attachment in the queue,
leftovers the sdk resumes from last time included, and ends each with one
`MediaUploadFinished`. sent, cancelled, its room forgotten, or
missed when that channel overflowed at the wrong moment, since then there's no
knowing which. a failure isn't an ending, a wedged upload can still be retried.
uploads don't belong to a room's actor, so closing the room leaves them be

that watcher is the only thing speaking about uploads. anything that ever
showed progress or a failure is followed until it gets exactly one
`MediaUploadFinished`, and nothing about it goes out after. the sdk sends
progress from a detached task, so a tick can turn up behind its own upload's
end, and the watcher remembers the last few hundred ends to drop those. the
engine stopping ends nothing, `Stopped` covers it

## sessions and secrets

`Session::login_password`, `begin_oauth` and `restore` share one client builder
with an encrypted store, refresh tokens and threads. the store is sqlite
natively and indexeddb in a browser. secret-bearing inputs don't derive `Clone`

the homeserver a person typed is checked before the client sees it, and a bad
one is `AccountError::InvalidHomeserver`. a bare server name passes untouched,
since that's what sends the sdk to `.well-known`, and a host that "helpfully"
adds `https://` turns `example.org` into a homeserver url and skips discovery.
a url needs a host, no userinfo, and https. plain http needs
`allow_insecure_loopback` and a loopback host. `SoftLogout::password_login` sets
it, since its homeserver passed once already

a `SessionStore` owns one login's record and its store. login saves the first
record, `restore` loads it, and the engine a `Session` builds saves every token
rotation after that, since a refresh token works once and a host that missed one
save has lost the login. the session changes are subscribed the moment the
client exists, because the sdk can refresh during restore before any engine
does. a save runs off the event stream, so a full stream never holds it back,
and `SessionTokensChanged` only goes out once it has landed, or
`SessionSaveFailed` if it didn't. natively the sdk's save callback runs inside
the refresh, before the new token is used, and `save_blocking` writes there and
then. in a browser there's only the async save, so a tab that dies between a
refresh and its save wakes up holding a spent token. a fork patch letting the
sdk await an async save before it uses the new token would close that, but it
changes how a refresh behaves rather than what's visible, and on 2026-10-22 we
held off so the fork doesn't drift that far that fast. upstream taking an async
save callback, or the fork already carrying behaviour changes, reopens it.
`without_persistence`
hands the job back to the host. `EncryptedSessionStore` seals the record with
the sdk's `StoreCipher` under the store's passphrase and replaces it by rename,
so a crash mid save leaves the old one. it keeps the record out of the sdk's
state store because sqlite writes custom values there in the clear, and a
client can't open its store before it knows the homeserver the record holds.
`discard` deletes the record, then the store, since a record without its store
would restore into empty crypto. the host follows `Termination::retain`.
`Nothing` discards. `Store` calls `soft_log_out`, which keeps the store and
swaps the record for one holding the homeserver, user, device and oauth client
id with no tokens. `restore` hands that back as `Restore::SoftLoggedOut`, so a
restart still knows which device to log in again as, and the login overwrites
it. one store directory per login. a login whose first save fails comes back as
`AccountError::Unsaved` holding the live session, so the host retries the save
rather than spending another device

`Accounts` picks the store, so a host never decides which account a login form
means. it's one owner's accounts under one root and one passphrase, a store and
record each, named by a random id. a password login that names a
soft-logged-out account, by localpart or full user id on the same server, logs
back in as that device into that store. the server is compared by name as well
as by url, since a typed `example.org` is stored as wherever discovery led. a
live account it names is `AlreadyLoggedIn`, anyone else gets a new store. oauth
can't know who it's for until it finishes, so it resumes only the device
`SoftLogout::oauth_login` names. a new store is locked before it's listed, so
`list` can sweep stores with no record, left by a login that died or a discard
that didn't finish, without catching one mid login. a browser lists accounts in
an IndexedDB database of their own, a key each, so two tabs adding at once
don't lose one

the set is per owner because resuming needs only the password, and the store
holds the device's e2ee keys. a set shared across people would hand someone who
knows a password the history a fresh login wouldn't get

one engine per store. before the client opens the store we take the platform's
own exclusive lock on it, `File::try_lock` on `macltok.lock` in the store
directory natively and a Web Lock named after the store in a browser. the
engine and every `ClientHandle` share it, so it goes when the last of them
does, since a lingering handle still has the client and its store open. anyone
else, in this process, another one or another tab, gets `StoreInUse` at once,
so the host decides whether to wait, hand over or say so. the os or the browser
lets go when the holder dies. a SharedWorker serving every tab from one engine
never contends for it. Web Locks only exist in a secure context, so a browser
store needs one too

the sdk's cross process lease can't do this job. in matrix-sdk-common 0.18
`CrossProcessLock::try_lock_once` (`cross_process_lock.rs` 389-448) leases for
500ms and renews every 50ms from a spawned task, which gives up for good on one
failed renewal or on finding the lease lapsed and retaken, and tells no one. a
stall past half a second, ordinary in a tab and certain during the record's
PBKDF2, or one `SQLITE_BUSY`, hands the store to a second engine while the
first still thinks it's alone. with our lock keeping everyone else out, clients
run with `CrossProcessLockConfig::SingleProcess` and the sdk's own sections skip
the lease writes and renewal tasks

a pending OAuth login survives the page going away for its redirect.
`begin_oauth` saves it as the store's record, sealed like any other since it
holds the PKCE verifier, through the fork's `OAuth::pending_authorization`.
`Accounts::finish_oauth` takes the callback url and finds the account by its
`state`, so the host keeps nothing across the redirect, and
`Session::finish_oauth` does the same for one store. a host that keeps the
process alive can still use `PendingOAuthLogin::finish`. while it waits,
`restore` answers `Restore::OAuthPending`. the record lives in the store
rather than with the host because `list` sweeps a store with no record, which
would delete a fresh login's store mid redirect

the pending record keeps whatever the store held before, so a login resuming a
soft-logged-out device that never comes back puts that device back, and a
password login can still resume it meanwhile. abort, a failed finish after the
redirect and an hour's wait all undo the login, a new account's store deleted
and an old record put back. a failed `PendingOAuthLogin::finish` still holds
the client, so it puts back an old record and leaves a new one to the hour.
codes live minutes, so an hour is generous

## room lists, spaces and profiles

`RoomSummary` has what a room list row needs without a second query. topic,
alias, member counts, heroes for nameless rooms, the inviter, where an upgraded
room went, a latest event preview, the joined spaces listing it, and for a
space the children it lists

parents come from one lazily built `SpaceService` shared by every handle.
children come straight from the space's local state, joined or not.
`space_hierarchy` pages through the server's view for browsing rooms you
haven't joined, and holds its place between pages

`own_profile` and `user_profile` read profiles. name and avatar changes are
commands, and uploading an avatar gets the same byte cap as any upload

## notifications and room data

the notification client is built lazily and shares the engine's sync service.
sdk ui types stay behind the boundary

`rooms` normalises on demand with bounded concurrency and sorts by latest event,
then name, then id. `RoomsChanged` is an invalidation, coalesced so one sync
touching fifty rooms is one event. a sync's room info lands before the event
cache has stored its events, so the watcher waits a tenth of a second after
anything wakes it and takes the lot as one. an event cache slower than that
costs a second one. a left room stays in the list with its
membership until it's forgotten. push rule changes never touch room info, so
`NotificationSettingsChanged` brings a `RoomsChanged` along for the modes in
the summaries

with threads on, the sdk leaves thread events out of a room's counts and keeps
none of its own for them. `thread_unreads` counts them per root from the event
cache store, which keeps every reply it ever took whatever it holds in memory,
so closing a room, the cache trimming itself or a restart leaves them be. a
reply counts once it's newer than a receipt for its thread (msc3771), an
unthreaded one, or the user's own reply. a main-timeline receipt doesn't reach
into threads. the store has lost stream order between a thread and the rest of
the room, so an event's timestamp stands for its place, and a receipt on an
event the store never had falls back to the receipt's own time. a thread's
mentions also count in the room's `unread_mentions`, so nobody misses one,
while its messages and notifications stay apart, since they don't move the main
timeline's read marker

the counts are kept in the state store per room, only for threads with
something unread, so `rooms` reads them without touching the event cache and a
cold room never gets one made for it. thread events and threaded receipts leave
room info alone, so the room list watcher recounts on its own. a synced thread
event names the thread to count, an event cache update recounts the room's
threads already holding something unread, since a late decryption can turn up
a mention, and so does a receipt of the user's own. counting in the sdk would
be cheaper, but that's a behaviour change in the fork

## link previews

a text message can carry its sender's own previews (msc4095), and those come
out on `Message::link_previews` without asking anyone, so they work in
encrypted rooms and leak nothing. an encrypted preview image is registered like
an attachment. a preview whose url isn't in the body is dropped, or a sender
could caption any link with any card

`url_preview` asks the homeserver, which fetches the page, re-hosts `og:image`
and answers with loose `og:` keys. that tells the homeserver every url, so
whether encrypted rooms ask is the host's call. answers are kept an hour and
misses ten minutes, and asks for one url at once share a request, since synapse
rate limits the endpoint and a timeline repeats links. synapse answers a page
it couldn't download with a 500, a dns failure with a 502 and its blocklist with
a 403, so those, 400 and 404 mean no preview. a genuine 500 hides in there too,
but a missing card is cheap. a
server that doesn't preview at all says so with `M_UNRECOGNIZED` once and isn't
asked again. a 429 anywhere is `ErrorKind::RateLimited`. page text from either
source is trimmed, 256 characters for a title or site name and 1024 for a
description, and an image that isn't an mxc uri is dropped

`url_previews_allowed` follows element, so both agree. the user's own choice
for the room, then their account wide default, then room state an admin set.
the room state comes in with the rest through the fork's extra required state

## image packs

msc2545 packs come from the user's account data, from room state, and from the
rooms the user enabled everywhere. the room list asks the server for every
room's `im.ponies.room_emotes`, through the fork's extra required state, so
packs sit in the store like any other state and a change reaches us even across
a gap in the timeline. that's every joined room's packs on the first sync,
tens of kilobytes for a big one, and only changes after. asking only for open
rooms would lose the enabled packs of rooms nobody has open

`image_packs` reads all of it from the store and hands back the lot, global
packs and each room's own, and `ImagePacks::for_room` puts a picker's list in
order. an enabled room the user hasn't joined is skipped, there's no state for
it. one bad image costs that image and a pack with nothing usable is dropped,
since pack contents are whatever someone's client wrote. `ImagePacksChanged`
says a pack moved somewhere, from a state event, an account data change or a
membership change taking a room's packs with it. it's an invalidation, and the
mirror refetches on it like it does rooms, keeping the packs in its state

## verification

`Accept`, `Confirm`, `Reject` and `Cancel`. a request another device sends
shows up as `Incoming` and waits for `Accept`, or `Cancel` to turn it down. one
is only taken up while nothing else is in flight. reject tells the other side
the emoji didn't match. the watchers subscribe before they look at the current
state, since the sdk only reports what changes after you ask. the watcher and
the action paths claim the terminal event, so a racing
cancel can't turn `Done` into `Cancelled`

## mirror

the mirror is a consumer like any other, behind the `mirror` feature, and it
takes any stream of events rather than the `EventStream` itself so a host keeps
the raw events without a second channel. state, its sequence number and the
broadcast send all move under one lock, which is how `subscribe` gets a
snapshot and a receiver with nothing between them. a lagged subscriber starts
over from a fresh state under the same lock

nothing on the draining path waits for an engine answer. a resync runs on its
own task, retrying with backoff until a reset or a close for its timeline
aborts it. rooms are fetched on a task too, one at a time with one more owed at
most. a diff for a timeline with no reset is a divergence too, so a wrapper
that swallowed the reset still heals. anything a late subscriber needs, failures
included, lives in the state rather than only in an update

## the sdk fork

we build against a fork of matrix-rust-sdk, branch `macltok`, cut from the
matrix-sdk 0.18.0 tag. turned down on 2026-09-26, taken up on 2026-10-22 once
it was holding back features every peer client has. the sdk crates are git
dependencies pinned to a fork rev, so dependents get the fork through macltok
with no patch block of their own, and macltok stays off crates.io until
upstream takes the patches. a dependent naming matrix-sdk itself has to point
at the same rev or it gets a second sdk

it carries visibility and config changes, each small enough to be an upstream
pr, and nothing that changes how the sdk behaves. if that stops being true,
especially in the timeline or the send queue, it's time to drop matrix-sdk and
matrix-sdk-ui for our own layer over matrix-sdk-base and matrix-sdk-crypto,
which keep their public sans-io apis

- `Room::prepare_encryption`, the send path's encryption prep made public
- `SyncServiceBuilder::with_room_list_extra_required_state`, for state the
  sdk's hard-coded list leaves out. image packs use it
- msc4027 reaction shortcodes, kept on the timeline's `ReactionInfo` and sent
  with `Timeline::toggle_reaction_with_shortcode`
- `SendQueue::enable_global_upload_progress`, upload progress on the
  client-wide send queue channel
- `OAuth::pending_authorization` and `restore_pending_authorization`, to
  finish an oauth login in another process

## public model

owned ruma ids and typed data wherever there's a stable shape, `Raw` where
arbitrary content has to survive. ruma is re-exported as `macltok::ruma`, so a
host parses ids at the version we're typed against. anything expected to grow is
`#[non_exhaustive]`. `Engine::from_client` stays public for people who built
their own client