# macltok design decisions

the contract the working tree actually implements. if the code disagrees with
this, one of them is lying and it should be fixed

## two ways in, one way out

state changes are commands. `ClientHandle::send` queues one and hands back a
`Pending`, `execute` queues and waits, and there's an awaited convenience method
for every command. each command answers exactly once with
`Result<CommandOutput, Error>`. drop the `Pending` if you don't care, the
command still runs

pure reads skip the engine and ask the sdk directly through the handle. rooms,
members, power levels, account data, devices, push notification resolution,
and media bytes all live here

the event stream carries state and only state. timelines, history, typing,
rooms, connection, encryption, verification, upload progress, session. no
command results ride on it. drain it from its own task, it's bounded and the
engine waits for you

one error type for everything, `Error { kind, message, challenge, server }`.
match on `kind`. `Interrupted` means the engine stopped, the room closed, or the
task doing the work died before it could answer. `challenge` turns up with
`InteractiveAuthRequired`

`message` is ours and says what we were doing. `server` is the homeserver's
errcode and message, dug out of the source chain when the error is built, so a
host can show "invalid password" without walking the chain. `AccountError`
carries the same, plus a `kind`. neither folds it into `Display`, since anything
printing the chain would say it twice

## engine

the run loop routes and does nothing slow. every task it spawns is registered
with a context. a command task that panics costs its caller an `Interrupted`
and gets logged. a watcher or a room actor that panics takes the engine down,
emitting `Failed` then `Stopped`. natively, anyway. on wasm a panic traps the
task mid poll and nothing is caught. the rest keeps running in an undefined
state and goes silent, no `Failed` and no `Stopped`, so the host hears of it
through `on_panic`, a free function since a panic hook belongs to the whole
instance rather than to one engine

sync restarts itself with backoff after an error or an unexpected stop, and
`ConnectionChanged` shows `Offline` meanwhile. a run that drops again inside
thirty seconds counts as a failure too, or a server that answers `/versions`
but refuses sync would bounce us between online and offline forever. every return to `Online` re-arms
the send queues, since the sdk parks a room's queue on any send failure and
never unparks it. a recoverable failure while online gets the room another go
after a few seconds

logout and a rejected token both stop the engine. `run` resolves to a
`Termination` naming which of those it was, or a shutdown, or every handle
dropping, so the host never works it out from the events. `Shutdown` answers once
`Stopped` has gone out, every one of them if several were queued

`Engine::run` hands back a boxed future. the real one nests deep enough that
every crate spawning it would trip the compiler's query depth

## rooms

one actor per open room. the engine holds a slot per room, `Active` with a
sender or `Closing` with a fifo of opens and closes that arrived while it wound
down. those replay in order once the actor exits. a room you leave, or get
kicked or banned from, closes itself. it tells the engine first, and anything
already queued to it comes back to be answered or replayed, so an open racing a
leave still gets a fresh actor

however an actor ends, short of the engine stopping, its typing goes out empty
and each of its timelines sends `TimelineClosed`. so does a closed thread. a
timeline that never sent a reset never sends a close either

opening sends `TimelineReset` on the stream before it answers. opening an
already open room sends a fresh reset, so a second frontend gets a snapshot

sends, edits and replies run on the actor in order, so they reach the send
queue in the order they were asked for. redactions, reactions and read
receipts are network calls, so they run beside the actor and a slow one
doesn't queue the room behind it. closing lets them finish rather than lie

threads belong to their room's actor. commands for a thread that's still
opening wait behind the open, up to the actor's queue capacity

pagination runs one request at a time per timeline and emits `HistoryChanged`
around each. the sdk remembers where history starts, we don't, because a gappy
sync can unload it again

the actor's watchers are supervised, and a panic in any of them panics the
actor. its command work isn't, a panicking redaction just answers `Interrupted`
and gets logged

## timelines

`TimelineDiff::try_apply` refuses a diff whose indices don't fit your snapshot
and leaves the snapshot alone. `try_apply_all` does the same for a whole batch,
checking every diff against lengths alone first, since that's all a diff's fit
depends on. snapshots are `imbl::Vector`s so keeping one is a pointer copy.
stop applying, `resync`, and carry on from the next `TimelineReset`. resync
kills the old watcher before the new reset goes out, so no stale diff can land
after it

`TimelineId` tells a room from a thread by its root event

every item is an event, a day divider, the read marker or the start. an event
carries its sender with avatar and name ambiguity, and a `TimelineContent`
saying what it actually is. messages keep their msgtype, membership and profile
changes and the common room state changes get their own variants, and anything
else is `OtherState` or `Other` with its type. nothing is flattened into english
in a body field

thread receipts for a thread that isn't open go straight to the room, so
marking a thread read works the same whether its room is on screen or not

replies carry the replied-to event once it's loaded. the watcher asks the sdk
for any it hasn't got, once per event, since the sdk won't on its own

edit, redact and react take an `EventRef`, so your own message can be edited or
reacted to before the server has echoed it back. redacting an unsent one just
cancels it

## warming up to send

the sdk does all its key work on the send path, in the room's send queue before
the `/send` itself. members synced, a `/keys/query` for anyone untracked or
dirty, then a `/keys/claim` and the room key sent to every device. over a slow
federation that's a minute of nothing. `prepare_to_send` runs all of it ahead
of time through the fork's `Room::prepare_encryption`, the same function the
send queue calls. the share sits behind the sdk's per-room dedup, so a send
landing mid-warm-up waits for it rather than sharing again, and a send after it
finds nothing left to do but the `/send`. a room whose encryption isn't known
yet gets asked, which the send would have done anyway. one warm-up per room at
a time, a second ask joins it. opening a room or starting to type runs one too,
unless `EngineConfig::prepare_to_send` is off. it's on because malice asked,
not because anyone decided

the session rotates after 100 messages or a week, and sharing isn't a message,
so a warm-up is only wasted on a room left idle about that long

## mutations

membership, moderation, power levels, room creation and account data writes go
through one keyed fifo scheduler. same room, same alias, same dm partner or
same account data type run in order, everything else runs at once.
`create_direct` hands back the existing dm when there is one

notifications and encryption each get one serial worker. the encryption worker
owns the verification controller outright

power level changes build one state event from the local mapping, since the
sdk's own patcher is private

## media

media is a read. `ClientHandle::media` takes a `MediaId`, from a retained
timeline or an avatar, through one bounded
download path with the byte cap the sdk's own download lacks, an hour's total
deadline instead of the sdk client's thirty seconds, and a minute's patience
for a stalled stream. concurrent asks for the same bytes share one flight.
a thumbnail the server won't make falls back to the file itself.
`MediaId` parses from a string, since a web client has to round trip it

the cache holds plaintext, keyed like the sdk keys it, so the sdk and macltok
can read each other's entries. that's how your own encrypted uploads come back.
the sdk keys plain and encrypted sources alike, so a plain event pointing at
someone's encrypted file can leave ciphertext in the cache. an encrypted read
checks the cached bytes against the file's hash and decrypts them if they match

media ids are registered as timelines render and forgotten when their timeline
closes, least recently seen first once the cap is hit

avatars are `MediaId`s too, so a host never handles an mxc uri. they aren't
registered, since they live in the room list and member lists for the whole
session and there can be thousands, and the cap would evict ones still on
screen. a plain avatar carries no keys, so its id is its mxc uri and resolves
without the registry. it never expires, and an mxc uri a host got some other way
parses into one. `set_avatar` still takes an mxc uri

streaming decryption inside macltok was declined. in a browser the service
worker answering range requests is a separate context from the worker running
macltok, and streaming through macltok would carry every chunk through wasm
memory, the very thing the host streams to avoid. `media_source` hands over the
url and keys `media` would have used, picked the same way, and `access_token`
the rest. the byte cap doesn't apply, nothing is buffered

uploads are commands that answer once the send queue has them. progress only
goes out on a room's own send queue channel, so each room that uploads gets a
forwarder, subscribed before the upload is queued and restarted for uploads the
sdk resumes from last time. everything but progress reaches the client-wide
channel, so one watcher there follows every attachment in the queue and ends
each with one `MediaUploadFinished`. sent, cancelled, its room forgotten, or
missed when that channel overflowed at the wrong moment, since then there's no
knowing which. a failure isn't an ending, a wedged upload can still be retried.
uploads don't belong to a room's actor, so closing the room leaves them be

the forwarders hand their progress to that watcher rather than the stream, so
it's the only thing speaking about uploads. anything that ever showed progress
or a failure is followed until it gets exactly one `MediaUploadFinished`, and
nothing about it goes out after. each forwarder relays its room's sends and
cancels behind its progress, so an end never overtakes the ticks before it. an
upload ended from anywhere else is remembered until its forwarder's own end
turns up, since until then there may be progress still queued for it. the
engine stopping ends nothing, `Stopped` covers it

## sessions and secrets

`Session::login_password`, `begin_oauth` and `restore` share one client builder
with an encrypted store, refresh tokens and threads. the store is sqlite
natively and indexeddb in a browser. secret-bearing inputs don't derive `Clone`

the homeserver a person typed is checked before the client sees it, and a bad
one is `AccountError::InvalidHomeserver`. a bare server name passes untouched,
since that's what sends the sdk to `.well-known`, and a host that "helpfully"
adds `https://` turns `example.org` into a homeserver url and skips discovery.
a url needs a host, no userinfo, and https. plain http needs
`allow_insecure_loopback` and a loopback host. `SoftLogout::password_login` sets
it, since its homeserver passed once already

a `SessionStore` owns one login's record and its store. login saves the first
record, `restore` loads it, and the engine a `Session` builds saves every token
rotation after that, since a refresh token works once and a host that missed one
save has lost the login. the session changes are subscribed the moment the
client exists, because the sdk can refresh during restore before any engine
does. a save runs off the event stream, so a full stream never holds it back,
and `SessionTokensChanged` only goes out once it has landed, or
`SessionSaveFailed` if it didn't. natively the sdk's save callback runs inside
the refresh, before the new token is used, and `save_blocking` writes there and
then. in a browser there's only the async save, so a tab that dies between a
refresh and its save wakes up holding a spent token. `without_persistence`
hands the job back to the host. `EncryptedSessionStore` seals the record with
the sdk's `StoreCipher` under the store's passphrase and replaces it by rename,
so a crash mid save leaves the old one. it keeps the record out of the sdk's
state store because sqlite writes custom values there in the clear, and a
client can't open its store before it knows the homeserver the record holds.
`discard` deletes the record, then the store, since a record without its store
would restore into empty crypto. the host follows `Termination::retain`.
`Nothing` discards. `Store` calls `soft_log_out`, which keeps the store and
swaps the record for one holding the homeserver, user, device and oauth client
id with no tokens. `restore` hands that back as `Restore::SoftLoggedOut`, so a
restart still knows which device to log in again as, and the login overwrites
it. one store directory per login. a login whose first save fails comes back as
`AccountError::Unsaved` holding the live session, so the host retries the save
rather than spending another device

`Accounts` picks the store, so a host never decides which account a login form
means. it's one owner's accounts under one root and one passphrase, a store and
record each, named by a random id. a password login that names a
soft-logged-out account, by localpart or full user id on the same server, logs
back in as that device into that store. the server is compared by name as well
as by url, since a typed `example.org` is stored as wherever discovery led. a
live account it names is `AlreadyLoggedIn`, anyone else gets a new store. oauth
can't know who it's for until it finishes, so it resumes only the device
`SoftLogout::oauth_login` names. a new store is locked before it's listed, so
`list` can sweep stores with no record, left by a login that died or a discard
that didn't finish, without catching one mid login. a browser lists accounts in
an IndexedDB database of their own, a key each, so two tabs adding at once
don't lose one

the set is per owner because resuming needs only the password, and the store
holds the device's e2ee keys. a set shared across people would hand someone who
knows a password the history a fresh login wouldn't get

one engine per store. before the client opens the store we take the platform's
own exclusive lock on it, `File::try_lock` on `macltok.lock` in the store
directory natively and a Web Lock named after the store in a browser. the
engine and every `ClientHandle` share it, so it goes when the last of them
does, since a lingering handle still has the client and its store open. anyone
else, in this process, another one or another tab, gets `StoreInUse` at once,
so the host decides whether to wait, hand over or say so. the os or the browser
lets go when the holder dies. a SharedWorker serving every tab from one engine
never contends for it. Web Locks only exist in a secure context, so a browser
store needs one too

the sdk's cross process lease can't do this job. in matrix-sdk-common 0.18
`CrossProcessLock::try_lock_once` (`cross_process_lock.rs` 389-448) leases for
500ms and renews every 50ms from a spawned task, which gives up for good on one
failed renewal or on finding the lease lapsed and retaken, and tells no one. a
stall past half a second, ordinary in a tab and certain during the record's
PBKDF2, or one `SQLITE_BUSY`, hands the store to a second engine while the
first still thinks it's alone. with our lock keeping everyone else out, clients
run with `CrossProcessLockConfig::SingleProcess` and the sdk's own sections skip
the lease writes and renewal tasks

a pending OAuth login isn't persisted. `begin_oauth` and `finish` have to run in
the same process, so a web host runs the login in a popup or in a worker that
outlives the redirect. the sdk keeps what `finish_login` needs, the PKCE
verifier among it, in `AuthorizationValidationData` inside a `pub(crate)` map
keyed by `CsrfToken` (matrix-sdk 0.18, `authentication/oauth/mod.rs` around
262-266 and 1744-1756). nothing public reads it or puts it back, and
`finish_login` on a fresh client answers `InvalidState`. doing the code grant
ourselves would copy some 250 lines of the sdk's private oauth behaviour and
drift from it every upgrade. patching the sdk to export and import that data is
small, and since we carry a fork now that's the way to do it. not built yet

## room lists, spaces and profiles

`RoomSummary` has what a room list row needs without a second query. topic,
alias, member counts, heroes for nameless rooms, the inviter, where an upgraded
room went, a latest event preview, the joined spaces listing it, and for a
space the children it lists

parents come from one lazily built `SpaceService` shared by every handle.
children come straight from the space's local state, joined or not.
`space_hierarchy` pages through the server's view for browsing rooms you
haven't joined, and holds its place between pages

`own_profile` and `user_profile` read profiles. name and avatar changes are
commands, and uploading an avatar gets the same byte cap as any upload

## notifications and room data

the notification client is built lazily and shares the engine's sync service.
sdk ui types stay behind the boundary

`rooms` normalises on demand with bounded concurrency and sorts by latest event,
then name, then id. `RoomsChanged` is an invalidation, coalesced so one sync
touching fifty rooms is one event. a left room stays in the list with its
membership until it's forgotten. push rule changes never touch room info, so
`NotificationSettingsChanged` brings a `RoomsChanged` along for the modes in
the summaries

## image packs

msc2545 packs come from the user's account data, from room state, and from the
rooms the user enabled everywhere. the room list asks the server for every
room's `im.ponies.room_emotes`, through the fork's extra required state, so
packs sit in the store like any other state and a change reaches us even across
a gap in the timeline. that's every joined room's packs on the first sync,
tens of kilobytes for a big one, and only changes after. asking only for open
rooms would lose the enabled packs of rooms nobody has open

`image_packs` reads all of it from the store and hands back the lot, global
packs and each room's own, and `ImagePacks::for_room` puts a picker's list in
order. an enabled room the user hasn't joined is skipped, there's no state for
it. one bad image costs that image and a pack with nothing usable is dropped,
since pack contents are whatever someone's client wrote. `ImagePacksChanged`
says a pack moved somewhere, from a state event, an account data change or a
membership change taking a room's packs with it. it's an invalidation, and the
mirror refetches on it like it does rooms, keeping the packs in its state

## verification

`Accept`, `Confirm`, `Reject` and `Cancel`. a request another device sends
shows up as `Incoming` and waits for `Accept`, or `Cancel` to turn it down. one
is only taken up while nothing else is in flight. reject tells the other side
the emoji didn't match. the watchers subscribe before they look at the current
state, since the sdk only reports what changes after you ask. the watcher and
the action paths claim the terminal event, so a racing
cancel can't turn `Done` into `Cancelled`

## mirror

the mirror is a consumer like any other, behind the `mirror` feature, and it
takes any stream of events rather than the `EventStream` itself so a host keeps
the raw events without a second channel. state, its sequence number and the
broadcast send all move under one lock, which is how `subscribe` gets a
snapshot and a receiver with nothing between them. a lagged subscriber starts
over from a fresh state under the same lock

nothing on the draining path waits for an engine answer. a resync runs on its
own task, retrying with backoff until a reset or a close for its timeline
aborts it. rooms are fetched on a task too, one at a time with one more owed at
most. a diff for a timeline with no reset is a divergence too, so a wrapper
that swallowed the reset still heals. anything a late subscriber needs, failures
included, lives in the state rather than only in an update

## the sdk fork

we build against a fork of matrix-rust-sdk, branch `macltok`, cut from the
matrix-sdk 0.18.0 tag. turned down on 2026-09-26, taken up on 2026-10-22 once
it was holding back features every peer client has. every workspace that builds
macltok needs the same `[patch.crates-io]` block, since patches don't reach
dependents, and it keeps macltok off crates.io until upstream takes the patches.
a vendored copy of macltok should drop its own block, cargo ignores it there
and says so on every build

it carries visibility and config changes, each small enough to be an upstream
pr, and nothing that changes how the sdk behaves. if that stops being true,
especially in the timeline or the send queue, it's time to drop matrix-sdk and
matrix-sdk-ui for our own layer over matrix-sdk-base and matrix-sdk-crypto,
which keep their public sans-io apis

- `Room::prepare_encryption`, the send path's encryption prep made public
- `SyncServiceBuilder::with_room_list_extra_required_state`, for state the
  sdk's hard-coded list leaves out. image packs use it

## public model

owned ruma ids and typed data wherever there's a stable shape, `Raw` where
arbitrary content has to survive. ruma is re-exported as `macltok::ruma`, so a
host parses ids at the version we're typed against. anything expected to grow is
`#[non_exhaustive]`. `Engine::from_client` stays public for people who built
their own client