macltok

macltok design decisions

the contract the working tree actually implements. if the code disagrees with this, one of them is lying and it should be fixed

two ways in, one way out

state changes are commands. ClientHandle::send queues one and hands back a Pending, execute queues and waits, and there’s an awaited convenience method for every command. each command answers exactly once with Result<CommandOutput, Error>. drop the Pending if you don’t care, the command still runs

pure reads skip the engine and ask the sdk directly through the handle. rooms, members, power levels, account data, devices, push notification resolution, and media bytes all live here

the event stream carries state and only state. timelines, history, typing, rooms, connection, encryption, verification, upload progress, session. no command results ride on it. drain it from its own task, it’s bounded and the engine waits for you

one error type for everything, Error { kind, message, challenge, server }. match on kind. Interrupted means the engine stopped, the room closed, or the task doing the work died before it could answer. challenge turns up with InteractiveAuthRequired

message is ours and says what we were doing. server is the homeserver’s errcode and message, dug out of the source chain when the error is built, so a host can show “invalid password” without walking the chain. AccountError carries the same, plus a kind. neither folds it into Display, since anything printing the chain would say it twice

engine

the run loop routes and does nothing slow. every task it spawns is registered with a context. a command task that panics costs its caller an Interrupted and gets logged. a watcher or a room actor that panics takes the engine down, emitting Failed then Stopped. natively, anyway. on wasm a panic traps the task mid poll and nothing is caught. the rest keeps running in an undefined state and goes silent, no Failed and no Stopped, so the host hears of it through on_panic, a free function since a panic hook belongs to the whole instance rather than to one engine

sync restarts itself with backoff after an error or an unexpected stop, and ConnectionChanged shows Offline meanwhile. a run that drops again inside thirty seconds counts as a failure too, or a server that answers /versions but refuses sync would bounce us between online and offline forever. every return to Online re-arms the send queues, since the sdk parks a room’s queue on any send failure and never unparks it. a recoverable failure while online gets the room another go after a few seconds

logout and a rejected token both stop the engine. run resolves to a Termination naming which of those it was, or a shutdown, or every handle dropping, so the host never works it out from the events. Shutdown answers once Stopped has gone out, every one of them if several were queued

Engine::run hands back a boxed future. the real one nests deep enough that every crate spawning it would trip the compiler’s query depth

rooms

one actor per open room. the engine holds a slot per room, Active with a sender or Closing with a fifo of opens and closes that arrived while it wound down. those replay in order once the actor exits. a room you leave, or get kicked or banned from, closes itself. it tells the engine first, and anything already queued to it comes back to be answered or replayed, so an open racing a leave still gets a fresh actor

however an actor ends, short of the engine stopping, its typing goes out empty and each of its timelines sends TimelineClosed. so does a closed thread. a timeline that never sent a reset never sends a close either

opening sends TimelineReset on the stream before it answers. opening an already open room sends a fresh reset, so a second frontend gets a snapshot

sends, edits and replies run on the actor in order, so they reach the send queue in the order they were asked for. redactions, reactions and read receipts are network calls, so they run beside the actor and a slow one doesn’t queue the room behind it. closing lets them finish rather than lie

threads belong to their room’s actor. commands for a thread that’s still opening wait behind the open, up to the actor’s queue capacity

pagination runs one request at a time per timeline and emits HistoryChanged around each. the sdk remembers where history starts, we don’t, because a gappy sync can unload it again

the actor’s watchers are supervised, and a panic in any of them panics the actor. its command work isn’t, a panicking redaction just answers Interrupted and gets logged

timelines

TimelineDiff::try_apply refuses a diff whose indices don’t fit your snapshot and leaves the snapshot alone. try_apply_all does the same for a whole batch, checking every diff against lengths alone first, since that’s all a diff’s fit depends on. snapshots are imbl::Vectors so keeping one is a pointer copy. stop applying, resync, and carry on from the next TimelineReset. resync kills the old watcher before the new reset goes out, so no stale diff can land after it

TimelineId tells a room from a thread by its root event

every item is an event, a day divider, the read marker or the start. an event carries its sender with avatar and name ambiguity, and a TimelineContent saying what it actually is. messages keep their msgtype, membership and profile changes and the common room state changes get their own variants, and anything else is OtherState or Other with its type. nothing is flattened into english in a body field

thread receipts for a thread that isn’t open go straight to the room, so marking a thread read works the same whether its room is on screen or not

replies carry the replied-to event once it’s loaded. the watcher asks the sdk for any it hasn’t got, once per event, since the sdk won’t on its own

edit, redact and react take an EventRef, so your own message can be edited or reacted to before the server has echoed it back. redacting an unsent one just cancels it

a custom emote reaction keys on the image’s mxc uri and carries its shortcode (msc4027), so clients without the pack can still label it. Reaction says which image it is and the shortcode any sender gave, and toggle_reaction takes the shortcode to send

warming up to send

the sdk does all its key work on the send path, in the room’s send queue before the /send itself. members synced, a /keys/query for anyone untracked or dirty, then a /keys/claim and the room key sent to every device. over a slow federation that’s a minute of nothing. prepare_to_send runs all of it ahead of time through the fork’s Room::prepare_encryption, the same function the send queue calls. the share sits behind the sdk’s per-room dedup, so a send landing mid-warm-up waits for it rather than sharing again, and a send after it finds nothing left to do but the /send. a room whose encryption isn’t known yet gets asked, which the send would have done anyway. one warm-up per room at a time, a second ask joins it. opening a room or starting to type runs one too, unless EngineConfig::prepare_to_send is off. it’s on because malice asked, not because anyone decided

the session rotates after 100 messages or a week, and sharing isn’t a message, so a warm-up is only wasted on a room left idle about that long

mutations

membership, moderation, power levels, room creation and account data writes go through one keyed fifo scheduler. same room, same alias, same dm partner or same account data type run in order, everything else runs at once. create_direct hands back the existing dm when there is one

notifications and encryption each get one serial worker. the encryption worker owns the verification controller outright

power level changes build one state event from the local mapping, since the sdk’s own patcher is private

media

media is a read. ClientHandle::media takes a MediaId, from a retained timeline or an avatar, through one bounded download path with the byte cap the sdk’s own download lacks, an hour’s total deadline instead of the sdk client’s thirty seconds, and a minute’s patience for a stalled stream. concurrent asks for the same bytes share one flight. a thumbnail the server won’t make falls back to the file itself. MediaId parses from a string, since a web client has to round trip it

the cache holds plaintext, keyed like the sdk keys it, so the sdk and macltok can read each other’s entries. that’s how your own encrypted uploads come back. the sdk keys plain and encrypted sources alike, so a plain event pointing at someone’s encrypted file can leave ciphertext in the cache. an encrypted read checks the cached bytes against the file’s hash and decrypts them if they match

media ids are registered as timelines render and forgotten when their timeline closes, least recently seen first once the cap is hit

avatars are MediaIds too, so a host never handles an mxc uri. they aren’t registered, since they live in the room list and member lists for the whole session and there can be thousands, and the cap would evict ones still on screen. a plain avatar carries no keys, so its id is its mxc uri and resolves without the registry. it never expires, and an mxc uri a host got some other way parses into one. set_avatar still takes an mxc uri

streaming decryption inside macltok was declined. in a browser the service worker answering range requests is a separate context from the worker running macltok, and streaming through macltok would carry every chunk through wasm memory, the very thing the host streams to avoid. media_source hands over the url and keys media would have used, picked the same way, and access_token the rest. the byte cap doesn’t apply, nothing is buffered

uploads are commands that answer once the send queue has them. the fork puts upload progress on the client-wide send queue channel too, which upstream keeps for each room’s own, so one watcher there follows every attachment in the queue, leftovers the sdk resumes from last time included, and ends each with one MediaUploadFinished. sent, cancelled, its room forgotten, or missed when that channel overflowed at the wrong moment, since then there’s no knowing which. a failure isn’t an ending, a wedged upload can still be retried. uploads don’t belong to a room’s actor, so closing the room leaves them be

that watcher is the only thing speaking about uploads. anything that ever showed progress or a failure is followed until it gets exactly one MediaUploadFinished, and nothing about it goes out after. the sdk sends progress from a detached task, so a tick can turn up behind its own upload’s end, and the watcher remembers the last few hundred ends to drop those. the engine stopping ends nothing, Stopped covers it

sessions and secrets

Session::login_password, begin_oauth and restore share one client builder with an encrypted store, refresh tokens and threads. the store is sqlite natively and indexeddb in a browser. secret-bearing inputs don’t derive Clone

the homeserver a person typed is checked before the client sees it, and a bad one is AccountError::InvalidHomeserver. a bare server name passes untouched, since that’s what sends the sdk to .well-known, and a host that “helpfully” adds https:// turns example.org into a homeserver url and skips discovery. a url needs a host, no userinfo, and https. plain http needs allow_insecure_loopback and a loopback host. SoftLogout::password_login sets it, since its homeserver passed once already

a SessionStore owns one login’s record and its store. login saves the first record, restore loads it, and the engine a Session builds saves every token rotation after that, since a refresh token works once and a host that missed one save has lost the login. the session changes are subscribed the moment the client exists, because the sdk can refresh during restore before any engine does. a save runs off the event stream, so a full stream never holds it back, and SessionTokensChanged only goes out once it has landed, or SessionSaveFailed if it didn’t. natively the sdk’s save callback runs inside the refresh, before the new token is used, and save_blocking writes there and then. in a browser there’s only the async save, so a tab that dies between a refresh and its save wakes up holding a spent token. a fork patch letting the sdk await an async save before it uses the new token would close that, but it changes how a refresh behaves rather than what’s visible, and on 2026-10-22 we held off so the fork doesn’t drift that far that fast. upstream taking an async save callback, or the fork already carrying behaviour changes, reopens it. without_persistence hands the job back to the host. EncryptedSessionStore seals the record with the sdk’s StoreCipher under the store’s passphrase and replaces it by rename, so a crash mid save leaves the old one. it keeps the record out of the sdk’s state store because sqlite writes custom values there in the clear, and a client can’t open its store before it knows the homeserver the record holds. discard deletes the record, then the store, since a record without its store would restore into empty crypto. the host follows Termination::retain. Nothing discards. Store calls soft_log_out, which keeps the store and swaps the record for one holding the homeserver, user, device and oauth client id with no tokens. restore hands that back as Restore::SoftLoggedOut, so a restart still knows which device to log in again as, and the login overwrites it. one store directory per login. a login whose first save fails comes back as AccountError::Unsaved holding the live session, so the host retries the save rather than spending another device

Accounts picks the store, so a host never decides which account a login form means. it’s one owner’s accounts under one root and one passphrase, a store and record each, named by a random id. a password login that names a soft-logged-out account, by localpart or full user id on the same server, logs back in as that device into that store. the server is compared by name as well as by url, since a typed example.org is stored as wherever discovery led. a live account it names is AlreadyLoggedIn, anyone else gets a new store. oauth can’t know who it’s for until it finishes, so it resumes only the device SoftLogout::oauth_login names. a new store is locked before it’s listed, so list can sweep stores with no record, left by a login that died or a discard that didn’t finish, without catching one mid login. a browser lists accounts in an IndexedDB database of their own, a key each, so two tabs adding at once don’t lose one

the set is per owner because resuming needs only the password, and the store holds the device’s e2ee keys. a set shared across people would hand someone who knows a password the history a fresh login wouldn’t get

one engine per store. before the client opens the store we take the platform’s own exclusive lock on it, File::try_lock on macltok.lock in the store directory natively and a Web Lock named after the store in a browser. the engine and every ClientHandle share it, so it goes when the last of them does, since a lingering handle still has the client and its store open. anyone else, in this process, another one or another tab, gets StoreInUse at once, so the host decides whether to wait, hand over or say so. the os or the browser lets go when the holder dies. a SharedWorker serving every tab from one engine never contends for it. Web Locks only exist in a secure context, so a browser store needs one too

the sdk’s cross process lease can’t do this job. in matrix-sdk-common 0.18 CrossProcessLock::try_lock_once (cross_process_lock.rs 389-448) leases for 500ms and renews every 50ms from a spawned task, which gives up for good on one failed renewal or on finding the lease lapsed and retaken, and tells no one. a stall past half a second, ordinary in a tab and certain during the record’s PBKDF2, or one SQLITE_BUSY, hands the store to a second engine while the first still thinks it’s alone. with our lock keeping everyone else out, clients run with CrossProcessLockConfig::SingleProcess and the sdk’s own sections skip the lease writes and renewal tasks

a pending OAuth login survives the page going away for its redirect. begin_oauth saves it as the store’s record, sealed like any other since it holds the PKCE verifier, through the fork’s OAuth::pending_authorization. Accounts::finish_oauth takes the callback url and finds the account by its state, so the host keeps nothing across the redirect, and Session::finish_oauth does the same for one store. a host that keeps the process alive can still use PendingOAuthLogin::finish. while it waits, restore answers Restore::OAuthPending. the record lives in the store rather than with the host because list sweeps a store with no record, which would delete a fresh login’s store mid redirect

the pending record keeps whatever the store held before, so a login resuming a soft-logged-out device that never comes back puts that device back, and a password login can still resume it meanwhile. abort, a failed finish after the redirect and an hour’s wait all undo the login, a new account’s store deleted and an old record put back. a failed PendingOAuthLogin::finish still holds the client, so it puts back an old record and leaves a new one to the hour. codes live minutes, so an hour is generous

room lists, spaces and profiles

RoomSummary has what a room list row needs without a second query. topic, alias, member counts, heroes for nameless rooms, the inviter, where an upgraded room went, a latest event preview, the joined spaces listing it, and for a space the children it lists

parents come from one lazily built SpaceService shared by every handle. children come straight from the space’s local state, joined or not. space_hierarchy pages through the server’s view for browsing rooms you haven’t joined, and holds its place between pages

own_profile and user_profile read profiles. name and avatar changes are commands, and uploading an avatar gets the same byte cap as any upload

notifications and room data

the notification client is built lazily and shares the engine’s sync service. sdk ui types stay behind the boundary

rooms normalises on demand with bounded concurrency and sorts by latest event, then name, then id. RoomsChanged is an invalidation, coalesced so one sync touching fifty rooms is one event. a left room stays in the list with its membership until it’s forgotten. push rule changes never touch room info, so NotificationSettingsChanged brings a RoomsChanged along for the modes in the summaries

link previews

a text message can carry its sender’s own previews (msc4095), and those come out on Message::link_previews without asking anyone, so they work in encrypted rooms and leak nothing. an encrypted preview image is registered like an attachment. a preview whose url isn’t in the body is dropped, or a sender could caption any link with any card

url_preview asks the homeserver, which fetches the page, re-hosts og:image and answers with loose og: keys. that tells the homeserver every url, so whether encrypted rooms ask is the host’s call. answers are kept an hour and misses ten minutes, and asks for one url at once share a request, since synapse rate limits the endpoint and a timeline repeats links. synapse answers a page it couldn’t download with a 500, a dns failure with a 502 and its blocklist with a 403, so those, 400 and 404 mean no preview. a genuine 500 hides in there too, but a missing card is cheap. a server that doesn’t preview at all says so with M_UNRECOGNIZED once and isn’t asked again. a 429 anywhere is ErrorKind::RateLimited. page text from either source is trimmed, 256 characters for a title or site name and 1024 for a description, and an image that isn’t an mxc uri is dropped

url_previews_allowed follows element, so both agree. the user’s own choice for the room, then their account wide default, then room state an admin set. the room state comes in with the rest through the fork’s extra required state

image packs

msc2545 packs come from the user’s account data, from room state, and from the rooms the user enabled everywhere. the room list asks the server for every room’s im.ponies.room_emotes, through the fork’s extra required state, so packs sit in the store like any other state and a change reaches us even across a gap in the timeline. that’s every joined room’s packs on the first sync, tens of kilobytes for a big one, and only changes after. asking only for open rooms would lose the enabled packs of rooms nobody has open

image_packs reads all of it from the store and hands back the lot, global packs and each room’s own, and ImagePacks::for_room puts a picker’s list in order. an enabled room the user hasn’t joined is skipped, there’s no state for it. one bad image costs that image and a pack with nothing usable is dropped, since pack contents are whatever someone’s client wrote. ImagePacksChanged says a pack moved somewhere, from a state event, an account data change or a membership change taking a room’s packs with it. it’s an invalidation, and the mirror refetches on it like it does rooms, keeping the packs in its state

verification

Accept, Confirm, Reject and Cancel. a request another device sends shows up as Incoming and waits for Accept, or Cancel to turn it down. one is only taken up while nothing else is in flight. reject tells the other side the emoji didn’t match. the watchers subscribe before they look at the current state, since the sdk only reports what changes after you ask. the watcher and the action paths claim the terminal event, so a racing cancel can’t turn Done into Cancelled

mirror

the mirror is a consumer like any other, behind the mirror feature, and it takes any stream of events rather than the EventStream itself so a host keeps the raw events without a second channel. state, its sequence number and the broadcast send all move under one lock, which is how subscribe gets a snapshot and a receiver with nothing between them. a lagged subscriber starts over from a fresh state under the same lock

nothing on the draining path waits for an engine answer. a resync runs on its own task, retrying with backoff until a reset or a close for its timeline aborts it. rooms are fetched on a task too, one at a time with one more owed at most. a diff for a timeline with no reset is a divergence too, so a wrapper that swallowed the reset still heals. anything a late subscriber needs, failures included, lives in the state rather than only in an update

the sdk fork

we build against a fork of matrix-rust-sdk, branch macltok, cut from the matrix-sdk 0.18.0 tag. turned down on 2026-09-26, taken up on 2026-10-22 once it was holding back features every peer client has. the sdk crates are git dependencies pinned to a fork rev, so dependents get the fork through macltok with no patch block of their own, and macltok stays off crates.io until upstream takes the patches. a dependent naming matrix-sdk itself has to point at the same rev or it gets a second sdk

it carries visibility and config changes, each small enough to be an upstream pr, and nothing that changes how the sdk behaves. if that stops being true, especially in the timeline or the send queue, it’s time to drop matrix-sdk and matrix-sdk-ui for our own layer over matrix-sdk-base and matrix-sdk-crypto, which keep their public sans-io apis

• Room::prepare_encryption, the send path’s encryption prep made public
• SyncServiceBuilder::with_room_list_extra_required_state, for state the sdk’s hard-coded list leaves out. image packs use it
• msc4027 reaction shortcodes, kept on the timeline’s ReactionInfo and sent with Timeline::toggle_reaction_with_shortcode
• SendQueue::enable_global_upload_progress, upload progress on the client-wide send queue channel
• OAuth::pending_authorization and restore_pending_authorization, to finish an oauth login in another process

public model

owned ruma ids and typed data wherever there’s a stable shape, Raw where arbitrary content has to survive. ruma is re-exported as macltok::ruma, so a host parses ids at the version we’re typed against. anything expected to grow is #[non_exhaustive]. Engine::from_client stays public for people who built their own client