macltok

macltok

macltok is a compact Rust application backend built above matrix-sdk and matrix-sdk-ui. It aims to provide the stateful behavior needed by a useful Matrix client without depending on a GUI toolkit or inventing another protocol SDK.

The crate currently provides

• typed commands, owned UI-safe events and bounded per-client transport
• Sliding Sync with normalized connection and room-list state
• independently retained live room timelines and backwards pagination
• text, emote, reply, edit, redaction, reaction, receipt and typing operations
• raw encrypted or unencrypted media acquisition with configurable byte limits
• encryption state, recovery-key import and SAS verification
• password login and session restoration with caller-owned encrypted SQLite or IndexedDB stores

Matrix room, event, user, device and transaction identifiers remain strongly typed. A toolkit timeline identity covers local echoes without pretending they already have remote event IDs.

Consuming the engine

spawn Engine::run, drain the EventStream from its own task, and talk to the engine through ClientHandle. every command answers once, awaited (handle.send_message(..).await?) or as a Pending you can await later or drop. a sent message answers when the sdk send queue has it, delivery shows up as the timeline item’s send state. reads like rooms, members and media go straight to the sdk. the stream only carries state

some commands put an event on the stream before they answer. Open, Close, OpenThread, CloseThread, Resync, Paginate, Logout and the verification actions all do. the stream is bounded, so a task that drains it and awaits one of those answers deadlocks once it fills. from the draining task, try_send the command and drop the Pending, the events tell you how it went

prepare_to_send does the key work a send into an encrypted room would start with, ahead of time, so the send itself is quicker. the engine runs it itself when a room opens or set_typing(room, true) comes in, unless EngineConfig::with_prepare_to_send(false). only part of that work is reachable without a matrix-sdk fork, docs/design-decisions.md says which

start a timeline from TimelineReset, then apply each TimelineChanged whole with TimelineDiff::try_apply_all. snapshots are imbl::Vectors, re-exported as macltok::imbl, so a copy costs nothing. when a batch doesn’t fit none of it lands. stop applying, call resync, and pick up from the next TimelineReset

media answers with the whole file, decrypted and checked, which is wrong for a long video. media_source says where the same bytes live instead, the url and, for an encrypted file, its AES-CTR key, iv and hash. access_token is what to send with it. a host can then fetch and decrypt ranges itself, but the hash covers the whole ciphertext, so a range read plays bytes nobody checked

with sqlite or indexeddb, a SessionStore holds one login’s record and its store. EncryptedSessionStore seals the record with the store’s passphrase, in a file you pick natively or an IndexedDB database beside the store in a browser. logging in saves the first record and Session::restore loads it. from then on the engine the Session builds keeps it current. every token rotation is saved before SessionTokensChanged goes out, and a save that fails says so with SessionSaveFailed. natively EncryptedSessionStore writes from inside the sdk’s refresh, before the new refresh token is ever used, and a store of your own gets the same by implementing SessionStore::save_blocking. in a browser the save can only follow the refresh, so a tab killed in between wakes with a spent token. Session::without_persistence hands the job back, and then it’s yours to save ClientHandle::session_record() on every SessionTokensChanged

Engine::run resolves to a Termination saying why it stopped, and Termination::retain says what to do about it. after LoggedOut or AccessRevoked the record and the store are dead, discard them once the client is dropped, since an open store won’t go quietly. SoftLoggedOut wants the store kept and a fresh login as the same device, or the e2ee keys go with it. soft_log_out swaps the record for one naming that device, so a restart gets Restore::SoftLoggedOut and SoftLogout::password_login rather than nothing, and the login overwrites it with a live record. ShutDown and HandlesDropped keep everything. an Err is a real failure and kills nothing. one store directory per login, the sdk won’t share a crypto store between devices

Accounts does that bookkeeping. it holds one person’s accounts under one root, login_password and begin_oauth pick the store, resuming a soft-logged-out device when the login names it, and list hands back an Account each to restore and to apply Termination::retain through sessions

The mirror

the loop above is easy to get subtly wrong, so the mirror feature ships one. Mirror::new takes a ClientHandle and the events, and Mirror::run drains them into one State with the rooms, every open timeline, connection, encryption, verification, typing, uploads and where the session stands. spawn it the way you spawn Engine::run. it takes any stream of events, so wrap the EventStream if you want to watch the raw ones go past

MirrorHandle is cheap to clone, one per frontend. snapshot hands back the current State, and subscribe opens with the whole State and then every Update after it, numbered, with nothing missed and nothing twice. a timeline update carries its diffs and the items they left, so apply the diffs or re-render the list, whichever suits. a subscriber that falls behind gets a fresh State instead of an error

a batch that doesn’t fit leaves the timeline diverged at its last good items, and so does a diff for a timeline it never saw reset. the mirror asks for the resync itself, off the draining path, and keeps asking until the reset or a close turns up. rooms are refetched on RoomsChanged, one fetch at a time with at most one more owed

failures stay in the State, so a late subscriber sees them too. what stopped the engine sits in engine_failure, and a failed save of the session record in session_save_failure until the next one works. the mirror never touches the store

wasm

macltok builds for wasm32-unknown-unknown. the indexeddb feature gives you StoreConfig::indexeddb in place of StoreConfig::sqlite, and asking for either on the wrong target won’t compile. Engine::run hands back a LocalBoxFuture there, so spawn it with wasm_bindgen_futures::spawn_local

getrandom picks its browser backend from a cfg flag, and .cargo/config.toml only reaches this crate’s own builds. every cdylib that pulls macltok in needs it too

[target.wasm32-unknown-unknown]
rustflags = ["--cfg", "getrandom_backend=\"wasm_js\""]

what’s different in a browser

• a panic doesn’t abort anything. it traps the task mid poll and the rest of the engine carries on in an undefined state and goes quiet, no Failed and no Stopped. register macltok::on_panic and have it tear the worker down and start another. natively a panicking command just costs its caller an Interrupted
• MediaUploadProgress only ever reports completion, the sdk’s wasm http path has no upload progress
• the sdk doesn’t retry requests or time them out individually. sync still restarts itself with backoff, that part is ours
• an OAuth login only finishes in the process that began it, since the sdk keeps its pending state in memory. run it in a popup, or in a worker that outlives the redirect, not by navigating the page away
• one engine per store. a second tab or worker opening a store that’s in use gets StoreInUse straight away, so serve many tabs from one engine in a SharedWorker rather than one each. the store’s Web Lock needs a secure context
• discard while another tab still has the store open answers StoreDeletePending. the delete lands once that tab lets go, and reopening the store waits behind it

Dependency policy

UI frameworks, image decoders, application-directory discovery and logging subscribers do not belong in this crate. SQLite remains optional through the sqlite and bundled-sqlite features, IndexedDB through indexeddb. Media decoding belongs to the consumer.

Verification

nix develop --command cargo fmt -- --check
nix develop --command cargo test --all-targets
nix develop --command cargo test --all-targets --features sqlite
nix develop --command cargo clippy --all-targets -- -D warnings
nix develop --command cargo clippy --all-targets --features sqlite -- -D warnings
nix flake check

--all-features doesn’t build and isn’t meant to. sqlite and indexeddb each belong to one target and refuse the other