macltok

# macltok

`macltok` is a compact Rust application backend built above `matrix-sdk` and

`matrix-sdk-ui`. It aims to provide the stateful behavior needed by a useful

Matrix client without depending on a GUI toolkit or inventing another protocol

SDK.

The crate currently provides

- typed commands, owned UI-safe events and bounded per-client transport

- Sliding Sync with normalized connection and room-list state

- independently retained live room timelines and backwards pagination

- text, emote, reply, edit, redaction, reaction, receipt and typing operations

- raw encrypted or unencrypted media acquisition with configurable byte limits

- encryption state, recovery-key import and SAS verification

- password login and session restoration with caller-owned encrypted SQLite or

  IndexedDB stores

Matrix room, event, user, device and transaction identifiers remain strongly

typed. A toolkit timeline identity covers local echoes without pretending they

already have remote event IDs.

## Consuming the engine

spawn `Engine::run`, drain the `EventStream` from its own task, and talk to the

engine through `ClientHandle`. every command answers once, awaited

(`handle.send_message(..).await?`) or as a `Pending` you can await later or

drop. a sent message answers when the sdk send queue has it, delivery shows up

as the timeline item's send state. reads like `rooms`, `members` and `media`

go straight to the sdk. the stream only carries state

some commands put an event on the stream before they answer. `Open`, `Close`,

`OpenThread`, `CloseThread`, `Resync`, `Paginate`, `Logout` and the

verification actions all do. the stream is bounded, so a task that drains it

and awaits one of those answers deadlocks once it fills. from the draining

task, `try_send` the command and drop the `Pending`, the events tell you how it

went

`prepare_to_send` does the key work a send into an encrypted room would start

with, ahead of time, so the send itself is quicker. the engine runs it itself

when a room opens or `set_typing(room, true)` comes in, unless

`EngineConfig::with_prepare_to_send(false)`. only part of that work is reachable

without a matrix-sdk fork, `docs/design-decisions.md` says which

start a timeline from `TimelineReset`, then apply each `TimelineChanged` whole

with `TimelineDiff::try_apply_all`. snapshots are `imbl::Vector`s, re-exported

as `macltok::imbl`, so a copy costs nothing. when a batch doesn't fit none of it

lands. stop applying, call `resync`, and pick up from the next `TimelineReset`

`media` answers with the whole file, decrypted and checked, which is wrong for a

long video. `media_source` says where the same bytes live instead, the url and,

for an encrypted file, its AES-CTR key, iv and hash. `access_token` is what to

send with it. a host can then fetch and decrypt ranges itself, but the hash

covers the whole ciphertext, so a range read plays bytes nobody checked

with `sqlite` or `indexeddb`, a `SessionStore` holds one login's record and its

store. `EncryptedSessionStore` seals the record with the store's passphrase, in a

file you pick natively or an IndexedDB database beside the store in a browser.

logging in saves the first record and `Session::restore` loads it. from then on

the engine the `Session` builds keeps it current. every token rotation is saved

before `SessionTokensChanged` goes out, and a save that fails says so with

`SessionSaveFailed`. natively `EncryptedSessionStore` writes from inside the

sdk's refresh, before the new refresh token is ever used, and a store of your

own gets the same by implementing `SessionStore::save_blocking`. in a browser

the save can only follow the refresh, so a tab killed in between wakes with a

spent token. `Session::without_persistence` hands the job back, and then it's

yours to save `ClientHandle::session_record()` on every `SessionTokensChanged`

`Engine::run` resolves to a `Termination` saying why it stopped, and

`Termination::retain` says what to do about it. after `LoggedOut` or

`AccessRevoked` the record and the store are dead, `discard` them once the

client is dropped, since an open store won't go quietly. `SoftLoggedOut` wants

the store kept and a fresh login as the same device, or the e2ee keys go with

it. `soft_log_out` swaps the record for one naming that device, so a restart

gets `Restore::SoftLoggedOut` and `SoftLogout::password_login` rather than

nothing, and the login overwrites it with a live record. `ShutDown` and

`HandlesDropped` keep everything. an `Err` is a real failure and kills nothing.

one store directory per login, the sdk won't share a crypto store between

devices

`Accounts` does that bookkeeping. it holds one person's accounts under one root,

`login_password` and `begin_oauth` pick the store, resuming a soft-logged-out

device when the login names it, and `list` hands back an `Account` each to

`restore` and to apply `Termination::retain` through `sessions`

## The mirror

the loop above is easy to get subtly wrong, so the `mirror` feature ships one.

`Mirror::new` takes a `ClientHandle` and the events, and `Mirror::run` drains

them into one `State` with the rooms, every open timeline, connection,

encryption, verification, typing, uploads and where the session stands. spawn

it the way you spawn `Engine::run`. it takes any stream of events, so wrap the

`EventStream` if you want to watch the raw ones go past

`MirrorHandle` is cheap to clone, one per frontend. `snapshot` hands back the

current `State`, and `subscribe` opens with the whole `State` and then every

`Update` after it, numbered, with nothing missed and nothing twice. a timeline

update carries its diffs and the items they left, so apply the diffs or

re-render the list, whichever suits. a subscriber that falls behind gets a fresh

`State` instead of an error

a batch that doesn't fit leaves the timeline diverged at its last good items,

and so does a diff for a timeline it never saw reset. the mirror asks for the

resync itself, off the draining path, and keeps asking until the reset or a

close turns up. rooms are refetched on `RoomsChanged`, one fetch at a time with

at most one more owed

failures stay in the `State`, so a late subscriber sees them too. what stopped

the engine sits in `engine_failure`, and a failed save of the session record in

`session_save_failure` until the next one works. the mirror never touches the

store

## wasm

macltok builds for `wasm32-unknown-unknown`. the `indexeddb` feature gives you

`StoreConfig::indexeddb` in place of `StoreConfig::sqlite`, and asking for either

on the wrong target won't compile. `Engine::run` hands back a `LocalBoxFuture`

there, so spawn it with `wasm_bindgen_futures::spawn_local`

getrandom picks its browser backend from a cfg flag, and `.cargo/config.toml`

only reaches this crate's own builds. every cdylib that pulls macltok in needs

it too

```toml

[target.wasm32-unknown-unknown]

rustflags = ["--cfg", "getrandom_backend=\"wasm_js\""]

```

what's different in a browser

- a panic doesn't abort anything. it traps the task mid poll and the rest of

  the engine carries on in an undefined state and goes quiet, no `Failed` and no

  `Stopped`. register `macltok::on_panic` and have it tear the worker down and

  start another. natively a panicking command just costs its caller an

  `Interrupted`

- `MediaUploadProgress` only ever reports completion, the sdk's wasm http path

  has no upload progress

- the sdk doesn't retry requests or time them out individually. sync still

  restarts itself with backoff, that part is ours

- an OAuth login only finishes in the process that began it, since the sdk

  keeps its pending state in memory. run it in a popup, or in a worker that

  outlives the redirect, not by navigating the page away

- one engine per store. a second tab or worker opening a store that's in use

  gets `StoreInUse` straight away, so serve many tabs from one engine in a

  SharedWorker rather than one each. the store's Web Lock needs a secure

  context

- `discard` while another tab still has the store open answers

  `StoreDeletePending`. the delete lands once that tab lets go, and reopening

  the store waits behind it

## Dependency policy

UI frameworks, image decoders, application-directory discovery and logging

subscribers do not belong in this crate. SQLite remains optional through the

`sqlite` and `bundled-sqlite` features, IndexedDB through `indexeddb`. Media

decoding belongs to the consumer.

## Verification

```sh

nix develop --command cargo fmt -- --check

nix develop --command cargo test --all-targets

nix develop --command cargo test --all-targets --features sqlite

nix develop --command cargo clippy --all-targets -- -D warnings

nix develop --command cargo clippy --all-targets --features sqlite -- -D warnings

nix flake check

```

`--all-features` doesn't build and isn't meant to. `sqlite` and `indexeddb` each

belong to one target and refuse the other