macltok

# macltok design decisions

the contract the working tree actually implements. if the code disagrees with

this, one of them is lying and it should be fixed

## two ways in, one way out

state changes are commands. `ClientHandle::send` queues one and hands back a

`Pending`, `execute` queues and waits, and there's an awaited convenience method

for every command. each command answers exactly once with

`Result<CommandOutput, Error>`. drop the `Pending` if you don't care, the

command still runs

pure reads skip the engine and ask the sdk directly through the handle. rooms,

members, power levels, account data, devices, push notification resolution,

and media bytes all live here

the event stream carries state and only state. timelines, history, typing,

rooms, connection, encryption, verification, upload progress, session. no

command results ride on it. drain it from its own task, it's bounded and the

engine waits for you

one error type for everything, `Error { kind, message, challenge, server }`.

match on `kind`. `Interrupted` means the engine stopped, the room closed, or the

task doing the work died before it could answer. `challenge` turns up with

`InteractiveAuthRequired`

`message` is ours and says what we were doing. `server` is the homeserver's

errcode and message, dug out of the source chain when the error is built, so a

host can show "invalid password" without walking the chain. `AccountError`

carries the same, plus a `kind`. neither folds it into `Display`, since anything

printing the chain would say it twice

## engine

the run loop routes and does nothing slow. every task it spawns is registered

with a context. a command task that panics costs its caller an `Interrupted`

and gets logged. a watcher or a room actor that panics takes the engine down,

emitting `Failed` then `Stopped`. natively, anyway. on wasm a panic traps the

task mid poll and nothing is caught. the rest keeps running in an undefined

state and goes silent, no `Failed` and no `Stopped`, so the host hears of it

through `on_panic`, a free function since a panic hook belongs to the whole

instance rather than to one engine

sync restarts itself with backoff after an error or an unexpected stop, and

`ConnectionChanged` shows `Offline` meanwhile. a run that drops again inside

thirty seconds counts as a failure too, or a server that answers `/versions`

but refuses sync would bounce us between online and offline forever. every return to `Online` re-arms

the send queues, since the sdk parks a room's queue on any send failure and

never unparks it. a recoverable failure while online gets the room another go

after a few seconds

logout and a rejected token both stop the engine. `run` resolves to a

`Termination` naming which of those it was, or a shutdown, or every handle

dropping, so the host never works it out from the events. `Shutdown` answers once

`Stopped` has gone out, every one of them if several were queued

`Engine::run` hands back a boxed future. the real one nests deep enough that

every crate spawning it would trip the compiler's query depth

## rooms

one actor per open room. the engine holds a slot per room, `Active` with a

sender or `Closing` with a fifo of opens and closes that arrived while it wound

down. those replay in order once the actor exits. a room you leave, or get

kicked or banned from, closes itself. it tells the engine first, and anything

already queued to it comes back to be answered or replayed, so an open racing a

leave still gets a fresh actor

however an actor ends, short of the engine stopping, its typing goes out empty

and each of its timelines sends `TimelineClosed`. so does a closed thread. a

timeline that never sent a reset never sends a close either

opening sends `TimelineReset` on the stream before it answers. opening an

already open room sends a fresh reset, so a second frontend gets a snapshot

sends, edits and replies run on the actor in order, so they reach the send

queue in the order they were asked for. redactions, reactions and read

receipts are network calls, so they run beside the actor and a slow one

doesn't queue the room behind it. closing lets them finish rather than lie

threads belong to their room's actor. commands for a thread that's still

opening wait behind the open, up to the actor's queue capacity

pagination runs one request at a time per timeline and emits `HistoryChanged`

around each. the sdk remembers where history starts, we don't, because a gappy

sync can unload it again

the actor's watchers are supervised, and a panic in any of them panics the

actor. its command work isn't, a panicking redaction just answers `Interrupted`

and gets logged

## timelines

`TimelineDiff::try_apply` refuses a diff whose indices don't fit your snapshot

and leaves the snapshot alone. `try_apply_all` does the same for a whole batch,

checking every diff against lengths alone first, since that's all a diff's fit

depends on. snapshots are `imbl::Vector`s so keeping one is a pointer copy.

stop applying, `resync`, and carry on from the next `TimelineReset`. resync

kills the old watcher before the new reset goes out, so no stale diff can land

after it

`TimelineId` tells a room from a thread by its root event

every item is an event, a day divider, the read marker or the start. an event

carries its sender with avatar and name ambiguity, and a `TimelineContent`

saying what it actually is. messages keep their msgtype, membership and profile

changes and the common room state changes get their own variants, and anything

else is `OtherState` or `Other` with its type. nothing is flattened into english

in a body field

thread receipts for a thread that isn't open go straight to the room, so

marking a thread read works the same whether its room is on screen or not

replies carry the replied-to event once it's loaded. the watcher asks the sdk

for any it hasn't got, once per event, since the sdk won't on its own

edit, redact and react take an `EventRef`, so your own message can be edited or

reacted to before the server has echoed it back. redacting an unsent one just

cancels it

a custom emote reaction keys on the image's mxc uri and carries its shortcode

(msc4027), so clients without the pack can still label it. `Reaction` says

which image it is and the shortcode any sender gave, and `toggle_reaction`

takes the shortcode to send

## thread lists

a room's thread list is its own thing, not a timeline and not the room actor's.

it needs only the room, so a room needn't be open for its list. one worker holds

every open list. `Open` sends an empty `ThreadListReset`, `Paginate` asks

`/rooms/{id}/threads` through the sdk's `Room::list_threads`, and the list

follows the room's event cache for as long as it's open. changes go out as

upserts keyed by root, which `ThreadListEntry::upsert` puts in order, newest

activity first and ties by root, so a list can't diverge the way an index diff

can

the sdk's `ThreadListService` was passed over. it never adds a thread that wasn't

on a page already loaded, never reorders, and adds one to the count for every

event the cache hands it, replays and back pagination included. here a reply

counts once, by id, when sync just brought it, it's newer than the thread's

latest and it was sent after the count was read off the server, all by the sdk's

corrected timestamp. the latest alone can't date a count. a summary from the

event cache names its latest reply without the event, and continuwuity 26.9

bundles the latest as bare content, so with only that check every reply the

cache re-inserted read as new and a page's count doubled. a newer reply that

doesn't count still becomes the latest, and an entry built without a dated

latest takes the newest reply the event cache holds. replays, reloads from the store and

back pagination never count. a reply sync brings to a thread the list hasn't

got fetches its root, four at a time, and adds it. one from anywhere else

doesn't, since that

says nothing about the thread being active, and a root the server doesn't have

isn't asked for again. a redacted reply doesn't uncount, an edit doesn't change

the latest reply's preview, and a preview's own reply target comes back

`Unavailable` rather than fetched

a lagged listener can't know what it missed, so the list fetches its first page

again and sends a `ThreadListReset` with it. leaving the room, or being kicked

or banned, closes the list with `ThreadListClosed` and `RoomLeft`, the shape an

event context closing on a leave has. `Close` sends one too, with `Requested`,

since a mirror holds the list and has to drop it. a list's media belongs to its

own scope and its ids carry it, like an event context's, bounded the same way so

a long open list sheds the previews its threads moved past, and released on

close

## warming up to send

the sdk does all its key work on the send path, in the room's send queue before

the `/send` itself. members synced, a `/keys/query` for anyone untracked or

dirty, then a `/keys/claim` and the room key sent to every device. over a slow

federation that's a minute of nothing. `prepare_to_send` runs all of it ahead

of time through the fork's `Room::prepare_encryption`, the same function the

send queue calls. the share sits behind the sdk's per-room dedup, so a send

landing mid-warm-up waits for it rather than sharing again, and a send after it

finds nothing left to do but the `/send`. a room whose encryption isn't known

yet gets asked, which the send would have done anyway. one warm-up per room at

a time, a second ask joins it. opening a room or starting to type runs one too,

unless `EngineConfig::prepare_to_send` is off. it's on because malice asked,

not because anyone decided

the session rotates after 100 messages or a week, and sharing isn't a message,

so a warm-up is only wasted on a room left idle about that long

## mutations

membership, moderation, power levels, room creation and account data writes go

through one keyed fifo scheduler. same room, same alias, same dm partner or

same account data type run in order, everything else runs at once.

`create_direct` hands back the existing dm when there is one

notifications and encryption each get one serial worker. the encryption worker

owns the verification controller outright

power level changes build one state event from the local mapping, since the

sdk's own patcher is private

## media

media is a read. `ClientHandle::media` takes a `MediaId`, from a retained

timeline or an avatar, through one bounded

download path with the byte cap the sdk's own download lacks, an hour's total

deadline instead of the sdk client's thirty seconds, and a minute's patience

for a stalled stream. concurrent asks for the same bytes share one flight.

a thumbnail the server won't make falls back to the file itself.

`MediaId` parses from a string, since a web client has to round trip it

the cache holds plaintext, keyed like the sdk keys it, so the sdk and macltok

can read each other's entries. that's how your own encrypted uploads come back.

the sdk keys plain and encrypted sources alike, so a plain event pointing at

someone's encrypted file can leave ciphertext in the cache. an encrypted read

checks the cached bytes against the file's hash and decrypts them if they match

media ids are registered as timelines render and forgotten when their timeline

closes, least recently seen first once the cap is hit

avatars are `MediaId`s too, so a host never handles an mxc uri. they aren't

registered, since they live in the room list and member lists for the whole

session and there can be thousands, and the cap would evict ones still on

screen. a plain avatar carries no keys, so its id is its mxc uri and resolves

without the registry. it never expires, and an mxc uri a host got some other way

parses into one. `set_avatar` still takes an mxc uri

streaming decryption inside macltok was declined. in a browser the service

worker answering range requests is a separate context from the worker running

macltok, and streaming through macltok would carry every chunk through wasm

memory, the very thing the host streams to avoid. `media_source` hands over the

url and keys `media` would have used, picked the same way, and `access_token`

the rest. the byte cap doesn't apply, nothing is buffered

uploads are commands that answer once the send queue has them. the fork puts

upload progress on the client-wide send queue channel too, which upstream keeps

for each room's own, so one watcher there follows every attachment in the queue,

leftovers the sdk resumes from last time included, and ends each with one

`MediaUploadFinished`. sent, cancelled, its room forgotten, or

missed when that channel overflowed at the wrong moment, since then there's no

knowing which. a failure isn't an ending, a wedged upload can still be retried.

uploads don't belong to a room's actor, so closing the room leaves them be

that watcher is the only thing speaking about uploads. anything that ever

showed progress or a failure is followed until it gets exactly one

`MediaUploadFinished`, and nothing about it goes out after. the sdk sends

progress from a detached task, so a tick can turn up behind its own upload's

end, and the watcher remembers the last few hundred ends to drop those. the

engine stopping ends nothing, `Stopped` covers it

## sessions and secrets

`Session::login_password`, `begin_oauth` and `restore` share one client builder

with an encrypted store, refresh tokens and threads. the store is sqlite

natively and indexeddb in a browser. secret-bearing inputs don't derive `Clone`

the homeserver a person typed is checked before the client sees it, and a bad

one is `AccountError::InvalidHomeserver`. a bare server name passes untouched,

since that's what sends the sdk to `.well-known`, and a host that "helpfully"

adds `https://` turns `example.org` into a homeserver url and skips discovery.

a url needs a host, no userinfo, and https. plain http needs

`allow_insecure_loopback` and a loopback host. `SoftLogout::password_login` sets

it, since its homeserver passed once already

a `SessionStore` owns one login's record and its store. login saves the first

record, `restore` loads it, and the engine a `Session` builds saves every token

rotation after that, since a refresh token works once and a host that missed one

save has lost the login. the session changes are subscribed the moment the

client exists, because the sdk can refresh during restore before any engine

does. a save runs off the event stream, so a full stream never holds it back,

and `SessionTokensChanged` only goes out once it has landed, or

`SessionSaveFailed` if it didn't. natively the sdk's save callback runs inside

the refresh, before the new token is used, and `save_blocking` writes there and

then. in a browser there's only the async save, so a tab that dies between a

refresh and its save wakes up holding a spent token. a fork patch letting the

sdk await an async save before it uses the new token would close that, but it

changes how a refresh behaves rather than what's visible, and on 2026-10-22 we

held off so the fork doesn't drift that far that fast. upstream taking an async

save callback, or the fork already carrying behaviour changes, reopens it.

`without_persistence`

hands the job back to the host. `EncryptedSessionStore` seals the record with

the sdk's `StoreCipher` under the store's passphrase and replaces it by rename,

so a crash mid save leaves the old one. it keeps the record out of the sdk's

state store because sqlite writes custom values there in the clear, and a

client can't open its store before it knows the homeserver the record holds.

`discard` deletes the record, then the store, since a record without its store

would restore into empty crypto. the host follows `Termination::retain`.

`Nothing` discards. `Store` calls `soft_log_out`, which keeps the store and

swaps the record for one holding the homeserver, user, device and oauth client

id with no tokens. `restore` hands that back as `Restore::SoftLoggedOut`, so a

restart still knows which device to log in again as, and the login overwrites

it. one store directory per login. a login whose first save fails comes back as

`AccountError::Unsaved` holding the live session, so the host retries the save

rather than spending another device

`Accounts` picks the store, so a host never decides which account a login form

means. it's one owner's accounts under one root and one passphrase, a store and

record each, named by a random id. a password login that names a

soft-logged-out account, by localpart or full user id on the same server, logs

back in as that device into that store. the server is compared by name as well

as by url, since a typed `example.org` is stored as wherever discovery led. a

live account it names is `AlreadyLoggedIn`, anyone else gets a new store. oauth

can't know who it's for until it finishes, so it resumes only the device

`SoftLogout::oauth_login` names. a new store is locked before it's listed, so

`list` can sweep stores with no record, left by a login that died or a discard

that didn't finish, without catching one mid login. a browser lists accounts in

an IndexedDB database of their own, a key each, so two tabs adding at once

don't lose one

the set is per owner because resuming needs only the password, and the store

holds the device's e2ee keys. a set shared across people would hand someone who

knows a password the history a fresh login wouldn't get

one engine per store. before the client opens the store we take the platform's

own exclusive lock on it, `File::try_lock` on `macltok.lock` in the store

directory natively and a Web Lock named after the store in a browser. the

engine and every `ClientHandle` share it, so it goes when the last of them

does, since a lingering handle still has the client and its store open. anyone

else, in this process, another one or another tab, gets `StoreInUse` at once,

so the host decides whether to wait, hand over or say so. the os or the browser

lets go when the holder dies. a SharedWorker serving every tab from one engine

never contends for it. Web Locks only exist in a secure context, so a browser

store needs one too

the sdk's cross process lease can't do this job. in matrix-sdk-common 0.18

`CrossProcessLock::try_lock_once` (`cross_process_lock.rs` 389-448) leases for

500ms and renews every 50ms from a spawned task, which gives up for good on one

failed renewal or on finding the lease lapsed and retaken, and tells no one. a

stall past half a second, ordinary in a tab and certain during the record's

PBKDF2, or one `SQLITE_BUSY`, hands the store to a second engine while the

first still thinks it's alone. with our lock keeping everyone else out, clients

run with `CrossProcessLockConfig::SingleProcess` and the sdk's own sections skip

the lease writes and renewal tasks

a pending OAuth login survives the page going away for its redirect.

`begin_oauth` saves it as the store's record, sealed like any other since it

holds the PKCE verifier, through the fork's `OAuth::pending_authorization`.

`Accounts::finish_oauth` takes the callback url and finds the account by its

`state`, so the host keeps nothing across the redirect, and

`Session::finish_oauth` does the same for one store. a host that keeps the

process alive can still use `PendingOAuthLogin::finish`. while it waits,

`restore` answers `Restore::OAuthPending`. the record lives in the store

rather than with the host because `list` sweeps a store with no record, which

would delete a fresh login's store mid redirect

the pending record keeps whatever the store held before, so a login resuming a

soft-logged-out device that never comes back puts that device back, and a

password login can still resume it meanwhile. abort, a failed finish after the

redirect and an hour's wait all undo the login, a new account's store deleted

and an old record put back. a failed `PendingOAuthLogin::finish` still holds

the client, so it puts back an old record and leaves a new one to the hour.

codes live minutes, so an hour is generous

## room lists, spaces and profiles

`RoomSummary` has what a room list row needs without a second query. topic,

alias, member counts, heroes for nameless rooms, the inviter, where an upgraded

room went, a latest event preview, the joined spaces listing it, and for a

space the children it lists

parents come from one lazily built `SpaceService` shared by every handle.

children come straight from the space's local state, joined or not.

`space_hierarchy` pages through the server's view for browsing rooms you

haven't joined, and holds its place between pages

`own_profile` and `user_profile` read profiles. name and avatar changes are

commands, and uploading an avatar gets the same byte cap as any upload

## notifications and room data

the notification client is built lazily and shares the engine's sync service.

sdk ui types stay behind the boundary

`rooms` normalises on demand with bounded concurrency and sorts by latest event,

then name, then id. `RoomsChanged` is an invalidation, coalesced so one sync

touching fifty rooms is one event. a sync's room info lands before the event

cache has stored its events, so the watcher waits a tenth of a second after

anything wakes it and takes the lot as one. an event cache slower than that

costs a second one. a left room stays in the list with its

membership until it's forgotten. push rule changes never touch room info, so

`NotificationSettingsChanged` brings a `RoomsChanged` along for the modes in

the summaries

with threads on, the sdk leaves thread events out of a room's counts and keeps

none of its own for them. `thread_unreads` counts them per root from the event

cache store, which keeps every reply it ever took whatever it holds in memory,

so closing a room, the cache trimming itself or a restart leaves them be. a

reply counts once it's newer than a receipt for its thread (msc3771), an

unthreaded one, or the user's own reply. a main-timeline receipt doesn't reach

into threads. the store has lost stream order between a thread and the rest of

the room, so an event's timestamp stands for its place, and a receipt on an

event the store never had falls back to the receipt's own time. a thread's

mentions also count in the room's `unread_mentions`, so nobody misses one,

while its messages and notifications stay apart, since they don't move the main

timeline's read marker

the counts are kept in the state store per room, only for threads with

something unread, so `rooms` reads them without touching the event cache and a

cold room never gets one made for it. thread events and threaded receipts leave

room info alone, so the room list watcher recounts on its own. a synced thread

event names the thread to count, an event cache update recounts the room's

threads already holding something unread, since a late decryption can turn up

a mention, and so does a receipt of the user's own. counting in the sdk would

be cheaper, but that's a behaviour change in the fork

## link previews

a text message can carry its sender's own previews (msc4095), and those come

out on `Message::link_previews` without asking anyone, so they work in

encrypted rooms and leak nothing. an encrypted preview image is registered like

an attachment. a preview whose url isn't in the body is dropped, or a sender

could caption any link with any card

`url_preview` asks the homeserver, which fetches the page, re-hosts `og:image`

and answers with loose `og:` keys. that tells the homeserver every url, so

whether encrypted rooms ask is the host's call. answers are kept an hour and

misses ten minutes, and asks for one url at once share a request, since synapse

rate limits the endpoint and a timeline repeats links. synapse answers a page

it couldn't download with a 500, a dns failure with a 502 and its blocklist with

a 403, so those, 400 and 404 mean no preview. a genuine 500 hides in there too,

but a missing card is cheap. a

server that doesn't preview at all says so with `M_UNRECOGNIZED` once and isn't

asked again. a 429 anywhere is `ErrorKind::RateLimited`. page text from either

source is trimmed, 256 characters for a title or site name and 1024 for a

description, and an image that isn't an mxc uri is dropped

`url_previews_allowed` follows element, so both agree. the user's own choice

for the room, then their account wide default, then room state an admin set.

the room state comes in with the rest through the fork's extra required state

## image packs

msc2545 packs come from the user's account data, from room state, and from the

rooms the user enabled everywhere. the room list asks the server for every

room's `im.ponies.room_emotes`, through the fork's extra required state, so

packs sit in the store like any other state and a change reaches us even across

a gap in the timeline. that's every joined room's packs on the first sync,

tens of kilobytes for a big one, and only changes after. asking only for open

rooms would lose the enabled packs of rooms nobody has open

`image_packs` reads all of it from the store and hands back the lot, global

packs and each room's own, and `ImagePacks::for_room` puts a picker's list in

order. an enabled room the user hasn't joined is skipped, there's no state for

it. one bad image costs that image and a pack with nothing usable is dropped,

since pack contents are whatever someone's client wrote. `ImagePacksChanged`

says a pack moved somewhere, from a state event, an account data change or a

membership change taking a room's packs with it. it's an invalidation, and the

mirror refetches on it like it does rooms, keeping the packs in its state

## verification

`Accept`, `Confirm`, `Reject` and `Cancel`. a request another device sends

shows up as `Incoming` and waits for `Accept`, or `Cancel` to turn it down. one

is only taken up while nothing else is in flight. reject tells the other side

the emoji didn't match. the watchers subscribe before they look at the current

state, since the sdk only reports what changes after you ask. the watcher and

the action paths claim the terminal event, so a racing

cancel can't turn `Done` into `Cancelled`

## mirror

the mirror is a consumer like any other, behind the `mirror` feature, and it

takes any stream of events rather than the `EventStream` itself so a host keeps

the raw events without a second channel. state, its sequence number and the

broadcast send all move under one lock, which is how `subscribe` gets a

snapshot and a receiver with nothing between them. a lagged subscriber starts

over from a fresh state under the same lock

nothing on the draining path waits for an engine answer. a resync runs on its

own task, retrying with backoff until a reset or a close for its timeline

aborts it. rooms are fetched on a task too, one at a time with one more owed at

most. a diff for a timeline with no reset is a divergence too, so a wrapper

that swallowed the reset still heals. anything a late subscriber needs, failures

included, lives in the state rather than only in an update

## search

`search_messages` takes words and a scope of one room, a list of them or every

joined room, and answers newest first. it opens a search the way a jump opens an

event context, a session owning the media of every hit it hands out under its

own bounded scope, paged with `search_more` and its cursor and freed with

`close_search`. at most eight stay open and opening another closes the oldest,

so a host that never closes anything still holds a bounded amount, and the

closed one goes out as `SearchClosed`. a hit's media ids carry their search's

owner prefix like a context's, and stop resolving once it closes. encrypted

rooms can't use the server's `/search`, so everything goes through one local

index, unencrypted

rooms too, so results and coverage mean the same thing everywhere. a hit comes

back the way event context shows its target, latest edit applied, with byte

ranges of the matched words in its text

the index is tantivy, but not the fork's matrix-sdk-search. that crate keeps its

schema private, keeps an index per room, opens a fresh `IndexWriter` for every

event, ranks by score, hands back bare event ids and turns mmap on, which wasm

can't build. `IndexWriter` can't run in wasm either, since it spawns indexing

threads and rayon pools. so macltok keeps its own segments. each flush writes an

immutable segment with `SingleSegmentIndexWriter` into a `RamDirectory`, edits

and redactions are tombstones in a bitset beside it, and segments of a size

class merge through `merge_filtered_segments` once six pile up, dropping the

tombstoned docs as they go. all of that is single threaded with the docstore's

dedicated compressor off. message text is indexed and not stored, a preview

reads the event back. the one exception is an edit waiting for its original,

whose text is kept so the original can take it over. a 2026-10-25 spike ran the

whole cycle in a headless firefox worker

it lives with the store, so it's per login and never synced, and it's only

opened for the user and homeserver that made it. natively it's

`macltok-search.sqlite3` beside the sdk's databases, created 0600, and in a

browser the `::macltok-search` IndexedDB database, and both go when the store is

deleted. every segment, every tombstone generation, the manifest and the waiting

undecryptable events are blobs sealed by a `StoreCipher` under the store

passphrase, stored under a keyed hash of their name like the mentions rows, the

name sealed inside so blobs can't be swapped. the manifest a commit saves is the

index as it stands once that commit lands, and blobs it doesn't point at are

dropped on the next load. one commit is one transaction, sqlite in wal with

normal sync and IndexedDB relaxed, so a crash can lose the last few seconds of

commits but never leaves half of one. the working copy lives in memory, all of

it, which is the price of tantivy's directory being synchronous and IndexedDB

not. a store that won't open or read, or five commits failing in a row, means

an index in memory for the rest of the engine, and coverage says

`persistent: false` with the reason in `in_memory`

it's fed from what the client sees. each joined room's event cache is followed

from its first update, so sync, pagination and the redecryptor's replacements

all arrive as diffs, and a room idle for five minutes is let go so the cache can

shrink. the generic update channel is followed from before the index loads, and

if it lags every joined room is followed, since a lost update could be a room's

only one. an edit lands on its original under the original's key, the latest by

the original's sender winning. one that arrives first waits under its target and

its own sender, out of results, until the original turns up and takes the

author's edit, so a stranger's early edit can't crowd it out. an edit of an edit

is ignored, and an edit to content we can't read clears the old text. a

redaction drops the key and is remembered for a while, so a stale copy can't

bring it back. redacting the latest edit reads the original again with whatever

edits of it are still around. a message still encrypted waits, raw, up to two

thousand of them in sixteen blobs, and is decrypted eight at a time when a room

key for its session arrives

backfill is `/messages` straight from the room, not through the event cache,

so it never grows a timeline someone has open, filtered to messages, encrypted

events, redactions, stickers and polls. one request at a time across every

room, two seconds apart, a hundred events a page and ten pages a room per

engine run, round robin. a room walks back from the newest event to the start,

and resumes from its saved token next time. a room that's been indexed before

also gets a head pass the first time it shows up in a run, from the newest event

back until it meets something that was already indexed when the engine started,

which picks up what happened while we were offline. a failing room backs off on

its own from a minute to fifteen. a saved token the server turns down with a 400

is dropped and the room starts again from the newest. leaving a room cancels its

page. all of that is `EngineConfig`, and zero pages a room turns backfill off

coverage is per room, the oldest and newest indexed message, whether backfill

reached the start, and where backfill is at, plus the undecryptable messages in

scope. it comes with every page

the analyser makes a token of every run of letters and digits up to 64 chars,

so hashes and long words fit, and lowercases and folds accents, so `cafe` finds

`Café`. han, kana, hangul and thai have no spaces to split on, so every char is

a token and so is every pair of neighbours, and a query for a run needs all its

pairs. that finds a word inside a sentence at the cost of the odd match across a

word boundary. emoji are tokens too. queries have no field syntax, so `re:` or a

url is just its words. words all have to match, quotes make a phrase and `-`

excludes, and a query with nothing left to find is `Invalid` rather than

everything in scope. there's no stemming

a hit is read back when it's shown, `load_or_fetch_event` for the original and

for its latest edit, which is the event cache and its store first and the

server's `/event` after, eight at a time. the original's bundled edit is ignored,

since the index already knows the newest one. one that's turned out redacted is

dropped from the page and from the index. one that can't be read at all, gone

from the cache with the server unreachable or refusing, is left out of that page

but stays indexed, so it shows up again once it can be read. the cursor still

moves past it

tantivy's merger times itself with `Instant::now()` whenever a `log` logger is on

at debug, and that panics in wasm. on 2026-10-25 we capped `log` at info at

compile time in wasm, `max_level_info` and `release_max_level_info` both, since

in release the release cap wins. it's all or nothing for the whole wasm build, so

a host gives up `log` debug and trace from every crate, not just tantivy, and

asking for any other cap is log's own "multiple max_level_* features set" error.

tantivy timing with something wasm can call, or reading `Instant` only behind

its own feature, reopens it

## the sdk fork

we build against a fork of matrix-rust-sdk, branch `macltok`, cut from the

matrix-sdk 0.18.0 tag. turned down on 2026-09-26, taken up on 2026-10-22 once

it was holding back features every peer client has. the sdk crates are git

dependencies pinned to a fork rev, so dependents get the fork through macltok

with no patch block of their own, and macltok stays off crates.io until

upstream takes the patches. a dependent naming matrix-sdk itself has to point

at the same rev or it gets a second sdk

it carries visibility and config changes, each small enough to be an upstream

pr, and nothing that changes how the sdk behaves. if that stops being true,

especially in the timeline or the send queue, it's time to drop matrix-sdk and

matrix-sdk-ui for our own layer over matrix-sdk-base and matrix-sdk-crypto,

which keep their public sans-io apis

- `Room::prepare_encryption`, the send path's encryption prep made public

- `SyncServiceBuilder::with_room_list_extra_required_state`, for state the

  sdk's hard-coded list leaves out. image packs use it

- msc4027 reaction shortcodes, kept on the timeline's `ReactionInfo` and sent

  with `Timeline::toggle_reaction_with_shortcode`

- `SendQueue::enable_global_upload_progress`, upload progress on the

  client-wide send queue channel

- `OAuth::pending_authorization` and `restore_pending_authorization`, to

  finish an oauth login in another process

## public model

owned ruma ids and typed data wherever there's a stable shape, `Raw` where

arbitrary content has to survive. ruma is re-exported as `macltok::ruma`, so a

host parses ids at the version we're typed against. anything expected to grow is

`#[non_exhaustive]`. `Engine::from_client` stays public for people who built

their own client