macltok design decisions
the contract the working tree actually implements. if the code disagrees with this, one of them is lying and it should be fixed
two ways in, one way out
state changes are commands. ClientHandle::send queues one and hands back a
Pending, execute queues and waits, and there’s an awaited convenience method
for every command. each command answers exactly once with
Result<CommandOutput, Error>. drop the Pending if you don’t care, the
command still runs
pure reads skip the engine and ask the sdk directly through the handle. rooms, members, power levels, account data, devices, push notification resolution, and media bytes all live here
the event stream carries state and only state. timelines, history, typing, rooms, connection, encryption, verification, upload progress, session. no command results ride on it. drain it from its own task, it’s bounded and the engine waits for you
one error type for everything, Error { kind, message, challenge, server }.
match on kind. Interrupted means the engine stopped, the room closed, or the
task doing the work died before it could answer. challenge turns up with
InteractiveAuthRequired
message is ours and says what we were doing. server is the homeserver’s
errcode and message, dug out of the source chain when the error is built, so a
host can show “invalid password” without walking the chain. AccountError
carries the same, plus a kind. neither folds it into Display, since anything
printing the chain would say it twice
engine
the run loop routes and does nothing slow. every task it spawns is registered
with a context. a command task that panics costs its caller an Interrupted
and gets logged. a watcher or a room actor that panics takes the engine down,
emitting Failed then Stopped. natively, anyway. on wasm a panic traps the
task mid poll and nothing is caught. the rest keeps running in an undefined
state and goes silent, no Failed and no Stopped, so the host hears of it
through on_panic, a free function since a panic hook belongs to the whole
instance rather than to one engine
sync restarts itself with backoff after an error or an unexpected stop, and
ConnectionChanged shows Offline meanwhile. a run that drops again inside
thirty seconds counts as a failure too, or a server that answers /versions
but refuses sync would bounce us between online and offline forever. every return to Online re-arms
the send queues, since the sdk parks a room’s queue on any send failure and
never unparks it. a recoverable failure while online gets the room another go
after a few seconds
logout and a rejected token both stop the engine. run resolves to a
Termination naming which of those it was, or a shutdown, or every handle
dropping, so the host never works it out from the events. Shutdown answers once
Stopped has gone out, every one of them if several were queued
Engine::run hands back a boxed future. the real one nests deep enough that
every crate spawning it would trip the compiler’s query depth
rooms
one actor per open room. the engine holds a slot per room, Active with a
sender or Closing with a fifo of opens and closes that arrived while it wound
down. those replay in order once the actor exits. a room you leave, or get
kicked or banned from, closes itself. it tells the engine first, and anything
already queued to it comes back to be answered or replayed, so an open racing a
leave still gets a fresh actor
however an actor ends, short of the engine stopping, its typing goes out empty
and each of its timelines sends TimelineClosed. so does a closed thread. a
timeline that never sent a reset never sends a close either
opening sends TimelineReset on the stream before it answers. opening an
already open room sends a fresh reset, so a second frontend gets a snapshot
sends, edits and replies run on the actor in order, so they reach the send queue in the order they were asked for. redactions, reactions and read receipts are network calls, so they run beside the actor and a slow one doesn’t queue the room behind it. closing lets them finish rather than lie
threads belong to their room’s actor. commands for a thread that’s still opening wait behind the open, up to the actor’s queue capacity
pagination runs one request at a time per timeline and emits HistoryChanged
around each. the sdk remembers where history starts, we don’t, because a gappy
sync can unload it again
the actor’s watchers are supervised, and a panic in any of them panics the
actor. its command work isn’t, a panicking redaction just answers Interrupted
and gets logged
timelines
TimelineDiff::try_apply refuses a diff whose indices don’t fit your snapshot
and leaves the snapshot alone. try_apply_all does the same for a whole batch,
checking every diff against lengths alone first, since that’s all a diff’s fit
depends on. snapshots are imbl::Vectors so keeping one is a pointer copy.
stop applying, resync, and carry on from the next TimelineReset. resync
kills the old watcher before the new reset goes out, so no stale diff can land
after it
TimelineId tells a room from a thread by its root event
every item is an event, a day divider, the read marker or the start. an event
carries its sender with avatar and name ambiguity, and a TimelineContent
saying what it actually is. messages keep their msgtype, membership and profile
changes and the common room state changes get their own variants, and anything
else is OtherState or Other with its type. nothing is flattened into english
in a body field
thread receipts for a thread that isn’t open go straight to the room, so marking a thread read works the same whether its room is on screen or not
replies carry the replied-to event once it’s loaded. the watcher asks the sdk for any it hasn’t got, once per event, since the sdk won’t on its own
edit, redact and react take an EventRef, so your own message can be edited or
reacted to before the server has echoed it back. redacting an unsent one just
cancels it
a custom emote reaction keys on the image’s mxc uri and carries its shortcode
(msc4027), so clients without the pack can still label it. Reaction says
which image it is and the shortcode any sender gave, and toggle_reaction
takes the shortcode to send
thread lists
a room’s thread list is its own thing, not a timeline and not the room actor’s.
it needs only the room, so a room needn’t be open for its list. one worker holds
every open list. Open sends an empty ThreadListReset, Paginate asks
/rooms/{id}/threads through the sdk’s Room::list_threads, and the list
follows the room’s event cache for as long as it’s open. changes go out as
upserts keyed by root, which ThreadListEntry::upsert puts in order, newest
activity first and ties by root, so a list can’t diverge the way an index diff
can
the sdk’s ThreadListService was passed over. it never adds a thread that wasn’t
on a page already loaded, never reorders, and adds one to the count for every
event the cache hands it, replays and back pagination included. here a reply
counts once, by id, when sync just brought it, it’s newer than the thread’s
latest and it was sent after the count was read off the server, all by the sdk’s
corrected timestamp. the latest alone can’t date a count. a summary from the
event cache names its latest reply without the event, and continuwuity 26.9
bundles the latest as bare content, so with only that check every reply the
cache re-inserted read as new and a page’s count doubled. a newer reply that
doesn’t count still becomes the latest. replays, reloads from the store and
back pagination never count. a reply sync brings to a thread the list hasn’t
got fetches its root, four at a time, and adds it. one from anywhere else
doesn’t, since that
says nothing about the thread being active, and a root the server doesn’t have
isn’t asked for again. a redacted reply doesn’t uncount, an edit doesn’t change
the latest reply’s preview, and a preview’s own reply target comes back
Unavailable rather than fetched
a lagged listener can’t know what it missed, so the list fetches its first page
again and sends a ThreadListReset with it. leaving the room, or being kicked
or banned, closes the list with ThreadListClosed and RoomLeft, the shape an
event context closing on a leave has. Close sends one too, with Requested,
since a mirror holds the list and has to drop it. a list’s media belongs to its
own scope and its ids carry it, like an event context’s, bounded the same way so
a long open list sheds the previews its threads moved past, and released on
close
warming up to send
the sdk does all its key work on the send path, in the room’s send queue before
the /send itself. members synced, a /keys/query for anyone untracked or
dirty, then a /keys/claim and the room key sent to every device. over a slow
federation that’s a minute of nothing. prepare_to_send runs all of it ahead
of time through the fork’s Room::prepare_encryption, the same function the
send queue calls. the share sits behind the sdk’s per-room dedup, so a send
landing mid-warm-up waits for it rather than sharing again, and a send after it
finds nothing left to do but the /send. a room whose encryption isn’t known
yet gets asked, which the send would have done anyway. one warm-up per room at
a time, a second ask joins it. opening a room or starting to type runs one too,
unless EngineConfig::prepare_to_send is off. it’s on because malice asked,
not because anyone decided
the session rotates after 100 messages or a week, and sharing isn’t a message, so a warm-up is only wasted on a room left idle about that long
mutations
membership, moderation, power levels, room creation and account data writes go
through one keyed fifo scheduler. same room, same alias, same dm partner or
same account data type run in order, everything else runs at once.
create_direct hands back the existing dm when there is one
notifications and encryption each get one serial worker. the encryption worker owns the verification controller outright
power level changes build one state event from the local mapping, since the sdk’s own patcher is private
media
media is a read. ClientHandle::media takes a MediaId, from a retained
timeline or an avatar, through one bounded
download path with the byte cap the sdk’s own download lacks, an hour’s total
deadline instead of the sdk client’s thirty seconds, and a minute’s patience
for a stalled stream. concurrent asks for the same bytes share one flight.
a thumbnail the server won’t make falls back to the file itself.
MediaId parses from a string, since a web client has to round trip it
the cache holds plaintext, keyed like the sdk keys it, so the sdk and macltok can read each other’s entries. that’s how your own encrypted uploads come back. the sdk keys plain and encrypted sources alike, so a plain event pointing at someone’s encrypted file can leave ciphertext in the cache. an encrypted read checks the cached bytes against the file’s hash and decrypts them if they match
media ids are registered as timelines render and forgotten when their timeline closes, least recently seen first once the cap is hit
avatars are MediaIds too, so a host never handles an mxc uri. they aren’t
registered, since they live in the room list and member lists for the whole
session and there can be thousands, and the cap would evict ones still on
screen. a plain avatar carries no keys, so its id is its mxc uri and resolves
without the registry. it never expires, and an mxc uri a host got some other way
parses into one. set_avatar still takes an mxc uri
streaming decryption inside macltok was declined. in a browser the service
worker answering range requests is a separate context from the worker running
macltok, and streaming through macltok would carry every chunk through wasm
memory, the very thing the host streams to avoid. media_source hands over the
url and keys media would have used, picked the same way, and access_token
the rest. the byte cap doesn’t apply, nothing is buffered
uploads are commands that answer once the send queue has them. the fork puts
upload progress on the client-wide send queue channel too, which upstream keeps
for each room’s own, so one watcher there follows every attachment in the queue,
leftovers the sdk resumes from last time included, and ends each with one
MediaUploadFinished. sent, cancelled, its room forgotten, or
missed when that channel overflowed at the wrong moment, since then there’s no
knowing which. a failure isn’t an ending, a wedged upload can still be retried.
uploads don’t belong to a room’s actor, so closing the room leaves them be
that watcher is the only thing speaking about uploads. anything that ever
showed progress or a failure is followed until it gets exactly one
MediaUploadFinished, and nothing about it goes out after. the sdk sends
progress from a detached task, so a tick can turn up behind its own upload’s
end, and the watcher remembers the last few hundred ends to drop those. the
engine stopping ends nothing, Stopped covers it
sessions and secrets
Session::login_password, begin_oauth and restore share one client builder
with an encrypted store, refresh tokens and threads. the store is sqlite
natively and indexeddb in a browser. secret-bearing inputs don’t derive Clone
the homeserver a person typed is checked before the client sees it, and a bad
one is AccountError::InvalidHomeserver. a bare server name passes untouched,
since that’s what sends the sdk to .well-known, and a host that “helpfully”
adds https:// turns example.org into a homeserver url and skips discovery.
a url needs a host, no userinfo, and https. plain http needs
allow_insecure_loopback and a loopback host. SoftLogout::password_login sets
it, since its homeserver passed once already
a SessionStore owns one login’s record and its store. login saves the first
record, restore loads it, and the engine a Session builds saves every token
rotation after that, since a refresh token works once and a host that missed one
save has lost the login. the session changes are subscribed the moment the
client exists, because the sdk can refresh during restore before any engine
does. a save runs off the event stream, so a full stream never holds it back,
and SessionTokensChanged only goes out once it has landed, or
SessionSaveFailed if it didn’t. natively the sdk’s save callback runs inside
the refresh, before the new token is used, and save_blocking writes there and
then. in a browser there’s only the async save, so a tab that dies between a
refresh and its save wakes up holding a spent token. a fork patch letting the
sdk await an async save before it uses the new token would close that, but it
changes how a refresh behaves rather than what’s visible, and on 2026-10-22 we
held off so the fork doesn’t drift that far that fast. upstream taking an async
save callback, or the fork already carrying behaviour changes, reopens it.
without_persistence
hands the job back to the host. EncryptedSessionStore seals the record with
the sdk’s StoreCipher under the store’s passphrase and replaces it by rename,
so a crash mid save leaves the old one. it keeps the record out of the sdk’s
state store because sqlite writes custom values there in the clear, and a
client can’t open its store before it knows the homeserver the record holds.
discard deletes the record, then the store, since a record without its store
would restore into empty crypto. the host follows Termination::retain.
Nothing discards. Store calls soft_log_out, which keeps the store and
swaps the record for one holding the homeserver, user, device and oauth client
id with no tokens. restore hands that back as Restore::SoftLoggedOut, so a
restart still knows which device to log in again as, and the login overwrites
it. one store directory per login. a login whose first save fails comes back as
AccountError::Unsaved holding the live session, so the host retries the save
rather than spending another device
Accounts picks the store, so a host never decides which account a login form
means. it’s one owner’s accounts under one root and one passphrase, a store and
record each, named by a random id. a password login that names a
soft-logged-out account, by localpart or full user id on the same server, logs
back in as that device into that store. the server is compared by name as well
as by url, since a typed example.org is stored as wherever discovery led. a
live account it names is AlreadyLoggedIn, anyone else gets a new store. oauth
can’t know who it’s for until it finishes, so it resumes only the device
SoftLogout::oauth_login names. a new store is locked before it’s listed, so
list can sweep stores with no record, left by a login that died or a discard
that didn’t finish, without catching one mid login. a browser lists accounts in
an IndexedDB database of their own, a key each, so two tabs adding at once
don’t lose one
the set is per owner because resuming needs only the password, and the store holds the device’s e2ee keys. a set shared across people would hand someone who knows a password the history a fresh login wouldn’t get
one engine per store. before the client opens the store we take the platform’s
own exclusive lock on it, File::try_lock on macltok.lock in the store
directory natively and a Web Lock named after the store in a browser. the
engine and every ClientHandle share it, so it goes when the last of them
does, since a lingering handle still has the client and its store open. anyone
else, in this process, another one or another tab, gets StoreInUse at once,
so the host decides whether to wait, hand over or say so. the os or the browser
lets go when the holder dies. a SharedWorker serving every tab from one engine
never contends for it. Web Locks only exist in a secure context, so a browser
store needs one too
the sdk’s cross process lease can’t do this job. in matrix-sdk-common 0.18
CrossProcessLock::try_lock_once (cross_process_lock.rs 389-448) leases for
500ms and renews every 50ms from a spawned task, which gives up for good on one
failed renewal or on finding the lease lapsed and retaken, and tells no one. a
stall past half a second, ordinary in a tab and certain during the record’s
PBKDF2, or one SQLITE_BUSY, hands the store to a second engine while the
first still thinks it’s alone. with our lock keeping everyone else out, clients
run with CrossProcessLockConfig::SingleProcess and the sdk’s own sections skip
the lease writes and renewal tasks
a pending OAuth login survives the page going away for its redirect.
begin_oauth saves it as the store’s record, sealed like any other since it
holds the PKCE verifier, through the fork’s OAuth::pending_authorization.
Accounts::finish_oauth takes the callback url and finds the account by its
state, so the host keeps nothing across the redirect, and
Session::finish_oauth does the same for one store. a host that keeps the
process alive can still use PendingOAuthLogin::finish. while it waits,
restore answers Restore::OAuthPending. the record lives in the store
rather than with the host because list sweeps a store with no record, which
would delete a fresh login’s store mid redirect
the pending record keeps whatever the store held before, so a login resuming a
soft-logged-out device that never comes back puts that device back, and a
password login can still resume it meanwhile. abort, a failed finish after the
redirect and an hour’s wait all undo the login, a new account’s store deleted
and an old record put back. a failed PendingOAuthLogin::finish still holds
the client, so it puts back an old record and leaves a new one to the hour.
codes live minutes, so an hour is generous
room lists, spaces and profiles
RoomSummary has what a room list row needs without a second query. topic,
alias, member counts, heroes for nameless rooms, the inviter, where an upgraded
room went, a latest event preview, the joined spaces listing it, and for a
space the children it lists
parents come from one lazily built SpaceService shared by every handle.
children come straight from the space’s local state, joined or not.
space_hierarchy pages through the server’s view for browsing rooms you
haven’t joined, and holds its place between pages
own_profile and user_profile read profiles. name and avatar changes are
commands, and uploading an avatar gets the same byte cap as any upload
notifications and room data
the notification client is built lazily and shares the engine’s sync service. sdk ui types stay behind the boundary
rooms normalises on demand with bounded concurrency and sorts by latest event,
then name, then id. RoomsChanged is an invalidation, coalesced so one sync
touching fifty rooms is one event. a sync’s room info lands before the event
cache has stored its events, so the watcher waits a tenth of a second after
anything wakes it and takes the lot as one. an event cache slower than that
costs a second one. a left room stays in the list with its
membership until it’s forgotten. push rule changes never touch room info, so
NotificationSettingsChanged brings a RoomsChanged along for the modes in
the summaries
with threads on, the sdk leaves thread events out of a room’s counts and keeps
none of its own for them. thread_unreads counts them per root from the event
cache store, which keeps every reply it ever took whatever it holds in memory,
so closing a room, the cache trimming itself or a restart leaves them be. a
reply counts once it’s newer than a receipt for its thread (msc3771), an
unthreaded one, or the user’s own reply. a main-timeline receipt doesn’t reach
into threads. the store has lost stream order between a thread and the rest of
the room, so an event’s timestamp stands for its place, and a receipt on an
event the store never had falls back to the receipt’s own time. a thread’s
mentions also count in the room’s unread_mentions, so nobody misses one,
while its messages and notifications stay apart, since they don’t move the main
timeline’s read marker
the counts are kept in the state store per room, only for threads with
something unread, so rooms reads them without touching the event cache and a
cold room never gets one made for it. thread events and threaded receipts leave
room info alone, so the room list watcher recounts on its own. a synced thread
event names the thread to count, an event cache update recounts the room’s
threads already holding something unread, since a late decryption can turn up
a mention, and so does a receipt of the user’s own. counting in the sdk would
be cheaper, but that’s a behaviour change in the fork
link previews
a text message can carry its sender’s own previews (msc4095), and those come
out on Message::link_previews without asking anyone, so they work in
encrypted rooms and leak nothing. an encrypted preview image is registered like
an attachment. a preview whose url isn’t in the body is dropped, or a sender
could caption any link with any card
url_preview asks the homeserver, which fetches the page, re-hosts og:image
and answers with loose og: keys. that tells the homeserver every url, so
whether encrypted rooms ask is the host’s call. answers are kept an hour and
misses ten minutes, and asks for one url at once share a request, since synapse
rate limits the endpoint and a timeline repeats links. synapse answers a page
it couldn’t download with a 500, a dns failure with a 502 and its blocklist with
a 403, so those, 400 and 404 mean no preview. a genuine 500 hides in there too,
but a missing card is cheap. a
server that doesn’t preview at all says so with M_UNRECOGNIZED once and isn’t
asked again. a 429 anywhere is ErrorKind::RateLimited. page text from either
source is trimmed, 256 characters for a title or site name and 1024 for a
description, and an image that isn’t an mxc uri is dropped
url_previews_allowed follows element, so both agree. the user’s own choice
for the room, then their account wide default, then room state an admin set.
the room state comes in with the rest through the fork’s extra required state
image packs
msc2545 packs come from the user’s account data, from room state, and from the
rooms the user enabled everywhere. the room list asks the server for every
room’s im.ponies.room_emotes, through the fork’s extra required state, so
packs sit in the store like any other state and a change reaches us even across
a gap in the timeline. that’s every joined room’s packs on the first sync,
tens of kilobytes for a big one, and only changes after. asking only for open
rooms would lose the enabled packs of rooms nobody has open
image_packs reads all of it from the store and hands back the lot, global
packs and each room’s own, and ImagePacks::for_room puts a picker’s list in
order. an enabled room the user hasn’t joined is skipped, there’s no state for
it. one bad image costs that image and a pack with nothing usable is dropped,
since pack contents are whatever someone’s client wrote. ImagePacksChanged
says a pack moved somewhere, from a state event, an account data change or a
membership change taking a room’s packs with it. it’s an invalidation, and the
mirror refetches on it like it does rooms, keeping the packs in its state
verification
Accept, Confirm, Reject and Cancel. a request another device sends
shows up as Incoming and waits for Accept, or Cancel to turn it down. one
is only taken up while nothing else is in flight. reject tells the other side
the emoji didn’t match. the watchers subscribe before they look at the current
state, since the sdk only reports what changes after you ask. the watcher and
the action paths claim the terminal event, so a racing
cancel can’t turn Done into Cancelled
mirror
the mirror is a consumer like any other, behind the mirror feature, and it
takes any stream of events rather than the EventStream itself so a host keeps
the raw events without a second channel. state, its sequence number and the
broadcast send all move under one lock, which is how subscribe gets a
snapshot and a receiver with nothing between them. a lagged subscriber starts
over from a fresh state under the same lock
nothing on the draining path waits for an engine answer. a resync runs on its own task, retrying with backoff until a reset or a close for its timeline aborts it. rooms are fetched on a task too, one at a time with one more owed at most. a diff for a timeline with no reset is a divergence too, so a wrapper that swallowed the reset still heals. anything a late subscriber needs, failures included, lives in the state rather than only in an update
search
search_messages takes words and a scope of one room, a list of them or every
joined room, and answers newest first. it opens a search the way a jump opens an
event context, a session owning the media of every hit it hands out under its
own bounded scope, paged with search_more and its cursor and freed with
close_search. at most eight stay open and opening another closes the oldest,
so a host that never closes anything still holds a bounded amount, and the
closed one goes out as SearchClosed. a hit’s media ids carry their search’s
owner prefix like a context’s, and stop resolving once it closes. encrypted
rooms can’t use the server’s /search, so everything goes through one local
index, unencrypted
rooms too, so results and coverage mean the same thing everywhere. a hit comes
back the way event context shows its target, latest edit applied, with byte
ranges of the matched words in its text
the index is tantivy, but not the fork’s matrix-sdk-search. that crate keeps its
schema private, keeps an index per room, opens a fresh IndexWriter for every
event, ranks by score, hands back bare event ids and turns mmap on, which wasm
can’t build. IndexWriter can’t run in wasm either, since it spawns indexing
threads and rayon pools. so macltok keeps its own segments. each flush writes an
immutable segment with SingleSegmentIndexWriter into a RamDirectory, edits
and redactions are tombstones in a bitset beside it, and segments of a size
class merge through merge_filtered_segments once six pile up, dropping the
tombstoned docs as they go. all of that is single threaded with the docstore’s
dedicated compressor off. message text is indexed and not stored, a preview
reads the event back. the one exception is an edit waiting for its original,
whose text is kept so the original can take it over. a 2026-10-25 spike ran the
whole cycle in a headless firefox worker
it lives with the store, so it’s per login and never synced, and it’s only
opened for the user and homeserver that made it. natively it’s
macltok-search.sqlite3 beside the sdk’s databases, created 0600, and in a
browser the ::macltok-search IndexedDB database, and both go when the store is
deleted. every segment, every tombstone generation, the manifest and the waiting
undecryptable events are blobs sealed by a StoreCipher under the store
passphrase, stored under a keyed hash of their name like the mentions rows, the
name sealed inside so blobs can’t be swapped. the manifest a commit saves is the
index as it stands once that commit lands, and blobs it doesn’t point at are
dropped on the next load. one commit is one transaction, sqlite in wal with
normal sync and IndexedDB relaxed, so a crash can lose the last few seconds of
commits but never leaves half of one. the working copy lives in memory, all of
it, which is the price of tantivy’s directory being synchronous and IndexedDB
not. a store that won’t open or read, or five commits failing in a row, means
an index in memory for the rest of the engine, and coverage says
persistent: false with the reason in in_memory
it’s fed from what the client sees. each joined room’s event cache is followed from its first update, so sync, pagination and the redecryptor’s replacements all arrive as diffs, and a room idle for five minutes is let go so the cache can shrink. the generic update channel is followed from before the index loads, and if it lags every joined room is followed, since a lost update could be a room’s only one. an edit lands on its original under the original’s key, the latest by the original’s sender winning. one that arrives first waits under its target and its own sender, out of results, until the original turns up and takes the author’s edit, so a stranger’s early edit can’t crowd it out. an edit of an edit is ignored, and an edit to content we can’t read clears the old text. a redaction drops the key and is remembered for a while, so a stale copy can’t bring it back. redacting the latest edit reads the original again with whatever edits of it are still around. a message still encrypted waits, raw, up to two thousand of them in sixteen blobs, and is decrypted eight at a time when a room key for its session arrives
backfill is /messages straight from the room, not through the event cache,
so it never grows a timeline someone has open, filtered to messages, encrypted
events, redactions, stickers and polls. one request at a time across every
room, two seconds apart, a hundred events a page and ten pages a room per
engine run, round robin. a room walks back from the newest event to the start,
and resumes from its saved token next time. a room that’s been indexed before
also gets a head pass the first time it shows up in a run, from the newest event
back until it meets something that was already indexed when the engine started,
which picks up what happened while we were offline. a failing room backs off on
its own from a minute to fifteen. a saved token the server turns down with a 400
is dropped and the room starts again from the newest. leaving a room cancels its
page. all of that is EngineConfig, and zero pages a room turns backfill off
coverage is per room, the oldest and newest indexed message, whether backfill reached the start, and where backfill is at, plus the undecryptable messages in scope. it comes with every page
the analyser makes a token of every run of letters and digits up to 64 chars,
so hashes and long words fit, and lowercases and folds accents, so cafe finds
Café. han, kana, hangul and thai have no spaces to split on, so every char is
a token and so is every pair of neighbours, and a query for a run needs all its
pairs. that finds a word inside a sentence at the cost of the odd match across a
word boundary. emoji are tokens too. queries have no field syntax, so re: or a
url is just its words. words all have to match, quotes make a phrase and -
excludes, and a query with nothing left to find is Invalid rather than
everything in scope. there’s no stemming
a hit is read back when it’s shown, load_or_fetch_event for the original and
for its latest edit, which is the event cache and its store first and the
server’s /event after, eight at a time. the original’s bundled edit is ignored,
since the index already knows the newest one. one that’s turned out redacted is
dropped from the page and from the index. one that can’t be read at all, gone
from the cache with the server unreachable or refusing, is left out of that page
but stays indexed, so it shows up again once it can be read. the cursor still
moves past it
tantivy’s merger times itself with Instant::now() whenever a log logger is on
at debug, and that panics in wasm. on 2026-10-25 we capped log at info at
compile time in wasm, max_level_info and release_max_level_info both, since
in release the release cap wins. it’s all or nothing for the whole wasm build, so
a host gives up log debug and trace from every crate, not just tantivy, and
asking for any other cap is log’s own “multiple max_level_* features set” error.
tantivy timing with something wasm can call, or reading Instant only behind
its own feature, reopens it
the sdk fork
we build against a fork of matrix-rust-sdk, branch macltok, cut from the
matrix-sdk 0.18.0 tag. turned down on 2026-09-26, taken up on 2026-10-22 once
it was holding back features every peer client has. the sdk crates are git
dependencies pinned to a fork rev, so dependents get the fork through macltok
with no patch block of their own, and macltok stays off crates.io until
upstream takes the patches. a dependent naming matrix-sdk itself has to point
at the same rev or it gets a second sdk
it carries visibility and config changes, each small enough to be an upstream pr, and nothing that changes how the sdk behaves. if that stops being true, especially in the timeline or the send queue, it’s time to drop matrix-sdk and matrix-sdk-ui for our own layer over matrix-sdk-base and matrix-sdk-crypto, which keep their public sans-io apis
Room::prepare_encryption, the send path’s encryption prep made publicSyncServiceBuilder::with_room_list_extra_required_state, for state the sdk’s hard-coded list leaves out. image packs use it- msc4027 reaction shortcodes, kept on the timeline’s
ReactionInfoand sent withTimeline::toggle_reaction_with_shortcode SendQueue::enable_global_upload_progress, upload progress on the client-wide send queue channelOAuth::pending_authorizationandrestore_pending_authorization, to finish an oauth login in another process
public model
owned ruma ids and typed data wherever there’s a stable shape, Raw where
arbitrary content has to survive. ruma is re-exported as macltok::ruma, so a
host parses ids at the version we’re typed against. anything expected to grow is
#[non_exhaustive]. Engine::from_client stays public for people who built
their own client