d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 1 | # segmented history |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 2 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 3 | a room's history is sections of events with durable gaps between them. the sdk |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 4 | event cache keeps the sections, gap identity, boundaries and each gap's cursor, |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 5 | encrypted at rest, across room close and restart. macltok shows gaps inline and |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 6 | fills them on request. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 7 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 8 | ## opening a room |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 9 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 10 | opening loads retained history and its gaps, then starts a recent load in the |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 11 | background, so opening never waits on the network. the recent load fetches up |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 12 | to four pages of 50 events from a fresh server position. if it reaches the |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 13 | newest retained event the sections join. if it doesn't, a gap is left between |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 14 | the fresh window and what was retained. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 15 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 16 | `Event::RecentHistoryChanged` reports it, one event per edge. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 17 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 18 | - `Loading` as it starts |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 19 | - `Loaded`, `Cancelled` or `Failed(message)` as it settles |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 20 | - `Idle` when the room closes, whatever was happening |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 21 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 22 | `Failed` stays until `HistoryRepairCommand::RetryRecent` or a reopen. a failed |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 23 | load falls back to paginating `EngineConfig::initial_history_limit` events, so |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 24 | the room isn't left empty. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 25 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 26 | ## filling a gap |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 27 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 28 | `HistoryRepairCommand::FillGap { room_id, gap_id }` walks one gap backwards, up |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 29 | to four pages of 50, each page committing its events and the gap's new cursor |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 30 | together. `Event::HistoryRepairChanged` carries a `HistoryRepairReport`, once |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 31 | with phase `Bridging` as the fill starts and once more as it settles. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 32 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 33 | - `Ready`, continuity `Verified`. the older section was reached and the gap is gone |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 34 | - `Paused` with a stop. `Budget` after four pages, `CursorStalled` when the |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 35 | server repeats a cursor, `HistoryUnavailable` (continuity `AccessibleStart`) |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 36 | when accessible history ends first, `Cancelled` |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 37 | - `Paused` with a `failure`, and the gap row goes `Failed` |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 38 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 39 | a gap that is already joined or unavailable answers at once and counts no page. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 40 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 41 | ## one operation per room |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 42 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 43 | a room runs one history operation at a time. `RetryRecent` joins a recent load |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 44 | that is already running. everything else that overlaps, a fill during a recent |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 45 | load, a recent load during a fill, a second fill, answers `RoomBusy`. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 46 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 47 | `HistoryRepairCommand::Cancel` stops whatever is running. its own answer is |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 48 | `Ok`, the cancelled command answers `Interrupted`, and the published outcome is |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 49 | `Cancelled`, never a failure. pages already committed stay. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 50 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 51 | ## the timeline |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 52 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 53 | gaps appear in `TimelineState.items` as `TimelineItem::HistoryGap`, between the |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 54 | sections they separate. a row keeps its id while its lifecycle changes, and is |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 55 | removed once the gap joins. joined gaps never appear. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 56 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 57 | history lands as `TimelineChanged` diffs, not a `TimelineReset`. the sdk |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 58 | currently replays the whole live timeline when it applies committed history, |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 59 | so those diffs can span the timeline until it gets a non-replacing fence. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 60 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 61 | a command's answer comes after its outcome is published, and after the mirror |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 62 | has applied it when one is attached. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 63 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 64 | ## known holes |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 65 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 66 | a limited sync for a room with no events in memory records a plain gap instead |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 67 | of a history gap (review 2026-10-25 S1), so an interval can still render as |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 68 | continuous. both `prev_batch` drops in the sdk's `handle_sync` remain. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 69 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 70 | ## wire shapes |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 71 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 72 | serde enums are externally tagged. |
c9f0fd0ee1cc | atagen | 2026-10-25 06:16 UTC | 73 | |
c9f0fd0ee1cc | atagen | 2026-10-25 06:16 UTC | 74 | ```json |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 75 | { "HistoryRepair": { "FillGap": { "room_id": "!room:example.org", "gap_id": "opaque" } } } |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 76 | { "HistoryRepair": { "RetryRecent": { "room_id": "!room:example.org" } } } |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 77 | { "HistoryRepair": { "Cancel": { "room_id": "!room:example.org" } } } |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 78 | ``` |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 79 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 80 | a gap row. |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 81 | |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 82 | ```json |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 83 | { |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 84 | "HistoryGap": { |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 85 | "id": "opaque", |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 86 | "older_boundary": "$older:example.org", |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 87 | "newer_boundary": "$newer:example.org", |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 88 | "lifecycle": "Idle", |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 89 | "failure": null |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 90 | } |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 91 | } |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 92 | ``` |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 93 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 94 | lifecycle is `Idle`, `Paused`, `Failed` or `Unavailable`. `failure` is set with |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 95 | `Failed`. |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 96 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 97 | a settled fill. |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 98 | |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 99 | ```json |
9eabb6b7587d | atagen | 2026-10-25 07:08 UTC | 100 | { |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 101 | "room_id": "!room:example.org", |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 102 | "gap_id": "opaque", |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 103 | "pages": 2, |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 104 | "events": 63, |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 105 | "phase": "Ready", |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 106 | "continuity": "Verified", |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 107 | "stop": null, |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 108 | "failure": null |
c9f0fd0ee1cc | atagen | 2026-10-25 06:16 UTC | 109 | } |
c9f0fd0ee1cc | atagen | 2026-10-25 06:16 UTC | 110 | ``` |
c9f0fd0ee1cc | atagen | 2026-10-25 06:16 UTC | 111 | |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 112 | the mirror keeps `State.recent_history`, a room without an entry being idle, |
d9b96952ec60 | atagen | 2026-10-25 09:07 UTC | 113 | and `State.repairs`, the last fill report per room. closing a room drops both. |