atagen

malice

public
Star0
Watch0
973f8fbdaf80e2ef6679dd071233fe6b853344b2
2
ssh://brewery@pls.lobotomise.me:33/atagen/malice.git

Blame: MACLTOK_MEDIA_STREAM.md

973f8fbdaf80atagen2026-09-28 09:10 UTC1
# macltok request, what a host needs to stream large media itself
973f8fbdaf80atagen2026-09-28 09:10 UTC2
973f8fbdaf80atagen2026-09-28 09:10 UTC3
## Problem
973f8fbdaf80atagen2026-09-28 09:10 UTC4
973f8fbdaf80atagen2026-09-28 09:10 UTC5
`ClientHandle::media(&MediaId, MediaSize)` answers with `Media { bytes: Arc<[u8]>, .. }`, the whole
973f8fbdaf80atagen2026-09-28 09:10 UTC6
file, decrypted and hash-checked, capped by `EngineConfig::max_media_bytes`. That is right for
973f8fbdaf80atagen2026-09-28 09:10 UTC7
avatars, thumbnails and images. It is wrong for a 20 minute video.
973f8fbdaf80atagen2026-09-28 09:10 UTC8
973f8fbdaf80atagen2026-09-28 09:10 UTC9
- In a browser the whole file sits in wasm linear memory, which never shrinks and tops out at 4 GB
973f8fbdaf80atagen2026-09-28 09:10 UTC10
  on wasm32. A few long videos and the worker is carrying hundreds of MB it can never give back.
973f8fbdaf80atagen2026-09-28 09:10 UTC11
- Nothing plays until the last byte has arrived and the hash has been checked, and seeking has to
973f8fbdaf80atagen2026-09-28 09:10 UTC12
  wait for the same.
973f8fbdaf80atagen2026-09-28 09:10 UTC13
973f8fbdaf80atagen2026-09-28 09:10 UTC14
malice's on-device build serves media through a service worker so `<video>` and `<audio>` get real
973f8fbdaf80atagen2026-09-28 09:10 UTC15
URLs and Range requests. For large media it wants to answer each Range itself, fetching the matching
973f8fbdaf80atagen2026-09-28 09:10 UTC16
ciphertext range from the homeserver's authenticated media endpoint and decrypting it in the service
973f8fbdaf80atagen2026-09-28 09:10 UTC17
worker with WebCrypto AES-CTR (the counter block is the IV plus `offset / 16`, so any 16-byte aligned
973f8fbdaf80atagen2026-09-28 09:10 UTC18
range decrypts on its own). Nothing is buffered and the browser caches only ciphertext. The hash can
973f8fbdaf80atagen2026-09-28 09:10 UTC19
only be checked on a whole read, so a partial read plays unverified bytes. malice accepts that.
973f8fbdaf80atagen2026-09-28 09:10 UTC20
973f8fbdaf80atagen2026-09-28 09:10 UTC21
What it can't get today is what the request needs. `MediaRegistration` and `MediaSourceInfo`
973f8fbdaf80atagen2026-09-28 09:10 UTC22
(`normalise/media.rs`) are `pub(crate)`, and there is no way to read the access token.
973f8fbdaf80atagen2026-09-28 09:10 UTC23
973f8fbdaf80atagen2026-09-28 09:10 UTC24
## Request
973f8fbdaf80atagen2026-09-28 09:10 UTC25
973f8fbdaf80atagen2026-09-28 09:10 UTC26
Two read-only accessors on `ClientHandle`. Suggested shapes, adjust freely.
973f8fbdaf80atagen2026-09-28 09:10 UTC27
973f8fbdaf80atagen2026-09-28 09:10 UTC28
```rust
973f8fbdaf80atagen2026-09-28 09:10 UTC29
/// where a registered piece of media lives and how to decrypt it, for a host that fetches it
973f8fbdaf80atagen2026-09-28 09:10 UTC30
/// itself. `None` for an unknown id, same as `media` answering `MediaUnknown`
973f8fbdaf80atagen2026-09-28 09:10 UTC31
pub fn media_source(&self, media_id: &MediaId, size: MediaSize) -> Option<MediaSourceDetails>;
973f8fbdaf80atagen2026-09-28 09:10 UTC32
973f8fbdaf80atagen2026-09-28 09:10 UTC33
#[non_exhaustive]
973f8fbdaf80atagen2026-09-28 09:10 UTC34
pub struct MediaSourceDetails {
973f8fbdaf80atagen2026-09-28 09:10 UTC35
    pub uri: OwnedMxcUri,
973f8fbdaf80atagen2026-09-28 09:10 UTC36
    pub mimetype: Option<String>,
973f8fbdaf80atagen2026-09-28 09:10 UTC37
    pub declared_size: Option<u64>,
973f8fbdaf80atagen2026-09-28 09:10 UTC38
    /// `Some` for an encrypted attachment
973f8fbdaf80atagen2026-09-28 09:10 UTC39
    pub encryption: Option<AttachmentEncryption>,
973f8fbdaf80atagen2026-09-28 09:10 UTC40
}
973f8fbdaf80atagen2026-09-28 09:10 UTC41
973f8fbdaf80atagen2026-09-28 09:10 UTC42
#[non_exhaustive]
973f8fbdaf80atagen2026-09-28 09:10 UTC43
pub struct AttachmentEncryption {
973f8fbdaf80atagen2026-09-28 09:10 UTC44
    /// the JWK `k`, unpadded url-safe base64, AES-256-CTR
973f8fbdaf80atagen2026-09-28 09:10 UTC45
    pub key: String,
973f8fbdaf80atagen2026-09-28 09:10 UTC46
    /// unpadded base64, the initial counter block
973f8fbdaf80atagen2026-09-28 09:10 UTC47
    pub iv: String,
973f8fbdaf80atagen2026-09-28 09:10 UTC48
    /// unpadded base64 SHA-256 of the ciphertext, for a host that does read it whole
973f8fbdaf80atagen2026-09-28 09:10 UTC49
    pub sha256: String,
973f8fbdaf80atagen2026-09-28 09:10 UTC50
}
973f8fbdaf80atagen2026-09-28 09:10 UTC51
973f8fbdaf80atagen2026-09-28 09:10 UTC52
/// the current access token and the homeserver it belongs to. it changes on every refresh, so
973f8fbdaf80atagen2026-09-28 09:10 UTC53
/// ask again after a 401 rather than keeping it
973f8fbdaf80atagen2026-09-28 09:10 UTC54
pub fn media_credentials(&self) -> Option<MediaCredentials>;
973f8fbdaf80atagen2026-09-28 09:10 UTC55
973f8fbdaf80atagen2026-09-28 09:10 UTC56
#[non_exhaustive]
973f8fbdaf80atagen2026-09-28 09:10 UTC57
pub struct MediaCredentials {
973f8fbdaf80atagen2026-09-28 09:10 UTC58
    pub homeserver: String,
973f8fbdaf80atagen2026-09-28 09:10 UTC59
    pub access_token: String,
973f8fbdaf80atagen2026-09-28 09:10 UTC60
}
973f8fbdaf80atagen2026-09-28 09:10 UTC61
```
973f8fbdaf80atagen2026-09-28 09:10 UTC62
973f8fbdaf80atagen2026-09-28 09:10 UTC63
Thumbnail sizes with no uploader thumbnail are server-side resizes, which are never encrypted, so
973f8fbdaf80atagen2026-09-28 09:10 UTC64
`media_source` for `MediaSize::Thumbnail` can answer the uploader's thumbnail when there is one and
973f8fbdaf80atagen2026-09-28 09:10 UTC65
the full source otherwise, whichever `media` would have used.
973f8fbdaf80atagen2026-09-28 09:10 UTC66
973f8fbdaf80atagen2026-09-28 09:10 UTC67
The token accessor is the sensitive one. It hands a caller the bearer token, which `session_record`
973f8fbdaf80atagen2026-09-28 09:10 UTC68
already does in serialized form, so it doesn't widen what a host can reach. It is worth a doc line
973f8fbdaf80atagen2026-09-28 09:10 UTC69
saying the token should stay in memory and only go to the homeserver's media endpoints.
973f8fbdaf80atagen2026-09-28 09:10 UTC70
973f8fbdaf80atagen2026-09-28 09:10 UTC71
## Also worth considering
973f8fbdaf80atagen2026-09-28 09:10 UTC72
973f8fbdaf80atagen2026-09-28 09:10 UTC73
Streaming decryption inside macltok, `media_stream(&MediaId, range) -> impl Stream<Item = Bytes>`,
973f8fbdaf80atagen2026-09-28 09:10 UTC74
would let every host stream without reimplementing AES-CTR and Range handling, and would keep the
973f8fbdaf80atagen2026-09-28 09:10 UTC75
token inside macltok. matrix-sdk 0.18 decrypts attachments whole (`Media::get_media_content`), so
973f8fbdaf80atagen2026-09-28 09:10 UTC76
it would mean reimplementing the attachment decryptor over the raw download. malice will do it
973f8fbdaf80atagen2026-09-28 09:10 UTC77
host-side for now, so this is not blocking.