Null-DRM-Official/docs/architecture.md

# Architecture

## What the system does

Given an Android app that streams DRM-protected live TV, it recovers the three
things a downloader needs — **manifest URL**, **Widevine content keys**, and any
**auth headers** — and keeps them fresh.

Two independent ways to get them:

1. **Catalog** (`drm catalog`) — the app's playback catalog is public, so a plain
   HTTPS request plus a local CDM is enough. No phone.
2. **Phone capture** (`drm capture`) — launch the real app under an on-device
   HTTPS MITM, drive its UI to the channel, and read the license exchange.

What it deliberately does not do: break DRM. The CDM used is a provisioned
Widevine device you supply (`.wvd`); the system only automates the request flow a
real client performs.

## One binary

```text
             ┌──────────────────────── bin/drm ────────────────────────┐
             │                                                         │
  modules ───┤ capture   catalog   agent   serve   proxy               │
  linked in  │                                                         │
             └─────────────────────────────────────────────────────────┘
```

`apps/cli` is the only `main`. It blank-imports `drmdecryption/modules/all`, which
is what makes `--app <name>` resolve. Each app module registers itself:

```go
func init() {
    appreg.Register(Name, New)        // --app rte
    appreg.RegisterFlags(bindFlags)   // --rte.<field> overrides
}
```

Adding a provider means adding a package and one import line. There is no plugin
loading, no scripting runtime, and nothing to deploy beside the binary.

## Logic vs values

This is the central constraint:

| `apps/modules/<name>/*.go` — **tracked** | `apps/modules/<name>/module.yaml` — **gitignored** (bbc published) |
|---|---|
| launch + auto-play sequence | android package id |
| navigation idioms (chip→Play, drawer→Live) | license URL, origin hostnames |
| manifest rewrite shape | channel KIDs, account id, policy key |
| catalog request flow | video ids, playback config URL |
| which capture fields are required | UI selector regexes, labels, aliases |

Module Go source contains no account id, policy key, video id, license URL, origin
host or KID. Values arrive by **flag → environment → module.yaml → Go default**,
and only mechanical things (timeouts, DASH timescales, card geometry) have
defaults in code.

`.gitignore` enforces the split:

```gitignore
apps/modules/**
!apps/modules/README.md
!apps/modules/go.mod
!apps/modules/go.sum
!apps/modules/*/
!apps/modules/*/*.go
!apps/modules/*/**/*.go
```

The directory re-include is required — git will not descend into an excluded
directory to find negated files.

## Go modules

Six modules, wired with `replace`:

```text
apps/pkg      drmdecryption                  shared, provider-neutral
apps/modules  drmdecryption/modules          app modules      -> pkg
apps/capture  drmdecryption/apps/capture     capture command  -> pkg
apps/agent    drmdecryption/apps/agent       agent            -> pkg
apps/streamd  drmdecryption/apps/streamd     control plane    -> pkg
apps/proxy    drmdecryption/apps/proxy       MITM host CLI    -> pkg
apps/cli      drmdecryption/apps/cli         the binary       -> all of the above

apps/proxy/device  appproxy                  standalone, cross-compiled for the phone
```

Command bodies live in non-`internal` packages (`capturecmd`, `agentcmd`,
`servecmd`, `proxyctlcmd`) because `internal/` is not importable across module
boundaries and `apps/cli` has to call them.

## Plugin contract

`apps/pkg/app` defines what a module must provide and what it may add.

Required (`app.App`):

| Method | Purpose |
|---|---|
| `Name`, `Package`, `LicenseURL` | identity |
| `Launch(c)` | cold-start the app, wait for its home UI |
| `AutoPlay(c, channel)` | navigate to a channel and confirm playback |
| `CaptureHints()` | which capture fields are required, plus URL scoring |
| `KeyMode()` | `"modulardrm"` or `"raw"` — never sniffed from the license URL |
| `MPDRewriter()` | manifest rewrite, or `mpd.Passthrough{}` |
| `Channels`, `HasChannel` | channel list and lookup |
| `ProxyBin`, `CAHash` | on-device MITM binary and CA hash |

Optional, detected by type assertion:

| Interface | Gives you |
|---|---|
| `app.Catalog` | `drm catalog --app <name>` works |
| `app.CatalogLister` | a richer `catalog --list` with catalog ids and provider flags |
| `app.StreamDefaults` | streamd downloader settings (selectors, rewriter, buffering) |
| `mpd.Rewriter` + `mpd.Register` | a named manifest rewrite streamd can resolve |

## Shared libraries (`apps/pkg`)

| Package | Role |
|---|---|
| `app` | plugin interface + registry + optional capability interfaces |
| `adb` | ADB client: shell, push/pull, UI dump, tap, swipe, playback probe |
| `uiflow` | UI primitives modules drive playback with — every threshold a parameter |
| `capture` | parse on-device capture JSON; score candidate manifest URLs |
| `phonecap` | one capture job end to end |
| `proxy` | push/start/stop the on-device MITM, CA install/reinject, pull captures |
| `wvkey` | shells out to `apps/wvkey/wvkey.py` |
| `brightcove` | Brightcove Playback API + HLS Widevine extraction — a *platform*, not a provider |
| `mpd` | rewriter interface + registry, synthetic live manifest builder, local server |
| `modcfg` | load a module's `module.yaml`, apply env overrides, resolve paths |
| `session` | write `outputs/<app>/<stamp>/` |
| `repo` | locate the repo root |

Nothing here names a streaming provider.

## Phone capture pipeline

```text
drm capture --app X --channel Y --auto-play
   │
   ├─ push + start apps/proxy/device (appproxy) on the phone
   ├─ set the phone's global http_proxy to 127.0.0.1:8080
   ├─ Magisk-mount the MITM CA into the system trust store
   ├─ module.Launch()     : force-stop → monkey launch → wait for a home marker
   ├─ module.AutoPlay()   : app-specific navigation → wait for playback
   ├─ capture.Wait()      : poll the device capture JSON + mirrored proxy log
   │                        until the module's required fields appear
   ├─ wvkey.py            : PSSH + license → KID:KEY (mode from module.KeyMode())
   └─ session.Write()     : outputs/<app>/<stamp>/ + latest/
```

The MITM runs **on the device**, so it cannot read a module's values file. A
module's manifest-scoring hosts are passed to it as flags
(`capture.ScoreCfg.ProxyArgs()` → `-score-host`, `-score-deny`).

### Why URL scoring exists

A MITM sees EPG feeds, analytics beacons and config blobs alongside the real
manifest. `capture.ScoreCfg` ranks candidates: format-level signals (`.mpd`,
`.m3u8`, `playlist-hls`, `/manifest`, `.isml`) are generic and live in `apps/pkg`;
provider hosts and noise needles come from `module.yaml`.

## Manifest rewriting

Some providers serve an ad-stitched manifest whose session expires (HTTP 410)
while the underlying origin keeps running. `apps/pkg/mpd` can publish a
**synthetic** manifest on the origin encoder's own segment grid so a downloader
keeps fetching real segments:

```text
mpd.LearnGrid        fetch the origin Smooth manifest once; derive segment
                     duration and phase offset per media type
mpd.BuildSynthetic   emit a dynamic MPD on that grid, with a sliding window
mpd.StartLocal       serve it on 127.0.0.1 and re-run the rewrite on every GET
mpd.Register/Lookup  resolve a rewriter by name, so streamd needs no provider import
```

`SyntheticConfig` holds every value that could differ — timescales, stream names,
bandwidths, codecs, resolution, segment templates, live-edge offset, window size.
The RTE module supplies origins and KIDs; when the upstream manifest is gone, a
`mpd.KIDResolver` maps the KID from a previous capture back to its origin.

## Control plane (`drm serve`)

| Piece | Path |
|---|---|
| Command | `apps/streamd/servecmd` |
| HTTP API | `apps/streamd/internal/api` |
| SQLite | `apps/streamd/internal/db` |
| Dashboard | `apps/streamd/internal/ui` |
| Media supervisor | `apps/streamd/internal/supervisor` |

Port 8083 by default. Mutating routes need `Authorization: Bearer <token>`.

Streamd stores no provider knowledge. `GET /api/apps` reports the compiled-in
modules, and the supervisor resolves per-stream downloader settings at start time:

```text
settingsFor(stream):
   stored row  ->  the stream's app module (app.StreamDefaults)  ->  neutral defaults
```

The rewriter is a registry lookup on the stream's `rewriter` column, and HLS is
detected from the manifest shape — not from an app name or a stream name.

## Agent (`drm agent run`)

```text
poll streamd ──► durable SQLite queue ──► claim a free ADB device
                                             │
                      POST credentials ◄── phone capture (phonecap)
```

Per-stream claims expire via a websocket heartbeat, so a dead agent releases its
phones. The agent never installs apps and never runs a downloader. It drives
whichever module a stream names; it contains no provider logic.

## Where things land

| Path | Contents |
|---|---|
| `data/` | durable secrets: `.wvd`, CA — gitignored |
| `outputs/<app>/<stamp>/` | capture + catalog sessions, with `latest/` as a copy |
| `outputs/discover/<stamp>/` | MITM discover dumps |
| `.cache/streamd/` | streamd SQLite, work dirs, HLS output |
| `.cache/ui-<app>/` | UI dumps used by the automation |

## Honest status

| Area | State |
|---|---|
| Capture (catalog + phone), both modules | working, verified against live services |
| Compiled-in modules, one binary | done |
| streamd API + SQLite + dashboard + claims | done |
| Synthetic manifest rewrite | done, verified against a live origin |
| Agent queue + phone refresh | done |
| Real downloader/ffmpeg workers in streamd | partial — the supervisor shells out to external tools and is the least exercised path |