Null-DRM-Official/docs/architecture.md

Architecture

What the system does

Given an Android app that streams DRM-protected live TV, it recovers the three things a downloader needs — manifest URL, Widevine content keys, and any auth headers — and keeps them fresh.

Two independent ways to get them:

1. Catalog (drm catalog) — the app's playback catalog is public, so a plain HTTPS request plus a local CDM is enough. No phone.
2. Phone capture (drm capture) — launch the real app under an on-device HTTPS MITM, drive its UI to the channel, and read the license exchange.

What it deliberately does not do: break DRM. The CDM used is a provisioned Widevine device you supply (.wvd); the system only automates the request flow a real client performs.

One binary

             ┌──────────────────────── bin/drm ────────────────────────┐
             │                                                         │
  modules ───┤ capture   catalog   agent   serve   proxy               │
  linked in  │                                                         │
             └─────────────────────────────────────────────────────────┘

apps/cli is the only main. It blank-imports drmdecryption/modules/all, which is what makes --app <name> resolve. Each app module registers itself:

func init() {
    appreg.Register(Name, New)        // --app rte
    appreg.RegisterFlags(bindFlags)   // --rte.<field> overrides
}

Adding a provider means adding a package and one import line. There is no plugin loading, no scripting runtime, and nothing to deploy beside the binary.

Logic vs values

This is the central constraint:

apps/modules/<name>/*.go — tracked	apps/modules/<name>/module.yaml — gitignored (bbc published)
launch + auto-play sequence	android package id
navigation idioms (chip→Play, drawer→Live)	license URL, origin hostnames
manifest rewrite shape	channel KIDs, account id, policy key
catalog request flow	video ids, playback config URL
which capture fields are required	UI selector regexes, labels, aliases

Module Go source contains no account id, policy key, video id, license URL, origin host or KID. Values arrive by flag → environment → module.yaml → Go default, and only mechanical things (timeouts, DASH timescales, card geometry) have defaults in code.

.gitignore enforces the split:

apps/modules/**
!apps/modules/README.md
!apps/modules/go.mod
!apps/modules/go.sum
!apps/modules/*/
!apps/modules/*/*.go
!apps/modules/*/**/*.go

The directory re-include is required — git will not descend into an excluded directory to find negated files.

Go modules

Six modules, wired with replace:

apps/pkg      drmdecryption                  shared, provider-neutral
apps/modules  drmdecryption/modules          app modules      -> pkg
apps/capture  drmdecryption/apps/capture     capture command  -> pkg
apps/agent    drmdecryption/apps/agent       agent            -> pkg
apps/streamd  drmdecryption/apps/streamd     control plane    -> pkg
apps/proxy    drmdecryption/apps/proxy       MITM host CLI    -> pkg
apps/cli      drmdecryption/apps/cli         the binary       -> all of the above

apps/proxy/device  appproxy                  standalone, cross-compiled for the phone

Command bodies live in non-internal packages (capturecmd, agentcmd, servecmd, proxyctlcmd) because internal/ is not importable across module boundaries and apps/cli has to call them.

Plugin contract

apps/pkg/app defines what a module must provide and what it may add.

Required (app.App):

Method	Purpose
Name, Package, LicenseURL	identity
Launch(c)	cold-start the app, wait for its home UI
AutoPlay(c, channel)	navigate to a channel and confirm playback
CaptureHints()	which capture fields are required, plus URL scoring
KeyMode()	"modulardrm" or "raw" — never sniffed from the license URL
MPDRewriter()	manifest rewrite, or mpd.Passthrough{}
Channels, HasChannel	channel list and lookup
ProxyBin, CAHash	on-device MITM binary and CA hash

Optional, detected by type assertion:

Interface	Gives you
app.Catalog	drm catalog --app <name> works
app.CatalogLister	a richer catalog --list with catalog ids and provider flags
app.StreamDefaults	streamd downloader settings (selectors, rewriter, buffering)
mpd.Rewriter + mpd.Register	a named manifest rewrite streamd can resolve

Shared libraries (apps/pkg)

Package	Role
app	plugin interface + registry + optional capability interfaces
adb	ADB client: shell, push/pull, UI dump, tap, swipe, playback probe
uiflow	UI primitives modules drive playback with — every threshold a parameter
capture	parse on-device capture JSON; score candidate manifest URLs
phonecap	one capture job end to end
proxy	push/start/stop the on-device MITM, CA install/reinject, pull captures
wvkey	shells out to apps/wvkey/wvkey.py
brightcove	Brightcove Playback API + HLS Widevine extraction — a platform, not a provider
mpd	rewriter interface + registry, synthetic live manifest builder, local server
modcfg	load a module's module.yaml, apply env overrides, resolve paths
session	write outputs/<app>/<stamp>/
repo	locate the repo root

Nothing here names a streaming provider.

Phone capture pipeline

drm capture --app X --channel Y --auto-play
   │
   ├─ push + start apps/proxy/device (appproxy) on the phone
   ├─ set the phone's global http_proxy to 127.0.0.1:8080
   ├─ Magisk-mount the MITM CA into the system trust store
   ├─ module.Launch()     : force-stop → monkey launch → wait for a home marker
   ├─ module.AutoPlay()   : app-specific navigation → wait for playback
   ├─ capture.Wait()      : poll the device capture JSON + mirrored proxy log
   │                        until the module's required fields appear
   ├─ wvkey.py            : PSSH + license → KID:KEY (mode from module.KeyMode())
   └─ session.Write()     : outputs/<app>/<stamp>/ + latest/

The MITM runs on the device, so it cannot read a module's values file. A module's manifest-scoring hosts are passed to it as flags (capture.ScoreCfg.ProxyArgs() → -score-host, -score-deny).

Why URL scoring exists

A MITM sees EPG feeds, analytics beacons and config blobs alongside the real manifest. capture.ScoreCfg ranks candidates: format-level signals (.mpd, .m3u8, playlist-hls, /manifest, .isml) are generic and live in apps/pkg; provider hosts and noise needles come from module.yaml.

Manifest rewriting

Some providers serve an ad-stitched manifest whose session expires (HTTP 410) while the underlying origin keeps running. apps/pkg/mpd can publish a synthetic manifest on the origin encoder's own segment grid so a downloader keeps fetching real segments:

mpd.LearnGrid        fetch the origin Smooth manifest once; derive segment
                     duration and phase offset per media type
mpd.BuildSynthetic   emit a dynamic MPD on that grid, with a sliding window
mpd.StartLocal       serve it on 127.0.0.1 and re-run the rewrite on every GET
mpd.Register/Lookup  resolve a rewriter by name, so streamd needs no provider import

SyntheticConfig holds every value that could differ — timescales, stream names, bandwidths, codecs, resolution, segment templates, live-edge offset, window size. The RTE module supplies origins and KIDs; when the upstream manifest is gone, a mpd.KIDResolver maps the KID from a previous capture back to its origin.

Control plane (drm serve)

Piece	Path
Command	apps/streamd/servecmd
HTTP API	apps/streamd/internal/api
SQLite	apps/streamd/internal/db
Dashboard	apps/streamd/internal/ui
Media supervisor	apps/streamd/internal/supervisor

Port 8083 by default. Mutating routes need Authorization: Bearer <token>.

Streamd stores no provider knowledge. GET /api/apps reports the compiled-in modules, and the supervisor resolves per-stream downloader settings at start time:

settingsFor(stream):
   stored row  ->  the stream's app module (app.StreamDefaults)  ->  neutral defaults

The rewriter is a registry lookup on the stream's rewriter column, and HLS is detected from the manifest shape — not from an app name or a stream name.

Agent (drm agent run)

poll streamd ──► durable SQLite queue ──► claim a free ADB device
                                             │
                      POST credentials ◄── phone capture (phonecap)

Per-stream claims expire via a websocket heartbeat, so a dead agent releases its phones. The agent never installs apps and never runs a downloader. It drives whichever module a stream names; it contains no provider logic.

Where things land

Path	Contents
data/	durable secrets: .wvd, CA — gitignored
outputs/<app>/<stamp>/	capture + catalog sessions, with latest/ as a copy
outputs/discover/<stamp>/	MITM discover dumps
.cache/streamd/	streamd SQLite, work dirs, HLS output
.cache/ui-<app>/	UI dumps used by the automation

Honest status

Area	State
Capture (catalog + phone), both modules	working, verified against live services
Compiled-in modules, one binary	done
streamd API + SQLite + dashboard + claims	done
Synthetic manifest rewrite	done, verified against a live origin
Agent queue + phone refresh	done
Real downloader/ffmpeg workers in streamd	partial — the supervisor shells out to external tools and is the least exercised path