Initial commit: Null DRM Official

Capture, decrypt, and restream toolkit with compiled-in app modules (RTE, TG4, BBC), on-device MITM proxy, streamd control plane, and www. BBC module.yaml is published (clear streams); other module values stay local.
2026-10-06 00:25:35 +02:00 · 2026-10-06 00:25:35 +02:00 · 2fa8f2435f
commit 2fa8f2435f
121 changed files with 17802 additions and 0 deletions
--- a/docs/modules.md
+++ b/docs/modules.md
@ -0,0 +1,379 @@
+# Writing an app module
+
+A module teaches the binary how to drive one streaming app. It is a Go package, it
+is compiled in, and it contains no secrets.
+
+Work through [capture.md](capture.md) first — you need the app's package id,
+license URL, manifest host and UI selectors before you can write any of this.
+
+## Anatomy
+
+```text
+apps/modules/myapp/
+  config.go      Config struct + defaults + flag binding
+  myapp.go       init(), the App type, Launch, AutoPlay, capability methods
+  steps.go       navigation idioms specific to this app
+  catalog.go     optional: resolve channels without a phone
+  mpd.go         optional: manifest rewriting
+  module.yaml    GITIGNORED - your values
+```
+
+Everything is registered from `init()`, and one line in `apps/modules/all/all.go`
+links it into the binary.
+
+## 1. Config
+
+Embed `provider.Common` for the fields every module has, then add your own.
+
+```go
+package myapp
+
+import (
+    "drmdecryption/modcfg"
+    "drmdecryption/modules/provider"
+)
+
+const Name = "myapp"
+
+type Channel struct {
+    Label         string   `yaml:"label"`
+    LiveCardIndex int      `yaml:"live_card_index"`
+    PlayDesc      []string `yaml:"play_desc"`
+}
+
+type Config struct {
+    provider.Common `yaml:",inline"`
+
+    // Provider values. Never give these defaults in code.
+    APIBase  string             `yaml:"api_base"`
+    Channels map[string]Channel `yaml:"channels"`
+
+    Timeouts Timeouts `yaml:"timeouts"`
+    UI       UI       `yaml:"ui"`
+}
+
+type Timeouts struct {
+    // provider.Duration accepts "25s" and bare seconds; time.Duration alone
+    // would force raw nanoseconds in YAML.
+    LaunchWaitUI provider.Duration `yaml:"launch_wait_ui"`
+    Playback     provider.Duration `yaml:"playback"`
+}
+
+type UI struct {
+    LaunchResourceContains []string `yaml:"launch_resource_contains"`
+    LaunchDescContains     []string `yaml:"launch_desc_contains"`
+}
+
+func (c Config) withDefaults() Config {
+    t := &c.Timeouts
+    t.LaunchWaitUI = provider.Duration(t.LaunchWaitUI.D(25 * time.Second))
+    t.Playback = provider.Duration(t.Playback.D(45 * time.Second))
+    return c
+}
+```
+
+`provider.Common` gives you `package`, `license_url`, `proxy_bin`, `ca_hash`,
+`aliases` and `score`.
+
+### Loading and overrides
+
+```go
+var flags struct {
+    packageName string
+    apiBase     string
+}
+
+func loadConfig() (Config, modcfg.Result, error) {
+    var cfg Config
+    res, err := modcfg.Load(Name, &cfg)      // reads apps/modules/myapp/module.yaml
+    if err != nil {
+        return cfg, res, err
+    }
+    cfg.Package = modcfg.Override(Name, "package", flags.packageName, cfg.Package)
+    cfg.APIBase = modcfg.Override(Name, "api_base", flags.apiBase, cfg.APIBase)
+    return cfg.withDefaults(), res, nil
+}
+```
+
+`modcfg.Override` applies **flag → environment (`MYAPP_API_BASE`) → file**. A
+missing `module.yaml` is not an error: the module must still construct so
+`drm modules` can report it as unconfigured.
+
+## 2. The App type
+
+```go
+func init() {
+    appreg.Register(Name, New)
+    appreg.RegisterFlags(bindFlags)
+}
+
+func bindFlags(fs *flag.FlagSet) {
+    fs.StringVar(&flags.packageName, Name+".package", "", "override "+Name+" package id")
+    fs.StringVar(&flags.apiBase, Name+".api-base", "", "override "+Name+" API base")
+}
+
+type App struct {
+    *provider.Base
+    cfg Config
+}
+
+func New() (appreg.App, error) {
+    cfg, res, err := loadConfig()
+    if err != nil {
+        return nil, err
+    }
+    labels := map[string]string{}
+    for id, ch := range cfg.Channels {
+        labels[id] = ch.Label
+    }
+    return &App{Base: provider.New(Name, cfg.Common, labels, res), cfg: cfg}, nil
+}
+```
+
+Flag binding is a separate hook from construction on purpose: flags must be bound
+to the config holder *before* any app is built, or overrides would be read too
+late.
+
+`provider.Base` supplies `Name`, `Package`, `LicenseURL`, `ProxyBin`, `CAHash`,
+`HasChannel`, `Label`, `ChannelIDs`, `CacheDir`, alias-aware `Resolve`, and
+`Hints(require…)` preloaded with your `score:` config.
+
+### Required methods
+
+```go
+func (a *App) KeyMode() string { return "raw" }   // or "modulardrm"
+
+func (a *App) CaptureHints() capture.Hints {
+    return a.Hints("license_url", "mpd")           // fields the MITM must yield
+}
+
+func (a *App) MPDRewriter() mpd.Rewriter { return mpd.Passthrough{} }
+
+func (a *App) Channels() []appreg.Channel {
+    out := make([]appreg.Channel, 0, len(a.cfg.Channels))
+    for _, id := range a.ChannelIDs() {
+        out = append(out, appreg.Channel{ID: id, Label: a.Label(id)})
+    }
+    return out
+}
+```
+
+`Hints(...)` names the JSON keys that must be present before a capture counts as
+complete. Common sets:
+
+| Field | Meaning |
+|---|---|
+| `mpd` | manifest URL (DASH or HLS — the field name is historical) |
+| `license_url` | license endpoint seen on the wire |
+| `pssh` | Widevine init data |
+| `auth` | Authorization header value |
+| `pid` | provider-specific id (e.g. a release/program id) |
+
+Ask only for what the app actually produces. Requiring `pssh` from an HLS app that
+puts it in the playlist will hang the capture.
+
+## 3. Launch and AutoPlay
+
+```go
+func (a *App) Launch(c *adb.Client) error {
+    c.EnsureAwake()
+    c.ForceStop(a.Package())
+    time.Sleep(400 * time.Millisecond)
+    if err := uiflow.MonkeyLaunch(c, a.Package()); err != nil {
+        return err
+    }
+    time.Sleep(time.Duration(a.cfg.Timeouts.LaunchSettle))
+    c.DismissShadeIfFocused()
+    return uiflow.WaitUI(c, a.CacheDir(), uiflow.Match{
+        ResourceContains: a.cfg.UI.LaunchResourceContains,
+        DescContains:     a.cfg.UI.LaunchDescContains,
+    }, time.Duration(a.cfg.Timeouts.LaunchWaitUI))
+}
+
+func (a *App) AutoPlay(c *adb.Client, channel string) error {
+    id, err := a.Base.Resolve(channel)
+    if err != nil {
+        return err
+    }
+    ch := a.cfg.Channels[id]
+
+    if err := a.openLivePage(c); err != nil {
+        return err
+    }
+    if err := uiflow.TapCard(c, a.CacheDir(), ch.LiveCardIndex,
+        time.Duration(a.cfg.Timeouts.Card), a.cfg.Cards); err != nil {
+        return err
+    }
+    return uiflow.WaitPlayback(c, a.CacheDir(), a.Package(), uiflow.PlaybackOpts{
+        Timeout:            time.Duration(a.cfg.Timeouts.Playback),
+        SoftFail:           true,
+        OrResourceContains: a.cfg.UI.PlaybackResourceContains,
+    })
+}
+```
+
+A `Launch` error is logged and tolerated by the capture pipeline, so pick a launch
+marker that genuinely exists — otherwise every run silently burns the full timeout
+before `AutoPlay` recovers. **Prefer resource ids over descriptions**: ids are
+stable and not localized. Dump the real tree to find them:
+
+```bash
+adb shell uiautomator dump /sdcard/ui.xml && adb shell cat /sdcard/ui.xml
+```
+
+### uiflow
+
+| Function | Use |
+|---|---|
+| `WaitUI(c, cache, Match, timeout)` | block until a node exists |
+| `TapUI(c, cache, Match, timeout)` | wait, then tap its centre |
+| `Find(c, cache, Match, timeout)` | get the node for custom handling |
+| `WaitPlayback(c, cache, pkg, PlaybackOpts)` | audio session, or a UI fallback signal |
+| `Cards(nodes, CardOpts)` | content rows in a dump, top to bottom |
+| `WaitCard` / `TapCard` | the nth content row, scrolling if needed |
+| `MonkeyLaunch(c, pkg)` | launcher-intent start |
+| `CompileRes([]string)` | case-insensitive regexes |
+
+`Match` combines `TextRE`, `DescRE`, `TextContains`, `DescContains`,
+`ResourceContains`, plus `Pick` (`largest`/`smallest`/`first`) and `MinArea`.
+
+`WaitPlayback` normally confirms via the audio session. Players that start without
+one need `SoftFail: true` plus a UI signal (`OrDescContains`,
+`OrResourceContains`) so capture proceeds anyway.
+
+`CardOpts` carries geometry (`MinY`, `MinH`, `MinW`) and denylists (`DescDeny`,
+`TextDeny`) so chrome is not mistaken for content. Defaults suit a 1080-wide
+phone; put real values in `module.yaml` if your app differs.
+
+## 4. Optional: catalog
+
+Implement `app.Catalog` and `drm catalog --app myapp` starts working.
+
+```go
+func (a *App) Resolve(channel string) (session.Stream, error) {
+    id, err := a.Base.Resolve(channel)
+    // ... hit the provider API, pick a manifest + license URL ...
+    return session.Stream{
+        Name: Name + "-" + id, App: Name, Channel: id,
+        MPD: manifestURL, LicenseURL: licenseURL,
+        HeadersJSON: "{}", Rewriter: "none", ManifestType: "hls",
+    }, nil
+}
+
+func (a *App) EnrichWithKeys(info *session.Stream, python, wvd string) error {
+    body, err := brightcove.Get(info.MPD)
+    // ... extract PSSH, then wvkey.FetchRawAll ...
+}
+```
+
+For a nicer `--list`, also implement `app.CatalogLister` returning
+`[]appreg.CatalogRow`.
+
+Decode remote config generically. Do not model a provider's JSON field names as Go
+struct fields — read into `map[string]any` and let `module.yaml` name the fields:
+
+```yaml
+channels:
+  main:
+    token_field: liveLoggedOutTokenIOI
+    stream_id_field: irelandStreamId
+```
+
+A renamed upstream field then needs an edit to your values file, not a rebuild.
+
+## 5. Optional: manifest rewriting
+
+Only needed when the captured manifest expires faster than the stream.
+
+```go
+func init() {
+    mpd.Register(Name, func() mpd.Rewriter { return NewRewriter() })
+}
+
+func (r *Rewriter) Name() string { return Name }
+
+func (r *Rewriter) Rewrite(upstreamXML []byte, kidHint string) ([]byte, error) {
+    // identify the origin channel from the manifest or the KID, then:
+    return mpd.BuildSynthetic(r.cfg.Synthetic, channel, originBase, kid)
+}
+
+// Optional: recover the origin when upstream is gone and only the KID remains.
+func (r *Rewriter) ResolveKID(kidHex string) (channel, originBase string, ok bool)
+func (r *Rewriter) Prime(channel, originBase string)
+```
+
+Register a **factory**, not an instance — a rewriter may hold per-stream state.
+
+## 6. Optional: streamd settings
+
+```go
+func (a *App) VideoSelect() string  { return "for=best" }
+func (a *App) AudioSelect() string  { return "lang=en:for=best" }
+func (a *App) RewriterName() string { return Name }    // or "none"
+func (a *App) LiveWaitSeconds() int { return 6 }
+func (a *App) TSReadyBytes() int64  { return 2 << 20 }
+```
+
+Without this, streamd uses neutral defaults. This is what replaced guessing from
+app and stream names.
+
+## 7. Link it in
+
+```go
+// apps/modules/all/all.go
+import (
+    _ "drmdecryption/modules/myapp"
+    _ "drmdecryption/modules/rte"
+    _ "drmdecryption/modules/tg4"
+)
+```
+
+```bash
+go -C apps/cli build -o ../../bin/drm .
+./bin/drm modules
+```
+
+## 8. Test it
+
+Assert the interfaces at compile time and prove no secrets leaked:
+
+```go
+var (
+    _ appreg.App            = (*App)(nil)
+    _ appreg.Catalog        = (*App)(nil)
+    _ appreg.StreamDefaults = (*App)(nil)
+)
+
+func TestNoHardcodedCredentials(t *testing.T) {
+    cfg := Config{}.withDefaults()
+    if cfg.Package != "" || cfg.APIBase != "" {
+        t.Fatal("provider values must come from module.yaml")
+    }
+}
+```
+
+`apps/modules/all/all_test.go` already checks every registered module constructs
+with no values file present, reports a valid key mode, and — if it declares a
+rewriter name — actually registered one.
+
+For anything needing the live service or your local values, use a build tag so the
+normal run stays hermetic:
+
+```go
+//go:build live
+```
+
+```bash
+go -C apps/modules test -tags live ./myapp/ -v
+```
+
+## Checklist
+
+- [ ] No account id, key, token, video id, hostname or KID in any `.go` file
+- [ ] `module.yaml` is gitignored (`git check-ignore -v apps/modules/myapp/module.yaml`)
+- [ ] `New()` succeeds with no `module.yaml`
+- [ ] Launch marker actually matches — watch for a silent full-timeout wait
+- [ ] `CaptureHints` asks only for fields the app really produces
+- [ ] `KeyMode` is correct (`raw` for binary challenges, `modulardrm` for JSON)
+- [ ] Added to `all/all.go`
+- [ ] `drm modules` lists it with the right channel count