Initial commit: Null DRM Official

Capture, decrypt, and restream toolkit with compiled-in app modules
(RTE, TG4, BBC), on-device MITM proxy, streamd control plane, and www.
BBC module.yaml is published (clear streams); other module values stay local.
This commit is contained in:
404errordeveloper 2026-10-06 00:25:35 +02:00
commit 2fa8f2435f
121 changed files with 17802 additions and 0 deletions

58
docs/providers/bbc.md Normal file
View file

@ -0,0 +1,58 @@
# BBC module — filling in `module.yaml`
BBC iPlayer (Android). Streams from the `mobile-phone-main` mediaset are **clear**
(CDN signed URLs); capture only needs the **MPD/HLS master**. Transparent MITM keeps
NordVPN UK on the phone.
`apps/modules/bbc/module.yaml` is published in the repo (package + channel map only;
no secrets). Skeleton:
```yaml
package: bbc.iplayer.android
proxy_bin: bin/proxy-android-arm64
ca_hash: "6c3578b4"
score:
hosts:
- open.live.bbc.co.uk
- vs-cmaf-push-uk
- vod-dash-uk
- akamaized.net
- bbci.co.uk
- bidi.net.uk
deny:
- telemetry.api.bbci
- bag.api.bbc
- thumbnail
aliases:
one: bbcone
bbc1: bbcone
channels:
bbcone:
label: BBC One
vpid: bbc_one_london
bbctwo:
label: BBC Two
vpid: bbc_two_england
```
## Capture
```bash
# Leave NordVPN UK ON. Magisk CA should already be mounted (skip --reinject).
./bin/drm capture --app bbc --channel bbcone --wait 120
# Play BBC One on the phone when prompted.
```
Console prints the MPD in a banner; `outputs/bbc/<stamp>/session.json` holds the
same fields (`mpd` required; no `key` / `pssh` for clear streams).
Replay:
```powershell
ffplay -user_agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120.0.0.0 Safari/537.36" -i "<mpd url>"
```
PC needs a UK egress IP (VPN). Some CDNs reject HEAD; use GET / ffplay as above.

224
docs/providers/rte.md Normal file
View file

@ -0,0 +1,224 @@
# RTE module — filling in `module.yaml`
DASH + ModularDrm Widevine, captured from the phone. There is no public catalog, so
every channel needs one phone capture to bootstrap.
Create `apps/modules/rte/module.yaml`. It is gitignored. Nothing below is shipped
in the repo — you obtain it yourself from your own device.
## Skeleton
```yaml
package: <android package id>
license_url: "<Widevine ModularDrm endpoint, incl. its account query string>"
proxy_bin: bin/proxy-android-arm64
ca_hash: "6c3578b4"
score:
hosts:
- <manifest CDN host>
- <ad-stitching host, if the app uses one>
deny:
- <EPG / schedules host fragment>
aliases:
one: chanone
two: chantwo
channels:
chanone:
label: "Channel One"
chip_desc: ["^chanone$"]
play_desc: ["play\\s*chanone"]
chantwo:
label: "Channel Two"
chip_desc: ["^chantwo$"]
play_desc: ["play\\s*chantwo"]
# Origin encoders behind the live channels.
origins:
vc11: "https://<origin host>/live/<path>/vc11/vc11.isml/"
vc12: "https://<origin host>/live/<path>/vc12/vc12.isml/"
kids:
vc11: "<uuid-form KID>"
vc12: "<uuid-form KID>"
```
## Where each value comes from
### `package`
Open the app on the phone, then:
```bash
adb shell dumpsys window | grep mCurrentFocus
```
The part before `/` is the package id.
### `license_url`
Run a discover session and play a channel ([capture.md](capture.md)):
```bash
./bin/drm proxy discover --install-ca --reinject
grep -iE 'ModularDrm|widevine|license' outputs/discover/*/appproxy.log
```
Take the full URL including its query string — the account reference in it is part
of the endpoint. The module also accepts it per run:
```bash
./bin/drm capture --app rte --rte.license-url 'https://.../ModularDrm?...'
```
### `score.hosts` / `score.deny`
`hosts` are the hosts that legitimately serve manifests; `deny` are hosts or URL
fragments that must never be mistaken for one.
```bash
grep -oE '"host":"[^"]+"' outputs/discover/*/appproxy_traffic.jsonl \
| sort | uniq -c | sort -rn | head -20
```
Put the origin and any ad-stitching host in `hosts`. Put EPG/schedule feeds and
analytics in `deny` — this app's schedule feed otherwise scores as a manifest
because its URL looks plausible.
Format-level scoring (`.mpd`, `.m3u8`, `/manifest`, `.isml`) is built in; you only
supply the provider-specific parts.
### `channels.*.chip_desc` and `play_desc`
These are the regexes that locate the channel chip on the Live tab and the large
hero Play button. They match the node **content-description**, case-insensitively.
Navigate to the Live tab by hand and dump the tree:
```bash
adb shell uiautomator dump /sdcard/ui.xml
adb shell cat /sdcard/ui.xml | grep -oE 'content-desc="[^"]*"' | sort -u
```
You are looking for two things:
- a small chip per channel — its description is usually the bare channel name, so
anchor the regex (`^chanone$`) to avoid matching a longer label
- a large "play <channel>" button that appears after the chip is tapped
The module taps the hero Play directly when it is already on screen, otherwise it
taps the chip first and waits for Play to appear. A match is only accepted as the
hero button if its area is at least `hero_min_area` (default 200000), which keeps
small list-item play icons from winning.
### `origins` and `kids` — the important pair
These let the manifest rewriter keep working after the ad-stitched manifest session
expires. Both come from a capture.
**The KID** is in the capture session:
```bash
./bin/drm capture --app rte --channel chanone --auto-play --wvd data/device.wvd
jq -r '.kid' outputs/rte/latest/session.json
```
Convert it to dashed UUID form for the `kids:` map
(`8-4-4-4-12`), e.g. `df1633821dddfdd5bec9822c1ec0f052` becomes
`df163382-1ddd-fdd5-bec9-822c1ec0f052`. Either form is accepted, but UUID form is
easier to read.
**The origin** is the Smooth Streaming base URL. Find it in the discover traffic:
```bash
grep -oE 'https?://[^"]+\.isml[^"]*' outputs/discover/*/appproxy_traffic.jsonl \
| sed -E 's#(\.isml).*#\1/#' | sort -u
```
The last path element before `.isml` is the encoder channel id, which becomes the
map key. So `https://host/live/tc-1/vc11/vc11.isml/` is keyed `vc11`.
**Pair them up** by capturing each channel in turn and noting which KID appeared:
```bash
for ch in chanone chantwo; do
./bin/drm capture --app rte --channel $ch --auto-play --wvd data/device.wvd
echo "$ch -> $(jq -r .kid outputs/rte/latest/session.json)"
done
```
Then confirm the mapping resolves:
```bash
go -C apps/modules test -tags live ./rte/ -v
```
That test fetches each configured origin's manifest, checks the KID resolves back
to its channel, and verifies the synthetic manifest lands on the live encoder's
segment grid. If a KID does not resolve, `kids:` and `origins:` disagree.
### `ca_hash`
The subject hash of the MITM CA, which is what the Magisk mount installs as
`<hash>.0`. `6c3578b4` is the bundled CA; change it only if you generated your own.
```bash
adb shell 'ls /system/etc/security/cacerts/'
```
## Optional tuning
All of these have working defaults — set them only if the app changes.
```yaml
encoder_channel_re: '^(vc|channel)\d+$' # which origin ids use the encoder timeline
origin_marker: .isml # path element identifying an origin BaseURL
hero_min_area: 200000 # smallest node accepted as the hero Play
ui:
launch_desc_contains: ["live tab", "header logo"]
live_tab_desc_re: ["Live tab"]
timeouts:
launch_wait_ui: 25s
live_tab: 30s
chip_wait: 20s
play_wait: 15s
playback: 45s
synthetic:
video_timescale: 600
audio_timescale: 48000
video_name: video
audio_name: audio_128k
video_bandwidth: 6000000
audio_bandwidth: 128000
width: 1920
height: 1080
live_edge_offset_s: 12
window_segments: 6
```
The `synthetic` block describes the origin encoder. Its defaults match a common
Smooth-to-DASH setup; if your origin uses different stream names or bitrates, read
them off the origin manifest:
```bash
curl -s 'https://<origin>/live/<path>/vc11/vc11.isml/Manifest' | head -40
```
`StreamIndex Name="…"` gives `video_name` / `audio_name`, and `Bitrate="…"` gives
the bandwidths. Segment duration and phase are learned at runtime, not configured.
## Verify
```bash
./bin/drm modules # channel count should match your file
./bin/drm capture --app rte --channel chanone --auto-play --wvd data/device.wvd
go -C apps/modules test -tags live ./rte/ -v
```
A good capture yields `auth`, `pid`, `pssh` and `mpd`, and a single `KID:KEY` whose
KID matches the `kids:` entry for that channel's encoder.

288
docs/providers/tg4.md Normal file
View file

@ -0,0 +1,288 @@
# TG4 module — filling in `module.yaml`
Brightcove Live: HLS manifests and a raw Widevine license. The playback catalog is
public, so channels resolve **without a phone** once you have the account id and
policy key. Phone capture also works and is the only route for channels the catalog
does not list.
Create `apps/modules/tg4/module.yaml`. It is gitignored. None of these values ship
in the repo.
## The DRM stack
| Piece | Detail |
|---|---|
| Manifest | HLS `.m3u8` (prefer the DVR playlist) |
| License | `POST` a raw binary challenge to `.../lic/wv?token=<JWT>` |
| License auth | JWT in the **query string**, not a header |
| Key mode | `raw` (not the JSON ModularDrm wrapper) |
| Keys per stream | several — a multi-KID SAMPLE-AES/cbcs stream needs all of them |
| PSSH | from the HLS playlist's Widevine `KEYFORMAT` line, not from a manifest |
Also published on the same host: FairPlay (`/lic/fp`) and PlayReady (`/lic/pr`).
## Skeleton
```yaml
package: <android package id>
proxy_bin: bin/proxy-android-arm64
ca_hash: "6c3578b4"
account_id: "<brightcove account id>"
policy_key: "BCpkAD..."
playback_config_url: "https://<host>/<path>/playback_config.json"
score:
hosts:
- <live CDN host>
deny:
- <analytics host fragment>
aliases:
main: chanone
plus1: plus
channels:
chanone:
label: "Main"
video_id: "<brightcove video id>"
stream_id_field: <field in playback_config holding that id>
token_field: <field in playback_config holding the playback JWT>
live_card_index: 0
kids:
label: "Kids"
video_id: "<brightcove video id>"
token_field: <field name>
live_card_index: 1
sport:
label: "Sport"
phone_only: true # no catalog entry; capture from the phone
live_card_index: 3
ui:
launch_resource_contains:
- <a view id present on the home screen>
```
## Where each value comes from
### `package`
```bash
adb shell dumpsys window | grep mCurrentFocus
```
### `playback_config_url`
The app fetches a JSON config at startup that carries the live stream ids and
playback tokens. Find it in a discover session:
```bash
./bin/drm proxy discover --install-ca --reinject
grep -oE 'https?://[^"]+(config|playback)[^"]*\.json' \
outputs/discover/*/appproxy_traffic.jsonl | sort -u
```
Fetch it and look at the keys:
```bash
curl -s '<playback_config_url>' | jq 'keys'
```
You will see entries along the lines of `<region>StreamId` and
`liveLoggedOutToken<REGION>`. **Those names go in your channel entries**, not in Go
— the module decodes the config generically, so a renamed upstream field is a
one-line edit here.
### `account_id`
The Brightcove account id appears in the catalog request path and in the manifest
URL:
```bash
grep -oE 'edge\.api\.brightcove\.com/playback/v1/accounts/[0-9]+' \
outputs/discover/*/appproxy_traffic.jsonl | sort -u
```
### `policy_key`
A `BCpkAD…` string the app sends as `Accept: application/json;pk=<policyKey>` on
every catalog call. It is embedded in the APK and visible on the wire:
```bash
grep -oE 'pk=BCpkAD[A-Za-z0-9_-]+' outputs/discover/*/appproxy_traffic.jsonl \
| sort -u
```
Treat it as a secret: keep it in this gitignored file, or pass it per run.
```bash
./bin/drm catalog --app tg4 --channel chanone --tg4.policy-key 'BCpkAD...'
export TG4_POLICY_KEY='BCpkAD...'
```
### `channels.*.video_id` vs `stream_id_field`
Two ways to name the Brightcove video for a channel:
- `video_id` — pin it directly. Stable, and what you want once you know it.
- `stream_id_field` — read it from `playback_config.json` at run time. Survives the
provider rotating ids.
Set both and `video_id` wins. Find a channel's id by matching the config field to
the manifest URL you captured — the id appears as the first path segment:
```text
https://<cdn>/<videoId>/<region>/<accountId>/<jwt>/playlist-hls-dvr.m3u8
```
### `channels.*.token_field`
The field in `playback_config.json` holding the long-lived JWT used as
`livePlaybackToken` on the catalog call. Each channel/region has its own. Without
the right one the catalog returns no playable source.
### `channels.*.live_card_index`
Only used for phone capture. It is the channel's position in the app's Live page
card list, counting from 0, top to bottom.
Open the Live page by hand and dump the tree:
```bash
adb shell uiautomator dump /sdcard/ui.xml
adb shell cat /sdcard/ui.xml
```
Content cards are the clickable rows below the title area. The module finds them by
geometry and ignores chrome (Search, Profile, Cast, drawer). Confirm your indices by
running a capture and checking the tap y-coordinate changes between channels:
```text
[*] tap card #0 @ 540,732
[*] tap card #1 @ 540,1494
```
If the wanted index is off screen the module scrolls and re-counts, so a long list
still works.
### `channels.*.phone_only`
Set it for a channel that has no catalog entry. `drm catalog` then fails with a
message pointing at phone capture instead of a confusing "no video id" error.
### `ui.launch_resource_contains`
This app's home screen exposes almost no useful `content-desc` — only chrome like
Search, Profile, Cast and the drawer button — so readiness is keyed off **view
ids**, which are stable and not localized.
```bash
adb shell uiautomator dump /sdcard/ui.xml
adb shell cat /sdcard/ui.xml | grep -oE 'resource-id="[^"]*"' \
| sed 's/.*\///' | sort -u
```
Pick a container that only exists once the home screen has rendered, e.g. the
content rail's refresh layout or the drawer toolbar.
This matters: without a marker that actually matches, every capture burns the full
launch timeout before auto-play recovers. It still works, just 30 s slower each time.
### `score.hosts` / `score.deny`
`hosts` is the live CDN host that serves manifests. `deny` is for analytics — this
provider's metrics beacons carry `.m3u8` in a query string and otherwise score as
manifests.
```bash
grep -oE '"host":"[^"]+"' outputs/discover/*/appproxy_traffic.jsonl \
| sort | uniq -c | sort -rn | head -20
```
## Verify
Catalog path first — it needs no phone:
```bash
./bin/drm modules
./bin/drm catalog --app tg4 --list
```
```text
CHANNEL CATALOG ID LABEL
chanone 6383454185112 Main
kids 6383455473112 Kids
sport - Sport
provider flags: cula4Enabled=true liveEnabled=true ...
```
A `-` in CATALOG ID means neither `video_id` nor `stream_id_field` resolved.
`provider flags` are the booleans from `playback_config.json`, handy for spotting a
channel the provider has switched off.
Then resolve one for real:
```bash
./bin/drm catalog --app tg4 --channel chanone --keys --wvd data/device.wvd
```
Expect an HLS DVR manifest, a `/lic/wv?token=` license URL, a PSSH, and several
`KID:KEY` lines.
Finally, cross-check the phone path against it — independent routes agreeing on the
PSSH and the full key set is strong evidence both are configured correctly:
```bash
./bin/drm catalog --app tg4 --channel chanone --keys --wvd data/device.wvd --json > a.json
./bin/drm capture --app tg4 --channel chanone --auto-play --wvd data/device.wvd
diff <(jq -r '.keys[]' a.json | sort) <(sort outputs/tg4/latest/key.txt)
```
## Optional tuning
```yaml
stream_name_prefix: tg4 # prefix for generated stream names
cards:
min_y: 300 # ignore nodes above this (title area / chrome)
min_h: 250
min_w: 600
desc_deny: ["Search", "Profile", "Cast", "Open", "Closed"]
text_deny: ["catch-up"]
ui:
live_label: live # text identifying the Live menu entry and title
live_title_resource: <view id of the Live page title>
drawer_desc: ["Open", "Closed"]
drawer_fallback_x: 68 # tapped when the drawer button cannot be found
drawer_fallback_y: 183
menu_max_x: 700 # bounds for a drawer menu entry
menu_min_y: 250
menu_max_y: 700
playback_desc_contains: ["video player"]
playback_resource_contains: [<player view id>]
timeouts:
launch_wait_ui: 30s
live_nav: 25s
card: 25s
playback: 60s
```
The player often starts without a dumpsys audio session, so playback is confirmed
by `playback_resource_contains` / `playback_desc_contains` and the wait is
soft-failing: capture continues even if neither signal appears.
## Troubleshooting
| Symptom | Cause |
|---|---|
| `INVALID_POLICY_KEY` (HTTP 401) | wrong or rotated `policy_key` |
| `no playback token for channel "x" (field Y)` | `token_field` is not a key in `playback_config.json` |
| `no video id for channel "x"` | set `video_id`, or a `stream_id_field` that exists |
| `no HLS source in playback response` | the channel is off air, or the token is for a different region |
| `launch: ui node not found` | `ui.launch_resource_contains` does not match — re-dump the tree |
| `tap card #N` picks the wrong channel | card order changed; re-check `live_card_index` |
| Fewer keys than expected | a multi-KID stream needs every key; check `key.txt`, not just `key` |