Content Presets: Teaching the Pipeline What Kind of Video It’s Processing

The attention module computes a weighted sum of six signals per frame: face centroid, pose centroid, optical flow direction, YOLO object centroid, audio energy, and speech activity. The weights are a configuration problem — and the right weights depend entirely on what kind of video you're processing.

A podcast subject barely moves. Smooth, slow following is correct. An action clip moves fast — tight crop tracking with optical flow dominance is correct. Nature footage may have no detectable face at all.

The 9 built-in presets

Each preset is a 6-weight vector summing to 1.0. A podcast preset up-weights face and speech. A sport preset up-weights optical flow and YOLO objects. Cinematic up-weights composition. The preset is the opinion about what matters in this frame.

Three architecture approaches considered

• A: Pass preset name to Lambda → Lambda looks up config.yaml
• B: Pass preset name to Batch container → container loads config.yaml
• C: Resolve weights at job creation time in the API → store as resolved_weights JSON on the job record → pass as WEIGHTS env var to Batch

Approach C was chosen: it eliminates a Lambda→DB dependency, avoids race conditions on custom preset rename/delete, and makes the job record self-contained. You can reconstruct exactly what the pipeline received without any external lookup.

Enterprise custom preset builder

Enterprise accounts get a drag-slider UI to define named presets with arbitrary weight vectors. These live in a per-account presets table, are validated on write and read, and flow through the same resolved_weights mechanism. No special cases in the pipeline — it only ever sees a weight vector.

The production incident

Three bugs in the first production deployment, all from the same commit. The most damaging: preset_resolver.py loads config.yaml at module import time. The Dockerfile only copied the api/ subdirectory. config.yaml lives at the repo root.

Result: every ECS task crash-looped on startup. 23 failed tasks before the fix — one line in the Dockerfile: COPY config.yaml .

The lesson is not "don't load config at import time." The lesson is: any file loaded at import time that lives outside the copied directory must be explicitly listed in the Dockerfile. Obvious in retrospect. Invisible until the container crashes.