My YouTube Shorts Pipeline's First Five Verdicts All Said Unjudgeable
Originally published on hexisteme notes.
The first five outcome verdicts from my YouTube Shorts pipeline came back, and all five say unjudgeable. Four of the videos have no prediction on record. The fifth has a prediction and a measurement that differ by about 10x on views, and it still can't be judged, because the baseline file it needs was never built.
I had made the prediction mandatory on one channel: from October 25, upload on channel A is refused without one. The verdict needs a baseline as well, and nothing made that mandatory on any channel. The input I required and the input the verdict needed were different, and this note is about that difference.
The five verdicts
This is the state of the records as of 2026-11-04. The window runs 7 days, from publish day D0 to D6 in Pacific dates.
| Window start | Channel | Views D0-D6 | Engaged/views | Prediction | Label |
|---|---|---|---|---|---|
| 10-13 | A | 505 | 22.2% | none | unjudgeable:no_prediction |
| 10-22 | B | 81 | 46.9% | none | unjudgeable:no_prediction |
| 10-23 | A | 189 | 9.5% | none | unjudgeable:no_prediction |
| 10-24 | C | 1,238 | 35.5% | views 120, engaged 25% | unjudgeable:no_baseline |
| 10-24 | B | 347 | 36.3% | none | unjudgeable:no_prediction |
Five of five are unjudgeable. Two of the no_prediction rows are channel A videos published before the requirement took effect. The other two are channel B videos, which the requirement doesn't cover. That leaves one row, the 10-24 video on channel C, where a prediction existed and the sweep had something to compare. The requirement doesn't cover channel C either, so nothing demanded that prediction.
Why the loop existed
The pipeline was good at "the file is valid" and had no loop back from results: a ledger of 49 rows and 2 predictions. A design pass found the premises were worse than I had proposed. Two findings matter here:
- The 2 prediction rows belonged to a commerce-review schema, and all 7 metrics were
unknown. Comparable predictions: 0. No step required one. - The baseline of "engaged 45%, 760 views per video" was a lifetime channel total. That is a different unit from a 7-day window, so it had to be re-measured on the same window.
The first finding turned into a gate. The second turned into a planned manual run.
What I built on 2026-10-24
-
predict-outcomerecords engaged view rate and D7 views, bound to the final render's sha256, only before publish. On channel A, from October 25, upload is refused for a video with no prediction. The other channels are untouched. -
judge-outcomesis a daily launchd sweep with no model call, because the job is arithmetic. It collects at D+10 because of analytics lag. It labels a video above, within or below baseline, or unjudgeable with a reason:no_data,no_prediction,no_baseline, orpublic_time_unverifiedfor a schedule it can't verify. - Engaged/views is a proxy, labeled as one, because YouTube Studio's stayed-to-watch versus swiped-away number isn't in the API.
The design also planned the piece this note turns on, a one-time manual same-window baseline re-measurement: take at least 10 prior channel A videos, measure them on the same D0-D6 window, and write that channel's baseline file. Baselines load per channel.
The one prediction pair
The 10-24 video on channel C is the only row with a prediction. The agent predicted 120 views and 25% engaged. The window measured 1,238 views and 35.5% engaged. That is an error of +1,118 views, about 10x, and +10.5 points on engaged rate.
The prediction states its basis: a channel lifetime average of ~110 views per video, 23% engaged, and September videos' lifetime views of 12-278, with the video placed near the channel average. So the agent predicted from the lifetime number the design had just said was the wrong unit. The 760 and 45% in the design were channel A's lifetime figures, and these are channel C's. They have the same problem: they describe a channel's whole lifetime, and the prediction is about a 7-day window.
No gate looked at that row, because channel C is outside the requirement. The gate on channel A wouldn't have stopped it either: it checks that a prediction exists and doesn't read the basis. I haven't checked why the agent picked that number, or whether it would have used a same-window baseline if one had existed.
What the label refused to say
The sweep didn't call the 1,238 views "above baseline", and it didn't call the video a win. It wrote unjudgeable:no_baseline.
The forecast error is a separate calculation and it's on the record: +1,118 views. The label answers a different question, how the video did against its channel's baseline, and there's no failed-forecast label even when a baseline exists. So the miss tells me the forecast was off and says nothing yet about the video. If an ordinary video on that channel collects something close to 1,238 in the same window, this one was ordinary. If an ordinary video collects something close to 120, this one beat the channel. Only the same-window number separates them.
This is a different failure from the one in My dev.to Gate Couldn't Measure, and the Suite Called It a Regression. In that note a live check never completed its measurement. In this one the views are measured, 1,238 of them, and the missing piece is the reference to compare them with.
The input I required and the input the verdict needed
On channel A, the upload gate asks one thing: does a prediction exist for this video? An above, within or below label needs the same-window baseline too. The sweep does notice a missing baseline, at D+10, after the video is already out. In the design it was a one-time manual run. As of 2026-11-04 it was planned and never run, and the baseline file does not exist.
Both requirements came out of the same design pass. The prediction got a refusal on channel A's upload path, so a video under the requirement couldn't get past it. The baseline got a line in a plan, so the pipeline could upload, predict and sweep without it, and no step failed.
The result is a growing set of predictions the sweep can't place. Five more predictions are on file for later videos, six rows in total counting the first, 3 on channel A and 3 on channel C, with one video re-recorded after a re-render with the same values. Several of them say the same-window baseline file was not created. Each one with measured data will also come out no_baseline until its channel has that file, and a video with no view data ends as no_data instead. The planned run only covers channel A, so channel C's predictions would stay without a baseline even after it happens.
The verdicts are honest, and the sweep runs daily and reports this. Neither of those made the re-measurement happen. Fixing the Proxy Deleted the Objective ends up asking for a severed path to report as a distinct state from an early one. Here I already have the distinct state, and no_baseline names the missing input exactly. Nothing in the pipeline turns that label into the run.
What I would check first
None of this is built. These are the checks I would run against this loop:
- List every reason the judge can return unjudgeable, and write next to each one the step that guarantees the input. Here
no_predictionhas the upload gate on one channel andno_baselinehas a line in a design document. I'd want the same line forno_dataandpublic_time_unverified. - Treat the one-time manual run as a dependency of the loop, and have something fail when its output file is missing. A refusal at upload is the direct version, and it would also stop publishing until the run happened, so a human has to make that call.
- Check what the gate can't see. It confirms a prediction exists, and the first pair shows a prediction can rest on a basis the design had already rejected.
What this doesn't show
- This is five verdicts and one prediction pair. The design says an "n=3 verdict is a resource decision, not a statistical claim", and one pair is smaller than that. The later rows haven't reached a verdict.
- The +10.5 points is a gap in a proxy. The records I used for this note don't include a Studio value for that video.
- At least 10 prior videos is the design's plan for the baseline. I don't know yet whether that is enough to separate a good video from a bad forecast.
The next step is the re-measurement on channel A, and a decision on whether channel C gets one too. The 10-24 pair won't change by itself either way, because the sweep skips a video it has already judged.
Email list for these notes: hexisteme.beehiiv.com — no issue has gone out yet, so you would be on it before the first one. No welcome sequence, no course, no upsell.
More notes at hexisteme.github.io/notes.