letsautom8
← Lab notes

A course is a list of claims: agents attend, and my own work tests every lesson

The lab has a new research project: AI-assisted video. A short drama series, narrated shorts, the whole pipeline from script to cut. I knew how to automate things. I didn't know how to make a face keep still across ten shots, or why a lip-sync falls apart on the first word.

The fastest way in was other people's experience, so I joined four courses on AI video and growing a channel, three of them paid. The biggest had 132 open lessons, most of them videos. Watching all of it myself would have taken weeks, and most of it was written for someone else's work: photoreal influencers, short social ads, long-form explainers.

A course is a list of claims. Some will hold for my work and some won't, and the only way to find out is to try them.

So agents attended the courses for me, in three days. Every lesson became a row in a claims ledger, and every row gets a cheap test on our own footage.

Attending

The agents attend with the video-watch tool from an earlier post: a video becomes labelled frames and a timestamped transcript, which Claude can read. Not every lesson needed the frames. Where the platform had a caption track, the agent read that instead, and some lessons were text all along.

Four steps. Attend: lesson videos as frames plus transcript, lesson videos by caption track only, and lesson text and guides. All of it feeds the claims ledger, one row per lesson with the claim, where it came from, the tag untested, a pass bar and the cost to test. Then the test on our work: free first on footage we already have, paid only under a cap set beforehand. The verdict is held for us or didn't hold, each with evidence. Every result is tagged M for measured, J for our judgement by eye, or claim for what the course says. Not the agents' call: which lessons get tested, the spending cap, and anything about how it looks or sounds.
The ledger sits between the course and my footage. Nothing goes into the pipeline straight from a lesson.

On one course, 76 of the 80 videos in its main lesson set went through video-watch with frames. On the others it was mostly caption tracks and lesson text. The agents only read: anything that would have posted to a course's community came back to me as a draft, and they bought nothing.

The ledger

Each lesson becomes a row with the same fields. Here is one, condensed from the repo:

Claim     Fake a three-camera shoot: punch in on one clip for wide, medium, close
Source    one course's guide pages, paraphrased                 [claim]
Test      crop one approved 720p take to 70% and 45%, scale to 1080×1920
Pass bar  sharp at phone size, by eye
Cost      US$0
Result    pass at phone size [J]; at full size the 45% crop is visibly soft

Every row starts as untested. The pass bar is written before the test runs, so a result can't drift to fit what came back. The tags say how much a result is worth: [M] we measured it, [J] it's our judgement by eye, [claim] the course says so and nobody has checked.

The order is free first. Plenty of claims can be tested on footage we already have, or by a script that reads our own prompts. The ones that need a paid generation ride on shots we need anyway where possible, and only run under a spending cap set before they start.

Three results

Three rows. Punch in on one take for wide, medium and close, a course claim: crop one approved 720p take to 70 and 45 percent and scale to phone size, passing if sharp at phone size by eye; verdict pass at phone size, judgement, soft at full size, can't change the angle, needs my eye too. Paste the identity paragraph into every image prompt, a course claim: one A/B on one shot against our own 'as in the reference' line, passing on less drift from the face; verdict didn't hold, n equals 1, judgement, no closer to the face, and it drew her standing, not sitting. A sync score can pass or fail a lip-synced take, our own idea in the same ledger: score 24 takes of 10 lines, passing if it ranks them the way I picked them; verdict a ranking aid, not a gate, measured, agreed on 8 of 10, painted faces score low, so no bar.
Three verdicts, three different strengths of evidence. The tag says which.

The punch-in held, with limits. At phone size, all three crops of one 720p take read clean, because the painted style hides the upscale. At full size the close crop, a 3.3× blow-up, is soft. It can't change the angle or the eyeline, and a centre crop cut a character out of the frame, so the crop window has to be planned per shot. The course said it only works from a 1080p source; for our look, 720p was enough at phone size, by the agent's eye.

The identity paragraph didn't hold, once. One guide says to paste a character's description word for word into every image prompt. Another lesson from the same course says never to re-describe a face that has a reference attached. One A/B settled it for us, for now: the face came out no closer to the reference, and the description's body words drew her standing when the shot had her sitting on a bed. That's one sample, so it's a lean, not proof. We keep our own line.

The sync score is a ranking aid, not a gate. This one wasn't from a course. It was our own untested idea, and the ledger takes those too. A lip-sync scorer rated 24 takes of 10 lines. It agreed with the take I'd picked on 8 of the 10, with one tie and one clear disagreement. But painted faces score low: 9 of my 10 picks scored under 5, where the tool suggests about 6 or more for real faces. So it can rank takes of one shot, and flag a big gap, but it can't pass or fail one.

The whole first pass made four paid image calls, for US$0.268. Everything else ran free, on footage we already had and a GPU at home.

What the agents don't decide

I decided that every lesson gets tested. The agents write the rows, run the free tests and draft the paid ones. Three things stay with people:

Useful for research, not for learning

This is a research method. It works when the question is "which of these claims hold for my work?", and you'd rather have evidence than a summary.

Don't use it for a course you ought to learn yourself. If it's a skill you'll need, a certification, or anything you'll be expected to know, an agent attending it for you teaches you nothing. You'd have a tidy ledger and no new ability. I can talk about what held for our videos because I looked at every result that mattered; I still couldn't teach the course.

Lesson: an agent can read a course for you, but only you can learn it. Use the ledger where the goal is a decision, not a skill.

Compared with NotebookLM

The obvious free alternative is Google's NotebookLM, renamed Gemini Notebook in July 2026. I didn't run it side by side; this is from Google's own help pages.

Two columns. NotebookLM, now Gemini Notebook: a public YouTube link imports the caption text only; with no captions or no speech the import fails; it summarises, answers and cites the transcript; it is free with nothing to install; it stops at understanding. Our video-watch tool plus a claims ledger: any link or file gives frames from every shot plus a transcript; with no captions it transcribes the audio itself; it turns each lesson into a claim with a pass bar; it needs Claude Code, ffmpeg and uv; it tests the claim on our own work. Footnote: most of our course reading was captions only too. Frames matter where the lesson is visual: a crop, a framing, a light.
The bigger difference isn't the reader. It's what happens after the reading.

Given a YouTube link, NotebookLM imports the video's text transcript and nothing else. It needs a public video with captions, and a video with no captions or no speech won't import. An uploaded video file is transcribed as audio. What it does well: it's free, needs no setup, and its answers cite the transcript, so you can check where a claim came from.

To be fair to it, most of our own course reading was captions only too. Frames earned their place on lessons where the visuals carry the lesson: a crop, a framing, a lighting setup, a before and after. For a talking-head lesson, a transcript is most of it.

The real gap is the step after. NotebookLM summarises and answers questions about what a course says. It doesn't test anything on your work. You could run the same ledger with NotebookLM as the reader, and the tests would be the same.

Where it falls short

Lesson: write down how strong each result is, next to the result. A verdict without its tag gets repeated as fact.

If you try it

The video-watch tool is free on GitHub; the earlier post has the install steps.

← All lab notesQuestions? Say hi →
Say hi hello@
letsautom8.dev

Write to me about a post, an experiment, or something you'd like me to try in the lab. If you'd like me on your project, testing or automation, that goes through my company, so you get proper terms and a proper invoice.

Work with me through Shiloah →