Turn any video into a breakdown a model can work from.
One API call. Video Analysis API reads the video once and returns what is in it as data: every word with its time, every cut and what kind it is, the shots and scenes, every caption and graphic with its box, and who is in it, with a picture of each.
Below: part of what the API returned for a 13-second vertical video. The whole result is 15 KB.
{
"video": {"name":"17-1003576625514683.mp4","duration":13.467,"width":720,"height":1280,"fps":30},
"transcript": { "language": "en", "words": [
{"w":"Dental","s":0.24,"e":0.72,"c":0.983,"spk":0},
{"w":"powder","s":0.72,"e":1.12,"c":0.965,"spk":0},
{"w":"cleans","s":1.12,"e":1.52,"c":0.999,"spk":0},
… 25 more ] },
"cuts": [ {"frame":105,"t":3.5,"kind":"cut"}, {"frame":202,"t":6.733,"kind":"cut"}, … ],
"shots": [ {"n":2,"s":3.5,"e":6.733,"scene":2,"kind":"cut"}, … ],
"elements": [
{"kind":"caption","text":"This effortless new way","s":3.5,"e":4.75,"box":{"x":0.104,"y":0.702,"w":0.791,"h":0.043}},
… 11 more ],
"characters": { "cast": [
{"label":"A","name":"Golden retriever","kind":"animal","role":"pet","scenes":[3,5,6],"portrait_url":"https://…signed…/characters/A/scene-03-2.jpg"} ] }
}How it works
Three calls with curl. The result is plain JSON, and the same facts as text when you ask for it.
Send the video
Ask for an upload URL, then put the bytes to it. Up to 10 minutes and 300 MB. A public https URL works too.
POST /v1/uploads
{ "filename": "clip.mp4", "bytes": 2812322 }Start the job
One request, nothing to choose: every job runs the whole analysis. The estimate is held on your credit.
POST /v1/analyses
{ "upload_id": "up_…" }Get the breakdown
Poll the job until it is done. Then fetch the result and hand it to any model or program.
GET /v1/analyses/an_…/result → the breakdown, as JSON
Every route, with request and response bodies, is in the route reference.
What comes back
Five parts, all on one clock: seconds from the start of the video. Positions are boxes as fractions of the frame, so they hold at any resolution.
| Part | What it holds |
|---|---|
| Transcript | Every word with its start, its end and its speaker, and the sentences they make. |
| Cuts | Every cut on its frame, and what kind it is: a hard cut, a jump cut, a punch in or out, a zoom. |
| Shots and scenes | The shots between the cuts, grouped into scenes: a hard cut starts a scene, the rest stay inside one. |
| Elements | Every caption, title, inserted image or clip, logo, sticker, arrow, price and button: its text, its box and its times. |
| Cast | Who the video is about, each with a label that holds across scenes, a description, and a crop of every appearance. |
Measured where a number can be measured: the cuts come from the pixels, and a caption's start and end are put on the frame where its box changes. Observed where a model looked: what an element says and who a person is. The result, field by field.
Why not let the model watch the video?
It can, and it will tell you roughly what happens. It will not give you the frame a cut lands on, the box a caption sits in, or the same answer twice. The breakdown is the part a model is bad at, done once and written down.
| Watch the video | Read the breakdown | |
|---|---|---|
| Times | Rough, and different on the next run. | Cuts on their frame; words and elements to the hundredth of a second. |
| Positions | "A caption near the bottom." | A box for every element and every person, as fractions of the frame. |
| The second question | The whole video again. | The same 15 KB of text. Nothing to upload. |
| A different model | Another upload, and possibly a different answer. | The same file and the same facts. |
| A program | Nothing to parse. | JSON: a script can cut, caption or re-time from it without a model. |
Questions the example's result answers
Each answer below is read from the result on the Example page, with the times and boxes it records.
- Where are the cuts? 5 cuts, at 3.50s, 6.73s, 8.67s, 10.07s, 11.37s: 6 shots in 6 scenes.
- What does the second caption say, and where does it sit? "This effortless new way", from 3.50s to 4.75s, in a box 79% of the frame wide, 70% down from the top.
- Who is in it? One subject, labelled A: golden retriever with light golden coat, in scenes 3, 5, 6, with 6 crops to show what it looks like.
- What is said, and when? 28 words in 2 sentences, one speaker. The first word, "Dental", starts at 0.24s.
What people build with it
Anything that has to work on a video without watching it: the breakdown is the structure, ready for an agent or a script.
- Remaking a video. An agent reads the shots, the lines, the captions and the cast, and rebuilds the same structure with new footage, shot for shot and caption for caption.
- Agentic video editing. Give an agent the breakdown instead of the footage. It picks the cut points and the lines from the data, and only touches the video to render the result.
- Research across many videos. Analyze a set of videos and ask questions across all of them: how fast they cut, what the captions say, what is on screen in the first three seconds.
- A searchable library. Thousands of videos become a folder of text that a model can search by spoken words, on-screen text and who appears.
- Subtitles and chapters. A script can derive these from the result without running a model: subtitles from the timed words, chapters from the scenes.
Start with $5 of credit
Enough for 5 minutes of video. No card needed to try it.