My phone edits a family music video every night while I sleep

Every night an agent picks the day’s clips, picks a song, cuts a music video to the beat, and leaves it on my phone by morning. I didn’t write a line of it — an agent built the whole app.

At Google I/O 2026, Google unveiled Gemini Omni — a model that watches your video, finds the best moments, trims them, and assembles a finished clip. It’s genuinely good.

I already had the same thing on my phone. The twist isn’t that an AI edits my videos — it’s that an AI built the app that does it.

The dead-footage problem

I have tens of gigabytes of video on my phone. Kids, holidays, ordinary Tuesdays. How much does anyone watch? None. It’s 3-minute shaky clips where nothing happens for half of them. I will never scrub 40 clips for the one second where the little one does a flip. So it had to happen by itself.

What coolvideo actually does

The core bet: the agent is the editor’s eye, deterministic code is the hands. Python does the mechanics — samples frames, lays them out as labelled thumbnail grids (“contact sheets”). The agent looks at the grids and picks the interesting moments. No scoring heuristic, no every-N-seconds fallback, no human picking a single second.

The loop closes through the cloud: I drop clips into an Android app → they land in my own Cloudflare R2 bucket → on my PC an agent pulls them down, analyses loudness and frames, picks the moments, and cuts to the beat → the finished reel comes back to the phone’s Download tab.

The agent picks the music too

In music-video mode the agent isn’t handed a song. It looks at the day’s frames, invents genre/era search queries (“motown soul”, ”80s synthpop” — iTunes responds to style, not to “happy upbeat”), searches iTunes previews or the Internet Archive, picks a track, then cuts to its beat: detects the tempo, starts on the downbeat, and paces shots to the song’s loudness arc — fast cuts in the chorus, calm shots in the verse. It once chose ELO’s Mr. Blue Sky for holiday footage. It fit perfectly.

War stories — because it didn’t work first try

The agent debugged real, nasty problems the way a human would — each one now a fix: commit with a test:

• The phone lies about orientation. ffprobe calls a portrait clip 1920×1080 with rotation=-90. Edit naively and the reel comes out on its side. You have to read the true size from a decoded frame.
• Cuts drifted off the music. Cutting with -ss start -to end stretched each ~0.34s shot by a few frames — across ~50 cuts that’s 2 seconds past the song, every cut sliding off the beat.
• The tempo detector locked onto half the BPM. Autocorrelation often hears 87 where the song is 174, so cuts come twice too slow. It now doubles a too-slow tempo when the signal supports it.
• Cloudflare served a stale reel for four hours. The edge cache holds GETs ~4h; without Cache-Control: no-store the app kept downloading yesterday’s reel after a rebuild. Invisible in tests, infuriating in real life.

That list is the whole difference between a demo and something that works on my actual videos, from my actual phone, two weeks later.

It runs while I sleep

A one-line cron entry builds and uploads a reel fully unattended at 21:30 — no tap required. The title isn’t a metaphor.

Who wrote it: not me

It’s my idea and my architecture. The code is the agent’s (Claude Code), autonomously:

• The entire Android app in a single afternoon — Kotlin + Jetpack Compose, the agent writing, testing, and fixing as it went.
• Test-driven from the first line: ~240 tests on the Python pipeline, 26 on the app, all offline. Dozens of small commits — test: → feat: → fix:, one step each. Not a code dump; iteration.
• Release APK ~4 MB, six editing modes, the bugs above all caught and fixed by the agent.

To be fair to Google: Gemini Omni does things mine doesn’t — conversational editing, video from text, captions, vertical reframing, no infrastructure of your own. For creative production from scratch, it wins. But the full experience is subscription-gated in the Gemini app (specifics in the footnote). For my family reels — free, private, editing my real footage, with my own modes — bespoke wins.

So what — if you’re not editing baby clips

This is exactly the work I do on weekdays, just on more boring data. The shift isn’t “AI did the task.” It’s that an agent will build you the tool that does the task — forever, unattended, with tests — instead of you renting a feature inside someone else’s product.

The tens of GB of dead footage now turn themselves into reels I actually send to grandma.

---

Gemini Omni was announced at Google I/O 2026 (May 19); clips are capped at ~10 seconds at launch (a deployment choice, not a model limit). The full conversational experience is behind an AI Plus/Pro/Ultra subscription in the Gemini app, with a free, capped version on YouTube Shorts and YouTube Create. Sources: Google blog, Gemini Omni overview.