← Lab

OUT OF MEMORY

Made with generative AI music: MiniMax-Music3 image: Qwen-Image 2.1 int8video: MiniMax H3 ref2va, turbo 8-stepvoice: VoxCPM2upscale: AetherScale ×2edit: HyperFrames

OUT OF MEMORY is a lyric video staged as a fashion show. The collection is GPU POOR 16 GB, and each look is a trick for squeezing a big AI model onto a home graphics card. Look 02 is int8 quantization: weights stored in 8 bits take half the memory of 16-bit weights. Look 03 is a LoRA, a small add-on trained on a few dozen photos to teach the model a face or style. In Look 04 the heroine drops her jacket, just as offloading shifts part of a model from the card’s memory into ordinary RAM.

A memory meter in the corner climbs throughout the film and redlines in every chorus. In the bridge the card runs out, the screen fills with “CUDA out of memory”, and then “Offloading to system memory” lets the show go on. The final look is FULL PRECISION, and the departures board flips to TASTE · FP32 · FITS: you can quantize anything except taste.

How it was made

One morning, before heading to the office, I gave the Claude Code agent another lyric video for inspiration, access to ComfyUI (the app I use to run AI models on my own machine) and a free hand. The agent borrowed the fashion-show format from that video; the theme, song and heroine are new. Every image, audio and video model ran locally on a single RTX 5080 with 16 GB of memory. Only the agent itself ran in the cloud.

  • It wrote the lyrics and generated several versions of the song in MiniMax Music 3, then, unable to hear them, picked one by measuring tempo, word clarity and how complete the structure was.
  • In Qwen-Image it cast the heroine and created the set images.
  • The moving shots are 21 MiniMax H3 clips. For the singing ones, the model got the matching slice of the song’s own vocal to sync the lips to.
  • It mapped the beat and word timings, then built the edit and graphics in HyperFrames, which renders HTML to video.
  • The last line is spoken by a clone of the AI singer’s voice, because the song from Music 3 stopped at the chorus.

The film runs 2:43. The first master was ready about 11 hours after the first message. Most of that went on two breaks after the agent hit its Claude usage limit, while the card worked through its queue. The agent itself worked about two hours. Generation cost nothing beyond electricity.

When I got back, I sent a few notes: captions drifting off the vocal in places, black text lost on a black jacket and a pointer to a different upscaler. The agent made those fixes too; the video above is the result.

What it shows

  • Clips locked to the song. Each clip is generated for its own stretch of the track and stays there, so the edit can cut anywhere. Lip sync has to be judged by eye, as automatic measurement proved unreliable. The same method suits a music video or an ad with a singing character.
  • Changes take minutes. All graphics follow the word timings and the beat, so moving a word or adding a clip takes a one-second rebuild and a few minutes of rendering. My notes made it into the film that same evening.
  • One card was enough. Every model used in the film fit on the same 16 GB card. Video is the slow part, at about 5.5 minutes per 10 seconds of footage. Another take costs GPU time and power, with no fee per attempt.

Who did what

Idea, lyrics, direction
Claude Code (Opus 5.5), autonomously
Prompts, edit, motion design
Claude Code (Opus 5.5), autonomously
Inspiration and pipeline
Marcin: the inspiration, a free hand and the local ComfyUI setup

Tools

  • ComfyUI 0.37 (local, RTX 5080 16 GB)
  • MiniMax Music 3 (the song)
  • whisper.cpp large-v3-turbo (word clarity when picking the song version)
  • Qwen-Image 2.1 int8 (the heroine and the frames)
  • MiniMax H3 ref2va, turbo 8-step (moving shots)
  • MelBandRoFormer (vocal stem)
  • MMS_FA and wav2vec2 LV60K in torchaudio (word timings, forced alignment)
  • librosa (beat grid)
  • SAM3.1 (character tracking)
  • BiRefNet (cutouts for parallax)
  • VoxCPM2 (the voice in the outro)
  • AetherScale: Restoration, Neural Rendering and VSR ×2 (upscaling)
  • HyperFrames 0.8.90, HTML and GSAP (motion design and edit)