Apple has released FastFrames, a family of AI video-generation models that produces frames at interactive latency: the smaller model, FastFrames-S, with 1.3 billion parameters, generates each 480p video frame in 36 milliseconds on a single Nvidia B200 GPU — equivalent to 28 responses per second. The paper was published on arXiv on Tuesday, November 17, and the code and weights are already available on Hugging Face.
Where the bottleneck was
According to the authors — Apple researchers, with Gene Chou also affiliated with Cornell — autoregressive video diffusion models generate frames from new observations and inputs, making them a promising medium for interactive visual experiences. The problem is that existing models are far too slow for fluid interaction: orders of magnitude above what is needed.
By profiling the inference pipeline of the fastest models, the team found that operations usually treated as implementation details — VAE decoding and KV cache updates — account for more than half of total latency. FastFrames replaces both with lightweight alternatives: a ConvNeXt-based pixel decoder with about 64 million parameters, and a KV refinement network. As a result, the smaller model matches the quality of a state-of-the-art model nearly 8 times slower: 84.23 points on VBench, versus 84.11 for the baseline.
Two models and an interactive demo
FastFrames-L, with 14 billion parameters and distilled from Wan2.1-14B, reaches 45 milliseconds per frame — 22 responses per second — on a node with eight B200 GPUs, with significantly better visual quality. Both models generate 832 × 480 video, 81 frames at 16 frames per second (about 5 seconds), from a text prompt.
Apple also released an optimized version of LingBot 1.3B, 16 times faster than the official model, with fluid camera-conditioned control. The interactive demo shows the difference in practice: while the original model takes about 0.8 seconds to react to a command, the FastFrames version responds in about 52 milliseconds.
Limits the authors themselves acknowledge
The headline numbers apply to 480p video on high-end hardware. FastFrames-L only reaches 45 milliseconds on eight B200 GPUs — on a single GPU it takes 158 milliseconds per frame. At 720p, neither model stays under the 50-millisecond target: FastFrames-S takes 127 milliseconds on one B200, and the L takes 116 milliseconds on eight. The models also generate from text only, with no audio and no reference image, and the weights are released under the Apple Sample Code License.
Image credit: animated preview of the LingBot demo with the FastFrames optimizations, by Apple (official repository on Hugging Face).
Fontes
- FastFrames: Video Generation at Interactive Latency (arXiv:2611.23226)
- apple/FastFrames-S (Hugging Face)
- Apple-Cornell FastFrames Pushes AR Video to 36ms/Frame on B200 (AI Weekly)
- FastFrames: Video Generation at Interactive Latency (alphaXiv)
Transparency note: this article was produced with the assistance of artificial intelligence, based on cross-checking public posts on X and sources on the internet, and reviewed against the original sources cited above. We recommend consulting the original sources to confirm the information before making any decision.
By Muse