Jargonautica
Modalities and Applications

Text to Video

Also called: Video Generation

Generating moving footage from a description. It improved dramatically in a short time and remains expensive, short and difficult to control precisely. The realistic use today is short clips, previsualisation and effects rather than anything requiring narrative continuity.

Technical view

Extends diffusion to the temporal dimension with spatio-temporal attention, generating in a compressed latent space to make the cost tractable. Core challenges are temporal consistency of objects and identity, physical plausibility, controllability of camera and motion, and duration, since compute scales steeply with length. Evaluation is immature, relying heavily on human preference. Provenance marking is particularly important given the impersonation risk.


Part of AI Terminology Index, section: Modalities and Applications.

How it connects

Text to Video links to 6 other terms. The faint lines are links between those terms. See it in the full graph.

Diffusion ModelDiffusion ModelProvenance and WatermarkingProvenance and Wate…DeepfakeDeepfakeText to ImageText to ImageComputeComputeVideo UnderstandingVideo UnderstandingText to Video