OmniLottie: Generating Vector Animations via Parameterized Lottie Tokens

NeurIPS 2025
Author 11,2*, Author 22*, Author 31, Author 42, Author 51,
Author 62, Author 72, Author 82‡, Author 91‡, Author 101
*Equal Contribution, ‡Correspondence Authors
1Fudan University, 2StepFun
arXiv Code Hugging Face Logo Weights Hugging Face Logo MMLottie-2M Dataset Demo

Abstract

OmniLottie is a versatile framework that generates high-quality vector animations from multi-modal instructions. For flexible motion and visual content control, we focus on Lottie, a light-weight JSON formatting for both shapes and animation behaviors representation. However, the raw Lottie JSON files contain extensive invariant structural metadata and formatting tokens, posing significant challenges for learning vector animation generation. Therefore, we introduce a well-designed Lottie tokenizer that transforms JSON files into structured sequences of commands and parameters representing shapes, animation functions and control parameters. Such tokenizer enables us to build OmniLottie upon pretrained vision–language models to follow multi-modal interleaved instructions and generate high-quality vector animations. To further advance research in vector animation generation, we curate MMLottie-2M, a large-scale dataset of professionally designed vector animations paired with textual and visual annotations. With extensive experiments, we validate that OmniLottie can produce vivid and semantically aligned vector animations that adhere closely to multi-modal human instructions.

Highlights

OmniLottie Teaser
OmniLottie is capable of generating high-quality vector animations across a wide spectrum of complexity — from simple icons to intricate animated illustrations. It demonstrates remarkable versatility through multiple generation modalities, including Text-to-Lottie, Text-Image-to-Lottie, and Video-to-Lottie, making it a powerful and flexible solution for diverse creative and professional animation tasks.

Method Overview

OmniLottie Pipeline
OmniLottie is built on a pre-trained vision-language model (Qwen2.5-VL) and incorporates a Lottie tokenizer. The model tokenizes both text and image/video inputs as prefix tokens, while the Lottie tokenizer encodes vector animation commands into a unified representation space. The Lottie tokenizer abstracts Lottie animations into compact sequences of animation commands and control parameters, removing redundant metadata while preserving full generative flexibility.

Lottie Tokenizer

Lottie Tokenizer
Our Lottie tokenizer converts raw JSON files into compact sequences of shape, effect, and animation commands with their associated parameters. This representation substantially shortens sequence lengths (81% reduction compared to raw JSON), improves token efficiency, and enables OmniLottie to learn a clean distribution over animation primitives. The tokenizer supports five fundamental layer types: Precomposition, Solid, Null, Shape, and Text layers, each identified by a unique type parameter.

Generation Process of OmniLottie

Text-to-Lottie Generation

Text-Image-to-Lottie Generation

Video-to-Lottie Generation

Comparison with SOTA Methods

Text-to-Lottie

OmniLottie achieves near-perfect success rates, the best FVD, and the strongest motion alignment compared to baselines including DeepSeek, GPT-4o, and Recraft.

Text-to-Lottie Comparison

Text-Image-to-Lottie

OmniLottie ranks first in FVD, object alignment, and motion alignment while maintaining high reliability. Methods such as AniClipart and LiveSketch exhibit low success rates and significantly longer runtimes.

Text-Image-to-Lottie Comparison

Video-to-Lottie

OmniLottie preserves temporal and structural fidelity most effectively, achieving the best FVD, PSNR, SSIM, and DINO scores.

Video-to-Lottie Comparison

Quantitative Results

Quantitative Results
OmniLottie achieves 97.3%, 92.0%, and 90.7% success rates for Text-to-Lottie, Text-Image-to-Lottie, and Video-to-Lottie tasks respectively, substantially outperforming all baselines while producing richer token sequences that enable more expressive and detailed vector animation generation.

MMLottie-2M Dataset

MMLottie-2M Dataset
The MMLottie-2M dataset consists of two million Lottie animations paired with multi-modal annotations including text descriptions, keyframe images, and rendered videos. Data is collected from major online platforms (LottieFiles, IconScout, Flaticon, Iconfont, Icons8) and augmented with SVG-derived animations. The dataset undergoes comprehensive processing including Lottie cleaning, spatial-temporal normalization, video rendering, and coarse-to-fine captioning using VLMs.

Data Processing Pipeline

Data Processing Pipeline
Our data processing pipeline includes five stages: (1) Data collection from web sources and SVG conversion, (2) Lottie cleaning to remove non-visual and non-parameterizable layers, (3) Spatial-temporal normalization to 512×512 canvas and unified timeline, (4) Video rendering with random backgrounds, and (5) Multi-modal annotation using a coarse-to-fine VLM captioning strategy.

Related Links

We thank the following excellent open-source works:

OmniSVG: A unified framework for SVG generation that leverages pre-trained VLMs. We build upon its methodology for vector graphics generation.

IconShop: The first work leveraging LLMs to generate monochrome, icon-level SVGs. We referred to its parametric implementation.

Concurrent Works:

AniClipart: Animates static clipart using text-to-video diffusion priors.

LiveSketch: Generates sketch animations through optimization-based approaches.

StarVector: Equips LLM with an image encoder for Image-to-SVG generation.

Acknowledgements

We thank all the contributors for dataset construction and valuable discussions.

BibTeX

@article{omnilottie2025,
  title={OmniLottie: A Unified Scalable Vector Animation Generation Model},
  author={Author1 and Author2 and Author3 and Author4 and Author5 and Author6 and Author7 and Author8 and Author9 and Author10},
  journal={arXiv preprint arXiv:XXXX.XXXXX},
  year={2025}
}