Tencent has released Youtu-Parsing-Omni, a compact 5-billion parameter omni-modal parsing model designed to convert diverse media inputs into structured JSON data. According to documentation hosted on Hugging Face and GitHub, the model is capable of processing seven distinct parsing families, ranging from document pages and natural images to audio clips and video files.
Technical Capabilities and Unified Schema
The model utilizes a unified schema to produce a single JSON envelope for various input types. Depending on the task prompt provided via the --task parameter, the model performs both perception and cognition tasks.
For document pages, the model extracts layout elements with bounding boxes, text, LaTeX, and Markdown charts or Mermaid flowcharts. When processing natural images, it identifies entities and text with accompanying captions and global descriptions. For audio inputs, the model provides timestamps, speaker identification, Automatic Speech Recognition (ASR), and acoustic event descriptions. In the case of video, the model can process both natural video—detailing visual elements, actions, and camera motion—and text-rich video, which generates a structured Markdown report combining OCR and ASR data.
The model employs an “omni encoder” that allows for the simultaneous processing of image, audio, and interleaved audio-visual inputs within a single architecture.
Performance Benchmarks
Technical reports indicate that Youtu-Parsing-Omni achieves high performance relative to its size. The model scored 96.96 on the OmniDocBench v1.6 and achieved an average score of 75.08 on the OmniParsingBench, making it the highest-performing open-weight model on that specific benchmark, trailing only Gemini-3-Pro.
The model also shows competitive results in specialized recognition tasks, specifically in chemical-structure (ChemOCR) and music-score (PDMX-Synth) recognition, according to data released on Hugging Face.
Deployment and Implementation
Tencent has designed the model for ease of service, providing it as a vLLM plugin. The release includes pinned serving settings, task prompts, and inference examples to facilitate integration.
While the model is optimized for vLLM, discussions on community platforms such as Reddit suggest potential interest in future support for other frameworks like llama.cpp. Developers have noted the model’s potential utility in automated workflows for data extraction and conversion across various media formats.