Qwen2.5-VL 7B Instruct AWQ

Qwen2.5-VL-7B-Instruct-AWQ is a vision-language model (VLM) that processes images, videos, and text to perform visual understanding, localization, and structured output generation. It is optimized for efficient inference through AWQ quantization while maintaining high performance across diverse multimodal benchmarks.

Key Capabilities

• Dynamic Resolution & Video Understanding: The model handles images at native resolution and supports variable frame rate sampling for videos, enabling comprehension of content over one hour long and pinpointing specific events.
• Visual Agent & Localization: It can act as a visual agent for computer and phone use, generate bounding boxes or points for object localization, and produce stable JSON outputs for coordinates and attributes.
• Structured Outputs: Scanned invoices, forms, tables, and similar documents are processed into structured data, benefiting finance, commerce, and document analysis workflows.
• Extended Context: The model supports YaRN for context extension beyond 32,768 tokens, allowing handling of longer sequences with a factor of 4.

Benchmark Performance (AWQ Quantized)

Evaluated using VLMEvalKit, the 7B AWQ model achieves the following scores:

Benchmark	Score
MMMU_VAL (Accuracy)	55.6
DocVQA_VAL (Accuracy)	94.6
MMBench_DEV_EN (Accuracy)	84.2
MathVista_MINI (Accuracy)	64.7

For comparison, the BF16 version of the same model achieves 58.4, 94.9, 84.1, and 67.9 respectively. The 7B model also outperforms GPT-4o-mini on several college-level, math, document understanding, and video understanding tasks (source: official blog).

Architecture

The vision encoder uses window attention, SwiGLU, and RMSNorm, aligned with the Qwen2.5 LLM. M-RoPE with absolute time alignment enables temporal sequence learning, and the model supports dynamic resolution for both spatial and temporal dimensions.