Moondream 2

Moondream 2 is a vision language model (VLM) that combines image understanding, captioning, visual question answering, object detection, and pointing in a compact architecture designed for efficient deployment across devices. The model is available in 2B and 0.5B parameter variants, with the larger version offering robust general-purpose performance.

Key Capabilities

• Grounded Reasoning: A step-by-step reasoning mode that explicitly ties answers to spatial positions in the image before responding, improving precision for tasks such as chart median calculations and accurate counting.
• Object Detection: Uses reinforcement learning on high-quality bounding-box annotations to reduce object clumping and distinguish fine-grained categories (e.g., “blue bottle” vs. “bottle”).
• Faster Text Generation: A “superword” tokenizer with a lightweight transfer hypernetwork yields 20–40% faster response generation without accuracy loss, and eases future multilingual support.
• UI Understanding: ScreenSpot [email protected] improved from 60.3 to 80.4 in the latest release, making the model effective for UI element localization.
• Visual Querying & Pointing: Supports free-form questions, long-form captioning, open-vocabulary image tagging, and point-based location (e.g., “person”).

Notable Benchmark Results

Training and Architecture Details

Moondream 2 (2025-04-14 release) was trained on approximately 450 billion tokens, significantly fewer than comparable models. A custom second-order optimizer balances conflicting gradients across tasks, and a self-supervised auxiliary image loss accelerates convergence. Reinforcement learning fine-tuning was applied across 55 vision-language tasks, with plans to expand to ~120 tasks. The model’s context length is 2048 tokens.