BLIP-2 OPT 2.7B

Salesforce/blip2-opt-2.7b is a vision-language model (VLM) that generates text conditioned on an image and optional textual input. It uses a frozen CLIP-like image encoder, a frozen OPT-2.7b language model, and a lightweight Querying Transformer (Q-Former) that learns to bridge the modality gap. The Q-Former is a BERT-like transformer that maps learnable query tokens into embeddings that the language model can attend to. The model is pre-trained in two stages: first for vision-language representation learning, then for vision-to-language generative learning, both keeping the image encoder and LLM frozen. BLIP-2 achieves strong zero-shot results on vision-language benchmarks. On VQAv2, it scores 65.0, outperforming Flamingo80B by 8.7 percentage points despite having 54× fewer trainable parameters. On NoCaps zero-shot captioning, it achieves a CIDEr score of 121.6, surpassing the previous best of 113.2. The model supports tasks including image captioning, visual question answering, and chat-like interactions by feeding the image and prior conversation as prompt. The model inherits limitations common to large language models, such as potential bias, hallucination, and sensitivity to training data quality. It has not been tested in real-world applications and is intended for research purposes. The pre-trained checkpoint is available through gigarouter’s hosted API, enabling access without local infrastructure.