SigLIP Base Patch16 256

google/siglip-base-patch16-256 is a zero-shot image classification model that performs language-image pre-training using a sigmoid loss function, enabling robust zero-shot classification and retrieval without requiring global pairwise similarity normalization.

How it works

Unlike standard contrastive learning with softmax normalization, SigLIP computes a pairwise sigmoid loss on each image-text pair independently. This decouples the loss from batch size, allowing effective training at both small and large batch sizes. The model was pre-trained on the WebLI dataset (English image-text pairs) at 256x256 resolution. Text inputs are tokenized to 64 tokens; images are resized and normalized with mean 0.5 and standard deviation 0.5 per channel.

Training and compute

The model was trained on 16 TPU-v4 chips over three days. The paper demonstrates that a batch size of 32k is sufficient, with diminishing returns up to 1 million. The sigmoid loss simultaneously supports scaling to larger batches and performs better at smaller ones.

Benchmark performance

The underlying approach achieved strong zero-shot results: the SigLiT variant (SigLIP combined with Locked-image Tuning) attained 84.5% top-1 accuracy on ImageNet zero-shot classification, trained on only 4 TPUv4 chips in two days. The evaluation comparison between SigLIP and CLIP from the original paper is shown below.

Additional details

The model was presented as an Oral paper at ICCV 2023. It was introduced in Zhai et al., Sigmoid Loss for Language Image Pre-Training (arXiv:2303.15343), and the pre-trained weights are open-sourced via Google Research’s big_vision repository (Apache 2.0 license).