amdenterpriseai/aim-google-gemma-4-12b-it repository overview

⁠Gemma 4 12B IT

Gemma 4 12B IT is the instruction-tuned variant of Google's Gemma 4 12B Unified (Gemma4UnifiedForConditionalGeneration). It is a dense (non-MoE) 48-layer model with a heterogeneous attention topology (head_dim 256 on the sliding-attention layers, global_head_dim 512 on the full-attention layers), a 1024-token sliding window, and 256K native max_position_embeddings. The architecture is encoder-free for every modality: it ships no SigLIP vision tower and no audio tower, instead projecting raw image patches and raw 16 kHz audio waveform frames directly into the text backbone through lightweight linear layers, so text, image, audio, and video are all first-class inputs. Audio clips are capped at 30 seconds and video is processed as a sequence of frames. These profiles serve Google's own quantization-aware-trained w4a16 checkpoint (compressed-tensors pack-quantized, 4-bit symmetric group-32 integer weights with bfloat16 activations), which the AIM catalog records as int4 precision. The 512-dim full-attention head restricts vLLM to the TRITON_ATTN and ROCM_AITER_UNIFIED_ATTN backends, which the profiles pin explicitly per accelerator.