Gemma 4 Model Sizes: Which One Should You Actually Run?

Google shipped four open-weight models under one name on April 2, 2026 — and the obvious choice, the biggest one, is probably wrong for most developers. With four Gemma 4 model sizes spanning edge devices to workstation-class reasoning, the question isn’t whether to use Gemma 4. It’s which one. This guide breaks down the hardware requirements, benchmark tradeoffs, and deployment targets for each model so you can skip the 20GB download you don’t need.

The Four Gemma 4 Model Sizes at a Glance

The naming convention tells you more than you’d think. The “E” prefix means Effective — as in effective parameters. These dense models use per-layer embeddings (PLE) that give each decoder layer its own small embedding table, maximizing parameter efficiency for on-device deployment. The total parameter count is larger, but “effective” compute is what the name reflects.

The “A” in 26B A4B means Active — it’s a Mixture-of-Experts (MoE) model where only a fraction of total parameters fire per token. The 31B is a straightforward dense model.

Model	Total Params	Active Params	Context	Modalities
E2B	5.1B (2.3B effective)	2.3B	128K	Text, Image, Audio, Video
E4B	8B (4.5B effective)	4.5B	128K	Text, Image, Audio, Video
26B A4B (MoE)	25.2B	3.8B	256K	Text, Image, Video
31B (Dense)	30.7B	30.7B	256K	Text, Image, Video

All four models ship under Apache 2.0, support native function calling, and include configurable thinking modes via the <|think|> token. Day-zero support covers Ollama, vLLM, llama.cpp, and LM Studio. The full specs are in the official Gemma 4 model card.

The non-obvious gotcha: audio input is edge-only. Only E2B and E4B support audio. The 26B and 31B handle text, image, and video — but not audio. If you’re building a voice-enabled agent, you either use an edge model or run a separate audio pipeline. Plan for this before you commit to a model size.

VRAM Decides Your Model — Here’s the Decision Matrix

Forget benchmarks for a moment. Your GPU’s memory ceiling picks your model before any performance metric does. Here’s what each model actually requires at Q4 quantization — the sweet spot between quality and memory for local deployment, per AvenChat’s hardware analysis.

Model	Q4 VRAM	Q8 VRAM	BF16 VRAM	Best GPU Fit
E2B	~2GB	~5GB	~15GB	Raspberry Pi 5, budget phones
E4B	~5GB	~8GB	~15GB	Laptops, M1/M2 8GB
26B A4B	~18GB	~28GB	~52GB	RTX 4090, M2/M3 Pro
31B	~20GB	~34GB	~62GB	RTX 5090, M2/M3 Max

Here’s the trap most developers fall into: the 31B fits on a 24GB RTX 4090 at Q4 — barely. But at that memory budget, you’re capped at roughly 45K context tokens. The 26B A4B fits on the same card with room to spare and delivers its full 256K context window. Same GPU, six times the context.

For Apple Silicon: M1/M2 with 8GB handles E2B and E4B at Q4. M2/M3 Pro with 18–36GB unified memory is the 26B A4B’s sweet spot. You need an M2/M3 Max (48–64GB) for the 31B at Q4/Q8, or an Ultra for BF16 full precision. NVIDIA’s RTX AI Garage has RTX-specific benchmarks if you want exact throughput numbers for your card — and for context on NVIDIA’s broader AI hardware roadmap, see our GTC 2026 coverage.

On the edge end, E2B runs on less than 1.5GB at 4-bit quantization, hitting 133 prefill tokens/sec on a Raspberry Pi 5 — CPU only, no GPU required.

Why the 26B A4B Is the Right Gemma 4 Model for Most Developers

The 26B A4B activates only 3.8B of its 25.2B total parameters per forward pass — 15% active compute. Yet it scores 88.3% on AIME 2026 versus the 31B’s 89.2%. One percentage point gap. One-sixth the compute. For context on how efficient that is: Kimi K2.5 has ~1T total parameters (MoE) with 32B active per token, and GLM-5 carries ~744B total parameters (MoE) with ~40B active — yet the A4B matches or approaches them on key benchmarks with just 3.8B active.

The practical math is simple. The 26B A4B fits on a single RTX 4090 (18GB at Q4) with full 256K context. The 31B on the same card loses 80% of its context window. Unless you have a 32GB+ card, the “bigger is better” instinct actively hurts you here.

Non-English quality adds another dimension. As @Hangsiin noted on X: “It doesn’t feel awkward in Korean at all… I had never felt this from a model of this size before.” For multilingual workloads, the A4B punches well above its active parameter count.

When should you choose the 31B instead? Three scenarios: you need maximum reasoning ceiling (the Codeforces ELO gap is real — 2150 vs. 1718), you have 32GB+ VRAM to spare, or you’re evaluating Gemma 4 as a frontier-adjacent API replacement. According to the Gemma 4 model card, the 31B also scores 84.3% on GPQA Diamond versus the A4B’s 82.3%. These gaps are real — but for the vast majority of development workflows, they don’t justify the hardware cost.

Apache 2.0: The License Change That Matters More Than Benchmarks

Gemma 4 is the first Gemma release under Apache 2.0, replacing the custom Gemma license that restricted commercial use. What that unlocks: unrestricted commercial deployment at any scale, no user limits, no blanket restrictions on using outputs to train other models, and irrevocable rights to use, reproduce, modify, and distribute.

Compare this directly to Llama 4. Meta’s license caps commercial use at 700 million monthly active users and requires any derivative model to include “Llama” in its name. Not a blanket prohibition, but legal friction Apache 2.0 doesn’t impose. As VentureBeat reported, that distinction matters more than benchmark deltas for companies building at scale.

Nathan Lambert put it sharply in his Interconnects analysis: “I will personally be so happy if the horrible Llama licenses were an ~18-month transient dynamic.” Qwen 3.5 also uses Apache 2.0, so the license alone doesn’t differentiate Gemma 4 — but combined with US-origin and hardware coverage from Raspberry Pi to data center, the package is hard to match.

Getting Started: One Command Per Platform

Ollama (v0.20.0+) is the fastest path. ollama run gemma4 pulls E4B by default. For other sizes:

ollama run gemma4:e2b    # Edge 2B
ollama run gemma4        # Edge 4B (default)
ollama run gemma4:26b    # MoE sweet spot
ollama run gemma4:31b    # Dense flagship

For production serving, vLLM handles the larger models well: vllm serve google/gemma-4-E4B-it --max-model-len 131072. For llama.cpp users: brew install llama.cpp && llama-server -m gemma-4-31b-Q4_K_M.gguf. Full setup details are in Google’s official Ollama integration guide.

Google recommends the same sampling parameters across all four models: Temperature 1.0, Top-p 0.95, Top-k 64. The zero-friction option for mobile developers is Google’s AI Edge Gallery app — it runs E2B and E4B fully offline on both iOS and Android. No setup, no server, no API keys. All models are also available on Hugging Face, Kaggle, Google AI Studio, and Vertex AI for cloud deployment.

Once running locally, Ollama exposes a REST API on port 11434 — same interface regardless of model size, making it simple to swap models without changing application code. Function calling and thinking modes work identically across all four sizes, which matters if you’re pairing Gemma 4 with Google ADK for multi-agent workflows.

One Caveat Before You Build: Fine-Tuning Friction Is Real

Lambert flags in his Interconnects piece that previous Gemma models had “tooling issues and poorer performance when being finetuned.” Gemma 4’s non-standard architecture — per-layer embeddings, hybrid sliding/global attention, softcapping, partial-dimension RoPE — may amplify this early on. These components can cause issues with fine-tuning frameworks expecting standard transformer layouts.

History suggests patience. Qwen 3.5 needed roughly 1.5 months post-release for proper open-source tooling integration. Axolotl v0.16 claims day-zero MoE + LoRA support with 15x speed and 40x memory gains — but claimed day-zero support and battle-tested support are different things. Lambert’s framing is worth internalizing: Gemma 4’s success hinges on “ecosystem integration and developer experience, not raw performance metrics.”

Practical guidance: if your workflow requires fine-tuning today, evaluate Qwen 3.5 27B in parallel until Gemma 4’s fine-tuning toolchain hardens. If you’re deploying pre-trained models — inference, RAG, agentic pipelines — proceed with confidence. And if you’re thinking about how Gemma 4 fits into the broader 2026 agent framework landscape, the pre-trained models are ready now.

The Bottom Line

The 26B A4B isn’t the compromise model — it’s the model that proves MoE efficiency has crossed the quality threshold where “smaller” no longer means “worse.” When 3.8B active parameters deliver 88.3% of the flagship’s AIME score on a single consumer GPU, the efficiency argument isn’t theoretical anymore. It’s the default.

The one question Gemma 4’s launch can’t answer: will its fine-tuning ecosystem catch up fast enough to lock in developer loyalty before Qwen 3.5’s tooling head start compounds into a permanent community advantage? The first community fine-tuning results — particularly for the MoE 26B A4B — should land on Hugging Face by late April 2026. That’s Gemma 4’s real benchmark.