How to Run DeepSeek V3.2 Locally: Complete Setup Guide for 2026

DeepSeek V3.2 is impressive on paper—685B total parameters, 37B active via Mixture-of-Experts (MoE), 128K context window. DeepSeek V3.2-Speciale scored 96% on AIME 2025, beating GPT-5 High’s 94.6% on mathematical reasoning. But if you want to run DeepSeek V3.2 locally, the full 685B model requires eight H200 GPUs and $200K. The good news? You don’t need the full model. Distilled versions at 32B, 14B, and 7B parameters deliver frontier-class performance on consumer hardware.

The architecture’s secret weapon is DeepSeek Sparse Attention (DSA)—an efficient attention mechanism that reduces computational complexity from O(L²) to O(Lk), delivering approximately 50% compute reduction for long-context tasks. This guide will show you three honest tiers for running DeepSeek V3.2 locally based on your actual hardware, not aspirational specs.

Understanding DeepSeek V3.2 specifications

DeepSeek V3.2 isn’t a monolithic 685B model—it’s a Mixture-of-Experts architecture where only 37B parameters activate per token. The 685B total includes 671B of main model weights and 14B of Multi-Token Prediction (MTP) module weights. This MoE design is what makes local deployment even theoretically possible.

The model uses Multi-head Latent Attention (MLA) combined with DeepSeekMoE architecture, available in FP8, BF16, F32, and F8_E4M3 precision formats. According to benchmark results from December 2025, DeepSeek V3.2 achieves 2-3× faster long-context inference and 6-7× reduced processing costs compared to V3.1. On HMMT 2025, it scored 99.2% versus Gemini 3 Pro’s 97.5%.

DSA enables context windows exceeding 1 million tokens while cutting computational costs by approximately 50% and reducing per-token GPU costs by up to 2× in long-sequence models. This isn’t marketing hype—it’s peer-reviewed research from the January 2026 DeepSeek V3.2 paper.

Hardware requirements: Three honest tiers

Let’s be brutally honest about what it takes to run DeepSeek V3.2 locally. The marketing materials skip this part, but your wallet won’t.

Enterprise tier: Full 685B model

• Hardware: 8x H200 GPUs (80GB each)
• Cost: $200,000+
• Reality: Not for individuals

Prosumer tier: 32B distilled (recommended)

• Hardware: Single RTX 4090 (24GB VRAM)
• Cost: $1,600-$2,000
• Performance: 30-38 tokens/second, outperforms GPT-4
• VRAM usage: ~14.9GB

Consumer tier: 7-14B distilled

• Hardware: RTX 4080, RTX 4070, or Apple M3 Max
• Cost: $800-$1,200
• Performance: Viable for development and testing
• VRAM usage: 5-10GB

Model Size	GPU	VRAM	Tokens/Sec	Use Case
685B	8x H200	640GB	N/A	Enterprise research
32B	RTX 4090	~15GB	30-38	Professional development
14B	RTX 4080	~8GB	15-25	Development/testing
7B	RTX 4070	~5GB	10-20	Local experimentation

According to comparative GPU benchmarks, the RTX 4090 offers approximately 75% of H100 performance at a fraction of the cost, making it the sweet spot for serious local deployment.

Option 1: Setting up with Ollama (fastest path)

Ollama is the easiest way to run DeepSeek V3.2 locally. It handles model downloads, quantization, and API setup automatically. Installation takes approximately 5 minutes.

Install Ollama

On macOS or Linux:

# macOS/Linux
curl -fsSL https://ollama.com/install.sh | sh

# Or via Homebrew
brew install ollama

On Windows, download the installer from ollama.com.

Pull and run DeepSeek V3.2

Ollama’s DeepSeek V3.2 library offers multiple parameter sizes. Replace X below with your desired size (7b, 14b, 32b, 671b):

# Pull the model (replace X with 7b, 14b, 32b, etc.)
ollama pull deepseek-v3.2:Xb

# Example: Pull the 32B model
ollama pull deepseek-v3.2:32b

# Run interactive session
ollama run deepseek-v3.2:32b

For cloud-based inference without local downloads, use the deepseek-v3.2:cloud variant, which connects to DeepSeek’s API but maintains Ollama’s familiar interface.

Use Ollama’s API

Ollama runs a local API server on localhost:11434. Here’s a Python example using the Ollama SDK:

import ollama

# Generate a response
response = ollama.chat(model='deepseek-v3.2:32b', messages=[
  {
    'role': 'user',
    'content': 'Explain DeepSeek Sparse Attention in one paragraph.',
  },
])

print(response['message']['content'])

# Or use the raw API
import requests

response = requests.post('http://localhost:11434/api/chat', json={
  'model': 'deepseek-v3.2:32b',
  'messages': [{'role': 'user', 'content': 'Hello!'}]
})

print(response.json())

This local API approach works identically to cloud-based LLM APIs but runs entirely on your hardware. Perfect for privacy-sensitive applications or offline development.

Option 2: Advanced setup with llama.cpp

For maximum control over quantization and performance, llama.cpp offers lower-level access to model inference. This path requires more technical knowledge but delivers better performance tuning.

Download GGUF models from Unsloth

Unsloth’s GGUF repository provides pre-quantized DeepSeek models optimized for llama.cpp. GGUF (GPT-Generated Unified Format) is the standard quantization format that balances model size, speed, and accuracy.

Quantization	Bits/Weight	Size (32B)	Quality	Use Case
Q4_K_M	4.0	~18GB	Good	Best balance
Q5_K_M	5.0	~22GB	Better	RTX 4090 sweet spot
Q8_0	8.0	~34GB	Excellent	Maximum quality
IQ1_S	1.58	~10GB	Experimental	Extreme compression

According to Unsloth’s dynamic quantization research, their 1.58-bit version scores 69.2% on benchmarks while the 2-bit version achieves 91.7%—proving that aggressive quantization can maintain usable accuracy.

Build llama.cpp

# Clone and build
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

# For NVIDIA GPUs (CUDA)
make LLAMA_CUDA=1

# For Apple Silicon (Metal)
make LLAMA_METAL=1

# For CPU only
make

Run the model

For an RTX 4090 with 24GB VRAM running the 32B Q5_K_M quantization:

./llama-cli \
  --model DeepSeek-V3.2-32B-Q5_K_M.gguf \
  --n-gpu-layers 40 \
  --threads 16 \
  --ctx-size 16384 \
  --temp 0.6 \
  --prompt "Explain DeepSeek Sparse Attention."

The --n-gpu-layers parameter controls how many transformer layers offload to the GPU. For 32B models on RTX 4090, 35-45 layers typically fits within VRAM. Monitor GPU memory usage with nvidia-smi and adjust accordingly.

The 32B sweet spot: Best performance-per-dollar

The 32B distilled DeepSeek model isn’t a compromise—it’s the practical choice that outperforms GPT-4 on many benchmarks while running on a single RTX 4090. According to real-world testing, properly configured RTX 4090 setups achieve 30-38 tokens per second with the 32B model.

The distilled 32B model maintains the core architecture innovations—DSA, MLA, and MoE—while reducing VRAM requirements from 640GB to ~15GB. GPU utilization typically exceeds 90% when optimized correctly, meaning you’re extracting maximum value from your hardware.

Compare this to the API pricing: at $0.28 per million input tokens and $0.42 per million output tokens, heavy local usage pays for an RTX 4090 within months. The 32B model’s 80-120 tokens/sec inference speed means responsive, production-grade performance for coding assistants, content generation, and research applications.

API alternative: When local doesn’t make sense

Not everyone should run DeepSeek V3.2 locally. DeepSeek’s API pricing as of September 2025 makes cloud inference compelling for many use cases:

• Cache hit (input): $0.028 per million tokens
• Cache miss (input): $0.28 per million tokens
• Output: $0.42 per million tokens

Context caching reduces costs by 90% for repeated requests with shared prefixes—automatically enabled by default. New users receive 5 million free tokens upon registration with no credit card required.

The break-even analysis is straightforward: if you’re processing less than 50 million tokens monthly, API access is cheaper than the upfront hardware investment. For reference, 50 million tokens equals approximately 37.5 million words or 75,000 pages of text. Most individual developers and small teams fall well below this threshold.

Local deployment makes financial sense when you have privacy requirements, need guaranteed availability, or consistently process massive volumes. Otherwise, the API’s $0.028 cached input pricing beats the electricity costs of running an RTX 4090 24/7.

VS Code integration with Continue

Once you have DeepSeek V3.2 running via Ollama, Continue.dev turns it into a coding assistant directly in VS Code. This is what makes local LLMs practical for daily development work.

Install the Continue extension from the VS Code marketplace, then configure it to point at your local Ollama instance. Edit ~/.continue/config.json:

{
  "models": [
    {
      "title": "DeepSeek V3.2 32B",
      "provider": "ollama",
      "model": "deepseek-v3.2:32b",
      "apiBase": "http://localhost:11434"
    }
  ],
  "tabAutocompleteModel": {
    "title": "DeepSeek V3.2 7B",
    "provider": "ollama",
    "model": "deepseek-v3.2:7b"
  }
}

This configuration uses the 32B model for chat and complex queries while falling back to the faster 7B model for autocomplete suggestions—balancing quality and responsiveness. Your code never leaves your machine, and there are no API rate limits.

For more on local AI coding workflows, see our comparison of AI coding assistants and the Claude Code tutorial.

Conclusion: Start with 14B, scale to 32B

Don’t start with the 32B model. Pull Ollama’s 14B variant first, test it on your hardware, and verify your workflows. The 14B model runs on most modern GPUs with 12GB+ VRAM and delivers surprisingly good results for experimentation.

Once you confirm DeepSeek V3.2 fits your use case, upgrade to the 32B model if your hardware supports it—or stick with the API for the full 685B experience. The beauty of DeepSeek’s MoE architecture and Ollama’s quantization is that you have genuine options beyond “pay for cloud” or “give up.”

For architectural deep dives, read our analysis of DeepSeek’s mHC architecture. The DSA innovation didn’t happen in isolation—it’s part of a broader research push to make frontier AI accessible on consumer hardware.