DeepSeek V3

DeepSeek V3 is DeepSeek's flagship dense mixture-of-experts model, released December 2024 (updated March 2025). It achieves frontier-level performance with significantly lower training cost than comparable models.

Key Specifications

• Parameters: 671B total, 37B activated per token
• Context window: 128K tokens
• Max output tokens: 8,192
• Architecture: MoE Transformer
• License: MIT (open-source)
• Training cost: ~$5.58M (estimated)

Architecture

DeepSeek V3 uses a MoE transformer with:

• Multi-head Latent Attention (MLA) — Efficient attention mechanism
• DeepSeekMoE — Fine-grained MoE with shared experts
• Multi-Token Prediction (MTP) — Predicts multiple tokens at once during training
• RoPE — Rotary Position Embeddings
• GQA — Grouped Query Attention

Capabilities

• Open weights — MIT licensed, freely available
• Code generation — Excellent coding performance
• Math reasoning — Strong mathematical capability
• Long context — 128K context window

Performance

DeepSeek V3 matches or exceeds GPT-4o on many benchmarks while being significantly cheaper to train and run. It's particularly strong on math and coding tasks.

Comparison

• See LLM API Pricing Comparison for cost comparison
• DeepSeek R1 — DeepSeek's reasoning model, built on V3
• Llama 4 — Meta's competing open model
• qwen-2.5 — Alibaba's competing open model
• See Open-Source vs Closed-Source LLMs for the broader comparison