Leaderboard
Leaderboard
The leaderboard system tracks model performance across benchmarks, providing data-driven rankings that power model selection for the Arena Workflow and Content Pipeline.
Model Performance Tracking
CodeIntel Benchmark Leaderboard
Hosted at codeintel.xyz/benchmarks/, tracking 19+ models on unified benchmarks:
Top Tier (85+ score):
- Groq GPT-OSS 20B — 86.9 (#1) — Best overall
- Groq GPT-OSS 120B — 86.4 (#2)
- DeepSeek V4 — top tier
- Qwen 3.5 Omni Plus — 85.6 (#5 overall)
- Nemotron — top tier
Mid Tier (77-82):
- Hy3
- Xiaomi MiMo
- Devstral
- Codestral
- Groq Llama models
- Gemma local
- Mistral Large
Artificial Analysis Benchmarks (ToolBrain Arena)
Three benchmark indices used for arena model selection:
- intelligence_index — General reasoning/intelligence
- coding_index — Code generation capability
- agentic_index — Agentic task performance
Four free models from OpenRouter qualified with all three benchmarks populated (Jul 2026):
| Model | Provider | Intel | Coding | Agentic |
|---|---|---|---|---|
| Gemma 4 26B | 25.7 | 39.3 | 11.0 | |
| Nemotron 3 Super | Nvidia | 25.4 | 37.7 | 8.7 |
| Nemotron 3 Nano 30B | Nvidia | 14.2 | 14.4 | 2.0 |
| GPT-OSS 20B | OpenAI | 14.9 | 20.7 | 3.1 |
Design Arena Leaderboard
From intelligence.ai/leaderboard (website category, Jul 2026):
| Rank | Model | Elo |
|---|---|---|
| #1 | GLM-5.2 (z.ai) | 1347 |
| ... | (visual design tasks) | ... |
Builder vs Judge Model Roles
Builder Models
Used for generating content, code, and tool pages. Selected based on task type:
| Task Type | Builder Model | Provider |
|---|---|---|
| Code / tool building | DeepSeek V4 Flash | DeepSeek |
| Research / analysis | GLM-5.2 | z.ai |
| Creative / design | Xiaomi MiMo 2.5 Pro | Xiaomi |
| Quick / simple | Xiaomi MiMo 2.5 | Xiaomi |
| Default fallback | DeepSeek V4 Flash | DeepSeek |
Judge Models
Used for evaluating builder outputs. Prefer free models:
| Judge Model | Provider | Cost |
|---|---|---|
| Gemma 4 26B | OpenRouter (free) | Free |
| Nemotron 3 Super | OpenRouter (free) | Free |
| Nemotron 3 Nano 30B | OpenRouter (free) | Free |
| GPT-OSS 20B | OpenRouter (free) | Free |
| GPT-OSS 20B (Groq) | Groq | Free |
Key Policy
- Paid DeepSeek models → Direct API only (not OpenRouter)
- Free models → OpenRouter is fine (e.g., nvidia/nemotron:free)
- GPT-OSS models → Groq outperforms OpenRouter (86.9 vs lower)
The Smart Task Router
A keyword-based router (~/.hermes/scripts/task-router.py) maps goals to optimal models using leaderboard data:
Code/build tasks → deepseek/deepseek-v4-flash
Research/analysis → z-ai/glm-5.2
Creative/design → xiaomi/mimo-v2.5-pro
Quick/simple → xiaomi/mimo-v2.5
Default → deepseek/deepseek-v4-flash
Related
- Arena Workflow — Uses leaderboard data for model selection
- Tool Building Workflow — ToolBench benchmark references leaderboard scores
- Content Pipeline — Content quality metrics feed into leaderboard analysis
- CodeIntel — Primary leaderboard host
- ToolBrain — Arena benchmark data source
- Design Agent — Design-specific arena leaderboard