Models for Different Tasks

Recommendations for choosing models based on your tasks and hardware.

🇷🇺 Russian version: models.ru.md

← Local models · Catalog →

Contents

1. Quick picks by task
2. By hardware
3. Key considerations
4. Coding models
5. Chat models
6. Reasoning models
7. Small models for 8 GB
8. What’s next

What affects model choice

When choosing a model, consider:

1. Task — coding, chat, text analysis, image processing
2. Available VRAM/RAM — how much memory you have
3. Quality needed — maximum quality or “good enough”
4. Speed requirements — real-time response or batch processing

1. Quick picks by task

2. By hardware

Your RAM	Best model to run	Approx. speed
8 GB	Phi-4-mini (3.8B), Qwen 3.5 4B	25-35 tok/s
16 GB	Qwen 2.5 Coder 7B, Qwen 3.5 9B	15-25 tok/s
24 GB	Qwen3.6-27B (Q4), Gemma 4 26B MoE	18-28 tok/s
32 GB	Llama 3.1 70B (Q3)	20-40 tok/s
48 GB	Llama 3.3 70B (Q4)	10-20 tok/s
64 GB	Gemma 4 31B (Q8)	8-15 tok/s
128 GB	DeepSeek-V4-Flash, Qwen3.5-397B	15-30 tok/s

3. Key considerations

• FIM (Fill-in-the-Middle) — needed for code autocomplete in IDEs. Qwen Coder has it.
• Tool calling — needed for agents. Qwen 3.5+, Llama 3.1+
• Long context — 128K+ for working with large codebases. Qwen 3.5 supports 256K
• Multimodal — vision models for image understanding (Llama 3.2 Vision, Gemma 4)

4. Coding models

Model	Size	Context	FIM	HumanEval	Best for
Qwen 2.5 Coder 7B	4.7 GB	128K		82%	General coding, multi-file
Qwen 2.5 Coder 1.5B	1.1 GB	32K		45%	Fast autocomplete, 8GB Mac
CodeGemma 2B	1.5 GB	8K		50%	Autocomplete, low RAM
Phi-4-mini (3.8B)	2.5 GB	128K		74%	Math + coding, small RAM
Yi-Coder 9B	6 GB	128K		70%	Long context coding

5. Chat models

6. Reasoning models

For reasoning and math: Phi-4 Mini (3.8B) — surprisingly strong for its size; Phi-4 (14B) — if you have resources, excellent at logic; Qwen 2.5 Math series — specifically tuned for mathematical tasks; DeepSeek Math 7B/7B-Instruct — strong math model.

Note: Reasoning models are slower (they “think” before answering) but give better results on complex tasks. Not suitable for real-time chat.

7. Small models for 8 GB

For Macs with 8 GB RAM or CPU-only inference:

MacBook Air M1 16GB reference

Models that work well on 16GB unified memory (M1/M2 chips):

Estimating RAM usage

Formula

Model memory ≈ (Parameters in billions) × (Bytes per parameter) + KV cache + overhead

Example calculations

RAM recommendations

Specialized models

Coding

For coding the best choices are: Qwen 2.5 Coder 7B — best balance of quality and speed; DeepSeek Coder V2 Lite (16B MoE, ~2.4B active) — if you have more resources; Stable Code 3B — very fast for autocomplete; CodeLlama 7B/13B — good alternative with large ecosystem.

Vision / Image understanding

• Gemma 4 (4B) — great balance of size and quality
• Llama 3.2 Vision (11B/90B) — maximum quality
• Pixtral (12B) — from Mistral, strong multimodal capabilities
• InternVL2 (2B/4B/8B) — excellent quality for size

Long context

• Qwen 3.5 series (8B/14B/32B) — up to 32K tokens
• Yi series (6B/9B/34B) — up to 200K tokens
• GLM-4 (9B) — up to 128K tokens
• Phi-3-mini-128k — special version with 128K context

How to read model names

Example: Qwen 2.5 Coder 7B Instruct Q4_K_M

• Qwen 2.5 — model family and version
• Coder — specialization (can be omitted for general models)
• 7B — parameter count (7 billion)
• Instruct — instruction-tuned version (vs base)
• Q4_K_M — quantization type (see quantization guide)

Main quantization types (GGUF)

Tip: Start with Q4_K_M for most tasks — best balance of quality, size and speed. Need more quality? Go Q5_K_M or Q6_K. Need speed or very low memory? Try Q3_K_M or Q2_K.

Hardware-specific recommendations

Apple Silicon (M1/M2/M3/M4)

• Best in GGUF format
• Via Ollama or LM Studio (auto-uses Metal)
• For max speed — mlx-lm (MLX format)
• Avoid large context windows (>16K) on M1/M2 due to limited memory bandwidth

NVIDIA GPU

• Best via llama.cpp with CUDA or inference frameworks
• GGUF and GPTQ formats work well
• Watch VRAM (not system RAM)
• 8GB VRAM: models up to 7B Q4/Q5
• 12GB VRAM: up to 13B Q4/Q5
• 16GB VRAM: up to 20-24B Q4
• 24GB+ VRAM: try 30-70B with appropriate quantization

AMD GPU

• Support via ROCm (limited)
• Use llama.cpp or frameworks with CPU fallback
• Similar memory recommendations as NVIDIA

CPU only

• Stick to models up to 3-4B for acceptable speed
• Use aggressive quantization (Q2_K/Q3_K)
• Expect 1-5 tok/s on modern CPUs
• Best for experiments and light tasks

Where to get models

Ollama library

Simplest way to start:

ollama pull qwen3:8b
ollama run qwen3:8b

Browse models at ollama.com/library

Hugging Face

For advanced users:

• Look for .gguf suffix in model names (ready for llama.cpp)
• Or download original models and convert via llama.cpp/convert_hf_to_gguf.py
• Popular orgs: Qwen, meta-llama, microsoft, google, NousResearch, TheBloke, lmstudio-community

Top HF collections

• TheBloke — almost all popular models in various GGUF quantizations
• lmstudio-community — optimized for LM Studio
• unsloth — optimized for fast fine-tuning

Stay updated

Models evolve fast. Recommended:

1. Follow releases from major labs (Meta, Mistral, Qwen, Microsoft, Google)
2. Check favorite HF repos every 2-4 weeks
3. Join r/LocalLLaMA and relevant Discord communities

8. What’s next

In section: getting-started · running-models · models · catalog · quantization · memory-and-context · tools · advanced-setup · troubleshooting · apple-silicon
Related sections: Zero Level · AI Agents · Use Cases
Navigation: ← Local Models · ↑ Back to main · 🇷🇺 Русский