GitHub - Andyyyy64/whichllm: Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly. · GitHub

Author: Andyyyy64 | Published: Unknown | Generated: 2026-05-15 | Domain: github.com
Tags: ‘#localLLM’ ‘#LLMbenchmarks’ ‘#hardware’ ‘#PythonCLI’ ‘#HuggingFace’ ‘#GPU’

---

TLDR

whichllm recommends local models based on whether they can run on a machine and how well they are likely to perform, combining live Hugging Face discovery with benchmark evidence, lineage-aware recency adjustments, and hardware-aware memory/speed estimates. It avoids the common “largest model that fits” heuristic by discounting weak or inherited evaluations, distinguishing full-GPU, partial-offload, and CPU-only execution, and accounting for quantization, KV cache, framework overhead, and MoE active parameters. The one-command CLI can simulate GPUs, filter for practical speeds, plan hardware for a target model, launch chats, emit Python snippets, and return JSON for automation.

Key Takeaways

• Evidence-first ranking: Scores merge LiveBench, Artificial Analysis, Aider, multimodal/vision benchmarks, Chatbot Arena ELO, and Open LLM Leaderboard data rather than prioritizing parameter count. Evidence is labeled direct, variant, base_model, line_interp, or self_reported, with confidence discounts.
• Recency and provenance safeguards: Stale benchmark sources are capped and demoted by model lineage; inherited scores are rejected when a model differs by more than 2× from its family’s dominant parameter size, preventing small forks from borrowing a much larger base model’s results.
• Hardware-aware fit and speed modeling: VRAM estimates include weights, GQA KV cache, activations, and roughly 500 MB of framework overhead. Speed estimates incorporate memory bandwidth, quantization, backend, full vs. partial offload, unified vs. discrete memory, and MoE active rather than total parameters.
• Practical recommendation controls: Default rankings can include near-edge VRAM fits and partial RAM offloads, while --gpu-only --speed usable --vram-headroom 1GB yields more conservative full-VRAM recommendations. Usable speed is 10–30 tok/s; fast is 30+ tok/s.
• Example RTX 4090 ranking: For a simulated RTX 4090, Qwen/Qwen3.6-27B at Q5_K_M ranks first with a 92.8 score and estimated 27 tok/s, ahead of the larger Qwen3-32B because it has stronger benchmark quality and newer-generation evidence. A Qwen3-30B-A3B MoE example is estimated at 102 tok/s due to its active-parameter profile.
• Automation and execution: uvx whichllm@latest runs without setup; commands support GPU simulation, multi-GPU configurations, context/quantization filters, --json, plan, upgrade, run, and snippet. The run command supports GGUF through llama-cpp-python, AWQ/GPTQ through Transformers plus relevant loaders, and FP16/BF16 through Transformers.

Images & Media

• CLI recommendation demo — Animated terminal demonstration of the model-recommendation workflow.
• Instant run/chat demo — Animated demonstration of downloading and chatting with a selected local model.
• Star History Chart — Repository star-growth visualization.

Referenced Links

• whichllm GitHub repository — Source code, issues, releases, and project documentation.
• whichllm on PyPI — Python package distribution.
• CLI reference — Command options and CLI usage documentation.
• How it works — Data pipeline, benchmark resolution, and ranking-engine details.
• Scoring documentation — Score components, confidence penalties, and evidence rules.
• Hardware detection and simulation — Supported hardware detection and GPU simulation behavior.