How many developers can one GPU serve?
The right question is not “which GPU?” It is “how many developers, using which model, with what context length, at the same time?”
Planning ranges are only a starting point:
- 1 Mac mini: 1–3 developers
- strong workstation / small GPU: 3–8
- 1× L40S / RTX 6000 Ada class: 5–15
- 1× H100: 15–40
- 4× H100/H200: 50–150
What changes the answer
- model size and active parameters
- quantization
- context length and KV cache pressure
- agent/tool-call loops versus simple chat
- accepted latency
- peak concurrency, not headcount
The honest way
Run a one-week load test with the company’s real prompts. Measure tokens/sec, concurrent sessions, p95 latency, eval pass rate, and cost per accepted suggestion. Then buy hardware.
Do not buy H100s from a spreadsheet. Measure the real workload first.
No Monkey Business installs and operates sovereign/local AI infrastructure. If this problem sounds familiar, book a 30-minute assessment.