An AI Server for Your Company: What to Buy for 7B–70B Models and What It Costs vs the Cloud
"Should we buy an AI server?" — this question hides three different ones: how much GPU memory you actually need, rent or buy, and where your data lives. Here are the numbers I collected for November 2026 (Russian market
"Should we buy an AI server?" — this question hides three different ones: how much GPU memory you actually need, rent or buy, and where your data lives. Here are the numbers I collected for November 2026 (Russian market prices, verified sources), plus a worked example: a RAG document assistant for 10 employees.
How much memory: 7B, 32B, 70B
You pay for memory, not parameters. At full precision each parameter costs 2 bytes — a 70B model takes ~140 GB. In practice everyone runs 4-bit quantization (Q4), which cuts that roughly 4×:
| Model | Q4 memory | + context | Good for |
|---|---|---|---|
| 7B–8B | 5–6 GB | +1–4 GB | Drafts, summarization, ticket triage |
| 32B | 18–20 GB | +2–8 GB | Document Q&A, reasoning — "not a toy anymore" |
| 70B | ~40 GB | +5–15 GB | Deep analysis, long documents, near-flagship quality |
First rule of buying a server: look at VRAM, not at the marketing tier. One RTX 4090 (24 GB) fits a 32B Q4 model with context room to spare. For 70B you need a 96 GB card (RTX PRO 6000) or multiple cards.
Renting GPUs in Russia (November 2026)
Public hourly rates from Russian providers (immers.cloud, Selectel, HOSTKEY, Teraferma, aggregated by GPURadar):
| GPU | VRAM | From, ₽/hr | 24/7 per month |
|---|---|---|---|
| RTX 3090 | 24 GB | 29 | ~21 000 ₽ |
| RTX 4090 | 24 GB | 55 | 40–60 000 ₽ |
| RTX 5090 | 32 GB | 72 | ~52 000 ₽ |
| RTX PRO 6000 | 96 GB | 195 | ~140 000 ₽ |
| A100 | 80 GB | 212 | ~152 000 ₽ |
| H100 | 80 GB | 291 | ~210 000 ₽ |
| H200 | 141 GB | 423 | ~305 000 ₽ |
Hourly billing × 720 hours destroys the "leave it running" economics. For always-on workloads, dedicated monthly servers win: a dedicated RTX 4090 server starts at 37 000 ₽/month (HOSTKEY) versus ~55 000 ₽/month for the same card as cloud instance (Selectel).
Building your own rack
GPU prices are ugly right now: RTX 4090 is discontinued and retails at $4 899 new (Newegg, Nov 17, 2026; ~$3 400 used on eBay) against the original $1 599. RTX 5090 lists at $1 999 MSRP but sells from $6 550. A server-class RTX PRO 6000 Blackwell 96 GB starts at ₽2 586 700 in Russian retail (VAT included).
A realistic "32B in production" build with a new 4090: ~500–650 thousand ₽ all-in. Running costs: ~450–550 W under load → 330–400 kWh/month → ₽2 000–2 800 of electricity. Depreciation over 3 years adds ₽14–18K/month. Total: ₽17–21K per month for your own 32B server, data never leaves the office.
The worked example: document assistant for 10 people
Typical RAG setup: 10 employees × 20 questions/day, ~8K tokens of retrieved context per query, ~500-token answers → ~35M input + 2.2M output tokens per month:
| Path | Per month | Data location |
|---|---|---|
| DeepSeek V4.1 Flash (off-peak) | ~600 ₽ | leaves the country |
| GigaChat Lite | ~2 400 ₽ | Russian cloud |
| YandexGPT Pro 5.1 (async) | ~15 000 ₽ | Russian cloud |
| RTX 4090 rental (10h × 22 days) | ~18 000 ₽ | provider DC |
| Own rack with 4090 | 17–21 000 ₽ | your office |
Three takeaways: the cheapest foreign API is pennies but usually fails the security review; Russian cloud APIs on this scale already cost more than owning hardware — and scale linearly with headcount; a stable workload above ~6 hours/day makes owned or dedicated hardware cheaper than API at a single-department scale.
Rules of thumb
- Start with an API pilot on a Russian model — ₽2–3K gives you real load statistics to size the GPU.
- Don't buy memory "for 70B" without proof — verify 32B isn't enough on your tasks first; the difference is ~2M ₽.
- Hybrid beats extremes: local 32B for routine, route hard questions to a cloud model.
- Data residency decides early: prompts with contracts and personal data sent to a foreign API is cross-border transfer under Russian law (152-FZ) — local models close this question physically.
Software stack
vLLM for production (dozens of concurrent sessions per card via batching) or Ollama for simple cases, exposed as an OpenAI-compatible API so internal apps don't care whether the model lives in your closet or in OpenAI's datacenter.
Originally published at klukyanov.ru — I write about local LLMs and AI implementation in Russian companies.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.