Dev.to AI 🤖 Ai 👁 0 📖 3 min read

An AI Server for Your Company: What to Buy for 7B–70B Models and What It Costs vs the Cloud

"Should we buy an AI server?" — this question hides three different ones: how much GPU memory you actually need, rent or buy, and where your data lives. Here are the numbers I collected for November 2026 (Russian market

"Should we buy an AI server?" — this question hides three different ones: how much GPU memory you actually need, rent or buy, and where your data lives. Here are the numbers I collected for November 2026 (Russian market prices, verified sources), plus a worked example: a RAG document assistant for 10 employees.

How much memory: 7B, 32B, 70B

You pay for memory, not parameters. At full precision each parameter costs 2 bytes — a 70B model takes ~140 GB. In practice everyone runs 4-bit quantization (Q4), which cuts that roughly 4×:

Model Q4 memory + context Good for
7B–8B 5–6 GB +1–4 GB Drafts, summarization, ticket triage
32B 18–20 GB +2–8 GB Document Q&A, reasoning — "not a toy anymore"
70B ~40 GB +5–15 GB Deep analysis, long documents, near-flagship quality

First rule of buying a server: look at VRAM, not at the marketing tier. One RTX 4090 (24 GB) fits a 32B Q4 model with context room to spare. For 70B you need a 96 GB card (RTX PRO 6000) or multiple cards.

Renting GPUs in Russia (November 2026)

Public hourly rates from Russian providers (immers.cloud, Selectel, HOSTKEY, Teraferma, aggregated by GPURadar):

GPU VRAM From, ₽/hr 24/7 per month
RTX 3090 24 GB 29 ~21 000 ₽
RTX 4090 24 GB 55 40–60 000 ₽
RTX 5090 32 GB 72 ~52 000 ₽
RTX PRO 6000 96 GB 195 ~140 000 ₽
A100 80 GB 212 ~152 000 ₽
H100 80 GB 291 ~210 000 ₽
H200 141 GB 423 ~305 000 ₽

Hourly billing × 720 hours destroys the "leave it running" economics. For always-on workloads, dedicated monthly servers win: a dedicated RTX 4090 server starts at 37 000 ₽/month (HOSTKEY) versus ~55 000 ₽/month for the same card as cloud instance (Selectel).

Building your own rack

GPU prices are ugly right now: RTX 4090 is discontinued and retails at $4 899 new (Newegg, Nov 17, 2026; ~$3 400 used on eBay) against the original $1 599. RTX 5090 lists at $1 999 MSRP but sells from $6 550. A server-class RTX PRO 6000 Blackwell 96 GB starts at ₽2 586 700 in Russian retail (VAT included).

A realistic "32B in production" build with a new 4090: ~500–650 thousand ₽ all-in. Running costs: ~450–550 W under load → 330–400 kWh/month → ₽2 000–2 800 of electricity. Depreciation over 3 years adds ₽14–18K/month. Total: ₽17–21K per month for your own 32B server, data never leaves the office.

The worked example: document assistant for 10 people

Typical RAG setup: 10 employees × 20 questions/day, ~8K tokens of retrieved context per query, ~500-token answers → ~35M input + 2.2M output tokens per month:

Path Per month Data location
DeepSeek V4.1 Flash (off-peak) ~600 ₽ leaves the country
GigaChat Lite ~2 400 ₽ Russian cloud
YandexGPT Pro 5.1 (async) ~15 000 ₽ Russian cloud
RTX 4090 rental (10h × 22 days) ~18 000 ₽ provider DC
Own rack with 4090 17–21 000 ₽ your office

Three takeaways: the cheapest foreign API is pennies but usually fails the security review; Russian cloud APIs on this scale already cost more than owning hardware — and scale linearly with headcount; a stable workload above ~6 hours/day makes owned or dedicated hardware cheaper than API at a single-department scale.

Rules of thumb

  1. Start with an API pilot on a Russian model — ₽2–3K gives you real load statistics to size the GPU.
  2. Don't buy memory "for 70B" without proof — verify 32B isn't enough on your tasks first; the difference is ~2M ₽.
  3. Hybrid beats extremes: local 32B for routine, route hard questions to a cloud model.
  4. Data residency decides early: prompts with contracts and personal data sent to a foreign API is cross-border transfer under Russian law (152-FZ) — local models close this question physically.

Software stack

vLLM for production (dozens of concurrent sessions per card via batching) or Ollama for simple cases, exposed as an OpenAI-compatible API so internal apps don't care whether the model lives in your closet or in OpenAI's datacenter.

Originally published at klukyanov.ru — I write about local LLMs and AI implementation in Russian companies.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.