FastAPI SSE Streaming for Real-Time Network Dashboards: Lessons from netpulse Integration
The Problem: Real-Time Router Fleet Monitoring Without WebSocket Overhead When integrating gnacho/netpulse into our multi-tenant infrastructure monitoring stack, we needed a lightweight mechanism to stream router healt
The Problem: Real-Time Router Fleet Monitoring Without WebSocket Overhead
When integrating gnacho/netpulse into our multi-tenant infrastructure monitoring stack, we needed a lightweight mechanism to stream router health metrics (CPU, memory, WireGuard peer status, AdGuard Home block counts) to browser dashboards without the complexity of WebSocket handshakes or bidirectional chat protocols.
Server-Sent Events (SSE) emerged as the natural fit: unidirectional push over plain HTTP, browser-native EventSource API, automatic reconnection logic. But production deployment revealed three critical failure modes that FastAPI's documentation glosses over.
Architecture Context
The netpulse backend exposes a /events endpoint that aggregates telemetry from multiple OpenWrt routers via their ubus JSON-RPC interface. Each client dashboard maintains a persistent SSE connection, receiving JSON payloads every 5 seconds:
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio
import json
app = FastAPI()
async def event_generator(router_ids: list[str]):
"""Async generator yielding SSE-formatted payloads."""
while True:
# Fetch telemetry from SQLite or live ubus poll
metrics = await fetch_router_metrics(router_ids)
payload = json.dumps({
"routers": metrics,
"timestamp": int(time.time())
})
yield f"data: {payload}\n\n"
await asyncio.sleep(5)
@app.get("/events")
async def stream_events(router_ids: str):
ids = router_ids.split(",")
return StreamingResponse(
event_generator(ids),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"X-Accel-Buffering": "no" # Disable nginx buffering
}
)
Failure Mode 1: Unclosed Database Connections in Infinite Loops
Our first production incident occurred 6 hours after deployment: the connection pool exhausted with 200+ idle SQLite connections. The root cause was subtle:
# BROKEN: Connection opened per iteration
async def event_generator(router_ids: list[str]):
while True:
async with get_db_session() as db:
metrics = await db.execute(select(Router).where(...))
yield f"data: {json.dumps(metrics)}\n\n"
await asyncio.sleep(5)
Each async with block acquired a new connection but the generator's infinite loop prevented garbage collection. Fix: hoist the session acquisition outside the loop or use a singleton connection with explicit health checks:
async def event_generator(router_ids: list[str]):
async with get_db_session() as db:
while True:
metrics = await db.execute(select(Router).where(...))
yield f"data: {json.dumps(metrics)}\n\n"
await asyncio.sleep(5)
Failure Mode 2: Client Disconnections Leave Zombie Generators
When a browser tab closes or the client network drops, FastAPI does not automatically stop the async generator. Our monitoring showed 50+ orphaned generators still polling the database after clients disconnected.
The solution requires explicit cancellation checks using asyncio.CancelledError:
async def event_generator(router_ids: list[str]):
try:
async with get_db_session() as db:
while True:
metrics = await db.execute(select(Router).where(...))
yield f"data: {json.dumps(metrics)}\n\n"
await asyncio.sleep(5)
except asyncio.CancelledError:
# Client disconnected - cleanup resources
await db.close()
raise
Crucially, FastAPI's StreamingResponse does raise CancelledError when the client disconnects, but only if you're awaiting something. A synchronous time.sleep(5) would block indefinitely.
Failure Mode 3: Synchronous Blocking Calls in Async Generators
During load testing with 100 concurrent SSE clients, we observed 2-second stalls where all connections froze. Profiling revealed a synchronous requests.get() call inside the generator:
# BROKEN: Blocks the event loop
async def event_generator(router_ids: list[str]):
while True:
response = requests.get(f"http://{router_ip}/ubus") # BLOCKING
yield f"data: {response.text}\n\n"
await asyncio.sleep(5)
The fix requires using httpx.AsyncClient or wrapping the blocking call in asyncio.to_thread():
import httpx
async def event_generator(router_ids: list[str]):
async with httpx.AsyncClient() as client:
while True:
response = await client.get(f"http://{router_ip}/ubus")
yield f"data: {response.text}\n\n"
await asyncio.sleep(5)
Production Hardening: Pydantic Validators for SSE Payloads
To prevent malformed JSON from breaking the event stream, we wrapped all outbound data in Pydantic v2 models:
from pydantic import BaseModel, Field, field_validator
from datetime import datetime
class RouterMetrics(BaseModel):
router_id: str
cpu_percent: float = Field(ge=0.0, le=100.0)
memory_mb: int = Field(ge=0)
wireguard_peers: int = Field(ge=0)
timestamp: datetime
@field_validator('cpu_percent')
@classmethod
def clamp_cpu(cls, v: float) -> float:
return min(max(v, 0.0), 100.0)
model_config = {
"json_encoders": {
datetime: lambda dt: int(dt.timestamp())
}
}
This caught a bug where negative CPU percentages from buggy router firmware would cause JSON serialization to fail mid-stream.
Benchmark: SSE vs. Polling in Multi-Tenant Scenarios
When comparing netpulse's SSE implementation against a naive 5-second polling REST API, we measured a 70% reduction in database query load (since the generator maintains a single long-lived connection per client rather than repeated short-lived requests). Network bandwidth dropped by 40% due to eliminating HTTP request headers on every poll.
For latency-sensitive telemetry where sub-second updates matter, SSE's persistent connection eliminated the cold-start overhead of TLS handshakes and TCP slow-start. In our tests routing through a multi-provider relay gateway (similar to production topologies using baseURL: "https://b-lost.com/v1" for model inference proxying), time-to-first-byte remained under 50ms even with 200 concurrent SSE streams.
Open Questions for High-Scale Deployments
One unresolved trade-off: FastAPI runs SSE generators in the same event loop as regular HTTP handlers. Under extreme load (500+ concurrent SSE clients), does the constant asyncio.sleep() wakeup churn degrade latency for non-streaming endpoints? Our profiling showed a 15ms p99 increase in regular REST API response times, suggesting a need for dedicated worker processes or load balancer affinity rules.
Another edge case: how do you handle schema evolution when clients maintain 8-hour SSE connections? Netpulse currently forces a client reload on backend restart, but a production system might need versioned event types or graceful degradation when new fields appear mid-stream.
What strategies have you used to manage long-lived async generators in production? Have you hit connection pool limits or event loop contention with high SSE concurrency?
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.