Building a Secure Streaming AI Endpoint with Next.js App Router

If you're building an AI SaaS or LLM wrapper in 2026, time-to-first-token (TTFT) has a huge impact on how the application feels.

Waiting several seconds for the entire completion before showing anything makes the UI feel slow, even when the model itself is responding normally.

Streaming fixes that by sending tokens to the client as they're generated.

But there's another side to it: implementing streaming carelessly can introduce problems around authentication, API-key exposure, rate limiting, and resource consumption.

Here's a simple architecture for building a streaming AI endpoint with Next.js App Router, Supabase authentication, and server-side rate limiting.

1. The Streaming Route Handler

Let's start with /api/ai/stream.

The native Web Streams API works well for this use case because we can send chunks to the client as they become available instead of buffering the entire response.

import { NextRequest, NextResponse } from "next/server";
import { createClient } from "@/lib/supabase/server";

export const runtime = "edge";

export async function POST(req: NextRequest) {
  try {
    // 1. Authenticate using Supabase SSR cookies
    const supabase = await createClient();

    const {
      data: { user },
    } = await supabase.auth.getUser();

    if (!user) {
      return NextResponse.json(
        { error: "Unauthorized" },
        { status: 401 }
      );
    }

    const { prompt } = await req.json();

    // 2. Create the response stream
    const encoder = new TextEncoder();

    const stream = new ReadableStream({
      async start(controller) {
        const chunks = [
          "Analyzing architecture... ",
          "Generating edge pipeline... ",
          "Done.",
        ];

        for (const chunk of chunks) {
          controller.enqueue(
            encoder.encode(
              `data: ${JSON.stringify({ text: chunk })}\n\n`
            )
          );

          await new Promise((resolve) => setTimeout(resolve, 60));
        }

        controller.enqueue(
          encoder.encode("data: [DONE]\n\n")
        );

        controller.close();
      },
    });

    return new Response(stream, {
      headers: {
        "Content-Type": "text/event-stream; charset=utf-8",
        "Cache-Control": "no-cache, no-transform",
        Connection: "keep-alive",
      },
    });
  } catch (error) {
    console.error("Stream error:", error);

    return NextResponse.json(
      { error: "Stream error" },
      { status: 500 }
    );
  }
}

The example above uses simulated chunks, but the same stream can be connected to an actual LLM provider.

The important part is that the server starts sending data immediately rather than waiting for the complete response.

2. Keep Your API Keys on the Server

One mistake I see frequently in AI wrappers is putting provider credentials in client-side code.

Don't do this.

Your browser should communicate with your application:

Browser
   ↓
/api/ai/stream
   ↓
Authentication
   ↓
Rate limiting
   ↓
LLM provider

The OpenAI, Anthropic, or other provider API key should remain on the server.

The client only needs to receive the streamed output.

3. Authenticate Before Starting the LLM Request

Streaming makes authentication even more important because an unauthorized user can potentially keep connections open and consume resources.

The sequence should be:

Request
  ↓
Validate session
  ↓
Check rate limit
  ↓
Validate input
  ↓
Call LLM
  ↓
Stream response

Don't start the expensive provider request and then perform authorization afterward.

Reject invalid requests as early as possible.

4. Add Rate Limiting

Authentication alone doesn't stop an authenticated user from making thousands of requests.

For an AI SaaS, this can become an expensive problem very quickly.

A rate limiter should generally be applied before the LLM request is created.

Depending on your architecture, you might use Redis or another shared store so limits work across multiple application instances.

For example:

user_123
   │
   ├── request 1 ✓
   ├── request 2 ✓
   ├── request 3 ✓
   └── request 4 → rate limited

For production systems, consider limiting based on both request frequency and the amount of work being requested.

5. Validate the Input

Don't blindly pass arbitrary request bodies to your LLM provider.

At minimum, validate:

• Prompt type
• Maximum prompt length
• Required fields
• Allowed parameters
• Model selection
• Token limits

This prevents malformed requests from reaching the expensive part of your stack.

6. The Architecture

Putting everything together:

                 ┌──────────────────┐
                 │     Browser      │
                 └────────┬─────────┘
                          │
                          ▼
                 ┌──────────────────┐
                 │ /api/ai/stream   │
                 └────────┬─────────┘
                          │
                    Authentication
                          │
                          ▼
                    Rate Limiting
                          │
                          ▼
                    Input Validation
                          │
                          ▼
                 ┌──────────────────┐
                 │   LLM Provider   │
                 └────────┬─────────┘
                          │
                     Token stream
                          │
                          ▼
                 ┌──────────────────┐
                 │   ReadableStream │
                 └────────┬─────────┘
                          │
                          ▼
                       Browser

The key idea isn't simply "use streaming."

It's to make streaming part of a properly controlled request pipeline.

Authenticate → rate-limit → validate → call the provider → stream the result.

That gives you a much better foundation for turning an LLM API into an actual SaaS feature rather than exposing a raw model endpoint.

One More Thing

Don't assume that runtime = "edge" automatically means your endpoint will have sub-100ms TTFT.

The runtime can reduce some latency, but actual TTFT still depends on your authentication path, region, network, provider, model, prompt size, and infrastructure.

Measure it. Don't market the number before you've measured it.