Switch an OpenAI SDK app to another model endpoint with minimal code changes
When an application already uses the OpenAI SDK, testing another model endpoint should not require rewriting the request layer. The useful comparison is small and reproducible: keep the prompt and client code stable, cha
When an application already uses the OpenAI SDK, testing another model endpoint should not require rewriting the request layer. The useful comparison is small and reproducible: keep the prompt and client code stable, change the endpoint and key, select a model from the live catalog, and record what changed.
This walkthrough uses Micropines as an example of an independent third-party gateway with an OpenAI-compatible API. It is not an official endpoint from the model providers. Model availability and rates can change, so check the live model directory and rate card before each test.
What changes
The OpenAI Python client accepts a base_url. The request shape stays the same, and the endpoint can live in configuration:
OpenAI(api_key=..., base_url=os.getenv("MICROPINES_BASE_URL", "https://micropines.net/v1"))
You still need an account, an API key, and a model ID currently shown in the catalog. Keep the key in an environment variable rather than in source code.
A runnable Python example
Create a virtual environment and install the SDK:
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade openai
Set the credentials and model ID:
export MICROPINES_API_KEY="replace-with-your-key"
export MICROPINES_MODEL="replace-with-a-model-id-from-the-live-catalog"
# Optional: set this to your original provider endpoint to switch back.
export MICROPINES_BASE_URL="https://micropines.net/v1"
Save this as main.py:
import os
from openai import OpenAI
api_key = os.environ["MICROPINES_API_KEY"]
model = os.environ["MICROPINES_MODEL"]
client = OpenAI(
api_key=api_key,
base_url=os.getenv("MICROPINES_BASE_URL", "https://micropines.net/v1"),
)
response = client.chat.completions.create(
model=model,
messages=[
{"role": "user", "content": "Give me three concise names for a plant-care app."}
],
temperature=0.2,
)
print(response.choices[0].message.content)
if response.usage:
print("usage:", response.usage)
Run it with:
python main.py
The request format is the familiar OpenAI chat-completions format. The important integration change is the base URL and the model ID selected from the current catalog.
The same switch in Node.js
Install the SDK:
npm install openai
Then use main.mjs:
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.MICROPINES_API_KEY,
baseURL: process.env.MICROPINES_BASE_URL ?? "https://micropines.net/v1",
});
const response = await client.chat.completions.create({
model: process.env.MICROPINES_MODEL,
messages: [
{ role: "user", content: "Give me three concise names for a plant-care app." },
],
temperature: 0.2,
});
console.log(response.choices[0]?.message?.content);
console.log("usage:", response.usage);
Run it with:
MICROPINES_API_KEY="replace-with-your-key" \
MICROPINES_MODEL="replace-with-a-model-id-from-the-live-catalog" \
MICROPINES_BASE_URL="https://micropines.net/v1" \
node main.mjs
Set MICROPINES_BASE_URL to the original provider's endpoint when you want to switch back. Keep the API key and model ID appropriate for that endpoint.
Make the comparison reproducible
A single successful request does not establish a price or latency advantage. For a useful comparison, keep these values fixed:
- exact model ID and request body;
- prompt and generation settings;
- timestamp and region;
- input and output token counts;
- latency measurement method;
- rate-card URL or snapshot used for the calculation;
- HTTP status and any retry behavior.
Run the same prompt against each endpoint, then record the results in a small CSV or notebook. Keep the script, exact prompt, and benchmark notes alongside the results so another developer can reproduce the comparison.
Keep a fallback path
Treat a third-party gateway as a configurable dependency. Keep the original endpoint available in configuration and make the base URL selectable through an environment variable. If a model is unavailable, the key is rejected, or the response shape differs, switch back to the original endpoint and inspect the gateway's current documentation before retrying.
For production traffic, add timeouts, bounded retries, error logging that excludes secrets, and application-level safeguards. Review the service's current Quickstart and user agreement for data handling, billing, balance, rate limits, and refunds before sending real workloads.
What I would measure next
The next useful test is not another announcement. It is a dated table showing the same request across two endpoints, with the exact model, token counts, latency, and current rate-card references. That makes it possible for another developer to reproduce the result and decide whether the integration fits their own workload.
If you try the example, feedback on SDK compatibility, error messages, model naming, and the clarity of the rate card is more useful than a generic βworksβ report.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.