Building Conversational AI Models with LLMs
In this tutorial we will build a conversational customer support agent that looks up order status and answers policy questions. I chose Oxlo.ai because its flat per-request pricing keeps costs predictable even when conve
In this tutorial we will build a conversational customer support agent that looks up order status and answers policy questions. I chose Oxlo.ai because its flat per-request pricing keeps costs predictable even when conversation history grows. If you run tier-1 support or an internal helpdesk, this gives you a working baseline you can ship today.
What you'll need
I assume you have Python 3.10 or newer installed. You will also need the OpenAI SDK and an Oxlo.ai API key.
- Python 3.10+
pip install openai- An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Set up the Oxlo.ai client
First, I initialize the OpenAI-compatible client pointing at Oxlo.ai. The only change from a standard OpenAI setup is the base URL.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "user", "content": "Say hello and confirm you are online."},
],
)
print(response.choices[0].message.content)
Step 2: Define the agent's system prompt
Next, I define the system prompt. This is the agent's job description: tone, boundaries, and tool instructions all live here.
SYSTEM_PROMPT = """You are a customer support agent for Acme Gadgets.
Your job is to help users with order status and return policy questions.
You have access to a lookup_order tool that requires a 6-digit order ID.
If the user does not provide an order ID, ask for it.
If the user asks about topics outside orders or returns, refuse politely and offer to connect them with a human agent.
Keep responses concise and friendly."""
Step 3: Add a tool for order lookups
An agent that only chats is not enough. I define a mock order database and register the lookup schema so Oxlo.ai can request it through function calling.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a customer support agent for Acme Gadgets.
Your job is to help users with order status and return policy questions.
You have access to a lookup_order tool that requires a 6-digit order ID.
If the user does not provide an order ID, ask for it.
If the user asks about topics outside orders or returns, refuse politely and offer to connect them with a human agent.
Keep responses concise and friendly."""
def lookup_order(order_id: str):
orders = {
"123456": {"status": "shipped", "eta": "2026-01-15"},
"999999": {"status": "processing", "eta": "2026-01-20"},
}
return orders.get(order_id, {"status": "not_found", "eta": None})
tools = [
{
"type": "function",
"function": {
"name": "lookup_order",
"description": "Look up the status and estimated delivery date for an order.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "The 6-digit order ID.",
}
},
"required": ["order_id"],
},
},
}
]
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": "Where is my order 123456?"},
],
tools=tools,
tool_choice="auto",
)
print(response.choices[0].message)
Step 4: Handle tool calls and maintain conversation memory
When the model requests a tool call, we must execute the function locally and feed the result back into the conversation history. This loop is what turns an LLM call into an agent.
from openai import OpenAI
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a customer support agent for Acme Gadgets.
Your job is to help users with order status and return policy questions.
You have access to a lookup_order tool that requires a 6-digit order ID.
If the user does not provide an order ID, ask for it.
If the user asks about topics outside orders or returns, refuse politely and offer to connect them with a human agent.
Keep responses concise and friendly."""
def lookup_order(order_id: str):
orders = {
"123456": {"status": "shipped", "eta": "2026-01-15"},
"999999": {"status": "processing", "eta": "2026-01-20"},
}
return orders.get(order_id, {"status": "not_found", "eta": None})
tools = [
{
"type": "function",
"function": {
"name": "lookup_order",
"description": "Look up the status and estimated delivery date for an order.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "The 6-digit order ID.",
}
},
"required": ["order_id"],
},
},
}
]
def run_agent(user_message: str) -> str:
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
]
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
tool_choice="auto",
)
assistant_message = response.choices[0].message
messages.append(assistant_message)
if assistant_message.tool_calls:
for tool_call in assistant_message.tool_calls:
if tool_call.function.name == "lookup_order":
args = json.loads(tool_call.function.arguments)
result = lookup_order(args["order_id"])
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"name": tool_call.function.name,
"content": json.dumps(result),
})
final_response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
)
return final_response.choices[0].message.content
return assistant_message.content
print(run_agent("Where is my order 123456?"))
Step 5: Add guardrails and a handoff rule
Before shipping, I add guardrails. The updated system prompt tells the agent to refuse medical or billing questions and escalate to a human.
from openai import OpenAI
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a customer support agent for Acme Gadgets.
Your job is to help users with order status and return policy questions.
You have access to a lookup_order tool that requires a 6-digit order ID.
If the user does not provide an order ID, ask for it.
If the user asks about billing disputes, medical advice, or any other off-topic subject, refuse politely and offer to connect them with a human agent.
Keep responses concise and friendly."""
def lookup_order(order_id: str):
orders = {
"123456": {"status": "shipped", "eta": "2026-01-15"},
"999999": {"status": "processing", "eta": "2026-01-20"},
}
return orders.get(order_id, {"status": "not_found", "eta": None})
tools = [
{
"type": "function",
"function": {
"name": "lookup_order",
"description": "Look up the status and estimated delivery date for an order.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "The 6-digit order ID.",
}
},
"required": ["order_id"],
},
},
}
]
def run_agent(user_message: str) -> str:
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
]
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
tool_choice="auto",
)
assistant_message = response.choices[0].message
messages.append(assistant_message)
if assistant_message.tool_calls:
for tool_call in assistant_message.tool_calls:
if tool_call.function.name == "lookup_order":
args = json.loads(tool_call.function.arguments)
result = lookup_order(args["order_id"])
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"name": tool_call.function.name,
"content": json.dumps(result),
})
final_response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
)
return final_response.choices[0].message.content
return assistant_message.content
# Test guardrails
print(run_agent("I need help with a medical issue."))
Run it
Here is the complete script. I run two tests: a valid order lookup and an off-topic question that should trigger the handoff rule.
from openai import OpenAI
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a customer support agent for Acme Gadgets.
Your job is to help users with order status and return policy questions.
You have access to a lookup_order tool that requires a 6-digit order ID.
If the user does not provide an order ID, ask for it.
If the user asks about billing disputes, medical advice, or any other off-topic subject, refuse politely and offer to connect them with a human agent.
Keep responses concise and friendly."""
def lookup_order(order_id: str):
orders = {
"123456": {"status": "shipped", "eta": "2026-01-15"},
"999999": {"status": "processing", "eta": "2026-01-20"},
}
return orders.get(order_id, {"status": "not_found", "eta": None})
tools = [
{
"type": "function",
"function": {
"name": "lookup_order",
"description": "Look up the status and estimated delivery date for an order.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "The 6-digit order ID.",
}
},
"required": ["order_id"],
},
},
}
]
def run_agent(user_message: str) -> str:
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
]
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
tool_choice="auto",
)
assistant_message = response.choices[0].message
messages.append(assistant_message)
if assistant_message.tool_calls:
for tool_call in assistant_message.tool_calls:
if tool_call.function.name == "lookup_order":
args = json.loads(tool_call.function.arguments)
result = lookup_order(args["order_id"])
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"name": tool_call.function.name,
"content": json.dumps(result),
})
final_response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages,
tools=tools,
)
return final_response.choices[0].message.content
return assistant_message.content
if __name__ == "__main__":
print("User: Where is my order 123456?")
print("Agent:", run_agent("Where is my order 123456?"))
print()
print("User: Can you diagnose my rash?")
print("Agent:", run_agent("Can you diagnose my rash?"))
Example output:
User: Where is my order 123456? Agent: Your order 123456 has shipped and is expected to arrive on January 15, 2026. User: Can you diagnose my rash? Agent: I cannot provide medical advice. I can connect you with a human agent who will be able to help. Would you like me to do that?
Wrap-up
That is a working conversational agent on Oxlo.ai. Because the platform uses flat per-request pricing, long transcripts and detailed prompts do not inflate costs the way token-based billing does. Two next steps I recommend: expose this via FastAPI to turn it into a web endpoint, or try swapping in Qwen 3 32B for stronger multilingual agent workflows. Review request limits at https://oxlo.ai/pricing.
📰 Read the original article on Dev.to AI
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.