Unlocking Dialogue Generation with LLM: Tips and Best Practices

I recently shipped a context-aware support agent for a SaaS product that needed to handle messy, multi-turn customer threads without sounding robotic. In this tutorial, I will walk you through the exact version I built, using Oxlo.ai's OpenAI-compatible API and request-based pricing so long conversations do not inflate your bill. You can follow along with any of Oxlo.ai's chat models, though I used Llama 3.3 70B for the initial version.

What you'll need

• Python 3.10 or newer.
• The OpenAI SDK: pip install openai.
• An Oxlo.ai API key from https://portal.oxlo.ai.

Step 1: Test the connection with a single turn

Before building state management, I verify that the client reaches Oxlo.ai and that the model responds in a reasonable voice. I use llama-3.3-70b because it is a reliable general-purpose model for support dialogue.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "user", "content": "My dashboard is blank after logging in. What should I do?"},
    ],
)

print(response.choices[0].message.content)

Step 2: Define the system prompt

The system prompt is the only place where you teach tone, boundaries, and escalation rules. I keep it in a dedicated variable so I can iterate without touching business logic.

SYSTEM_PROMPT = """You are a Tier-1 support agent for a SaaS analytics platform. Your name is Alex.

Tone and style:
- Be concise. Use no more than three sentences per reply unless the user asks for detail.
- Be friendly but professional. Do not use marketing language.
- Always confirm the user's issue before proposing a fix.

Boundaries:
- You can help with account access, dashboard bugs, and billing questions.
- If the user reports a security breach, data loss, or expresses anger with profanity, you must escalate.
- If you do not know the answer, ask one clarifying question. Do not guess.

Escalation:
- If the user needs human support, say exactly: "I am connecting you with a specialist now."
"""

Step 3: Manage multi-turn history

Dialogue falls apart if you drop context. I use a simple list that appends each turn and trims when it grows too large, which is affordable on Oxlo.ai because request-based pricing means deep history does not raise the per-turn cost.

class SupportAgent:
    def __init__(self, client, model="llama-3.3-70b", max_turns=10):
        self.client = client
        self.model = model
        self.max_turns = max_turns
        self.history = [{"role": "system", "content": SYSTEM_PROMPT}]

    def _trim_history(self):
        # Always preserve the first system message, keep last N user/assistant pairs
        if len(self.history) > (self.max_turns * 2) + 1:
            self.history = [self.history[0]] + self.history[-(self.max_turns * 2):]

    def chat(self, user_message: str) -> str:
        self.history.append({"role": "user", "content": user_message})
        self._trim_history()

        response = self.client.chat.completions.create(
            model=self.model,
            messages=self.history,
        )

        reply = response.choices[0].message.content
        self.history.append({"role": "assistant", "content": reply})
        return reply

Step 4: Add escalation guardrails

I add a lightweight keyword guardrail that injects a reminder into the context when the user shows frustration. This avoids building a separate classifier and keeps the stack simple.

FRUSTRATION_SIGNALS = ["angry", "frustrated", "useless", "terrible", "refund", "lawsuit", "cancel everything"]

class SupportAgent:
    def __init__(self, client, model="llama-3.3-70b", max_turns=10):
        self.client = client
        self.model = model
        self.max_turns = max_turns
        self.history = [{"role": "system", "content": SYSTEM_PROMPT}]

    def _trim_history(self):
        if len(self.history) > (self.max_turns * 2) + 1:
            self.history = [self.history[0]] + self.history[-(self.max_turns * 2):]

    def _is_frustrated(self, text: str) -> bool:
        return any(signal in text.lower() for signal in FRUSTRATION_SIGNALS)

    def chat(self, user_message: str) -> str:
        self.history.append({"role": "user", "content": user_message})

        if self._is_frustrated(user_message):
            self.history.append({
                "role": "system",
                "content": "REMINDER: The user is frustrated. Follow escalation rules."
            })

        self._trim_history()

        response = self.client.chat.completions.create(
            model=self.model,
            messages=self.history,
        )

        reply = response.choices[0].message.content
        self.history.append({"role": "assistant", "content": reply})
        return reply

Step 5: Return a structured hand-off summary

When the agent escalates, I want a JSON object instead of free text so my backend can open a ticket automatically. Oxlo.ai supports JSON mode, so I force the model to emit a structured payload without extra parsing logic.

import json

class SupportAgent:
    def __init__(self, client, model="llama-3.3-70b", max_turns=10):
        self.client = client
        self.model = model
        self.max_turns = max_turns
        self.history = [{"role": "system", "content": SYSTEM_PROMPT}]

    def _trim_history(self):
        if len(self.history) > (self.max_turns * 2) + 1:
            self.history = [self.history[0]] + self.history[-(self.max_turns * 2):]

    def _is_frustrated(self, text: str) -> bool:
        return any(signal in text.lower() for signal in FRUSTRATION_SIGNALS)

    def chat(self, user_message: str) -> str:
        self.history.append({"role": "user", "content": user_message})

        if self._is_frustrated(user_message):
            self.history.append({
                "role": "system",
                "content": "REMINDER: The user is frustrated. Follow escalation rules."
            })

        self._trim_history()

        response = self.client.chat.completions.create(
            model=self.model,
            messages=self.history,
        )

        reply = response.choices[0].message.content
        self.history.append({"role": "assistant", "content": reply})
        return reply

    def handoff_summary(self) -> dict:
        summary_prompt = (
            "Based on the conversation so far, produce a JSON object with exactly these keys: "
            "escalate (boolean), reason (string), and summary (string). "
            "The summary should be one sentence describing the user's issue."
        )
        temp_history = self.history + [{"role": "user", "content": summary_prompt}]

        response = self.client.chat.completions.create(
            model=self.model,
            messages=temp_history,
            response_format={"type": "json_object"},
        )

        return json.loads(response.choices[0].message.content)

Run it

The script below simulates a short conversation, prints each assistant reply, and outputs a structured hand-off when frustration is detected.

if __name__ == "__main__":
    client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
    agent = SupportAgent(client)

    turns = [
        "My dashboard is blank after logging in.",
        "I tried that already. It is still useless.",
        "I want to cancel everything and talk to a human.",
    ]

    for turn in turns:
        print(f"User: {turn}")
        reply = agent.chat(turn)
        print(f"Agent: {reply}\n")

        if "connecting you with a specialist" in reply.lower():
            summary = agent.handoff_summary()
            print("Hand-off payload:", summary)
            break

Example output:

User: My dashboard is blank after logging in.
Agent: I am sorry to hear that. To help you, can you confirm whether you see any error message, or is the page completely blank?

User: I tried that already. It is still useless.
Agent: I understand your frustration. Let me connect you with a specialist now.

User: I want to cancel everything and talk to a human.
Agent: I am connecting you with a specialist now.

Hand-off payload: {'escalate': True, 'reason': 'Customer expressed frustration and requested human agent.', 'summary': 'User sees a blank dashboard after login and previous troubleshooting failed.'}

Next steps

Swap in qwen-3-32b or kimi-k2.6 if you need stronger agentic reasoning for tool use, such as looking up account status via function calling before replying. If you want to experiment at no cost, deepseek-v3.2 offers a free tier on Oxlo.ai and handles coding and reasoning tasks well enough for a prototype.