Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 6 min read

Building NovaDesk AI: Moving Beyond Basic RAG with Query Rewriting and HyDE

Setting up a basic RAG (Retrieval-Augmented Generation) pipeline takes ten minutes. You load a few PDFs, chunk them, push the embeddings into a vector store like FAISS, and run a similarity search. In a clean demo, it

Building NovaDesk AI: Moving Beyond Basic RAG with Query Rewriting and HyDE

Setting up a basic RAG (Retrieval-Augmented Generation) pipeline takes ten minutes. You load a few PDFs, chunk them, push the embeddings into a vector store like FAISS, and run a similarity search.

In a clean demo, it looks like magic. In practice, it breaks down the moment a real user types a messy, conversational question.

While building NovaDesk AIβ€”my stateful hotel concierge powered by LangGraph, FastAPI, and SQLiteβ€”I needed the system to answer unstructured hotel policy questions reliably without hallucinating. This meant moving past naive retrieval.

You can check out the previous post here where I have introduced this project:

In this post, I will walk through how I evolved the RAG pipeline in NovaDesk AI across three distinct stages: Standard RAG, Query Rewriting, and HyDE (Hypothetical Document Embeddings).

Stage 1: Standard (Naive) RAG and Where It Broke

My initial setup was a textbook RAG pipeline. I took the hotel policy documents (covering check-in times, pet rules, cancellation fees, and amenities), split them into chunks, and indexed them in a FAISS vector store.

When a user asked a question, the system embedded the raw query, fetched the top-k most similar chunks, and passed them to the LLM to generate an answer.

The Problem

Standard RAG works well when the user's phrasing closely mirrors the wording in your documents (e.g., "What is the standard check-in time?").

However, real hotel guests do not speak like policy manuals. When I tested the concierge with realistic, messy queries, retrieval quality dropped sharply:

  • Laymen language & typos: "Can I bring my husky and how much extra?"
  • Multi-part questions: "Do you have airport pickup and is breakfast free if I book a deluxe room?"
  • Conversational noise: "Hey, I'm running super late for my flight, will I lose my room if I show up at 2 AM?"

Because the raw user query was full of conversational fillerβ€”and structurally very different from formal policy textβ€”the vector search frequently pulled irrelevant chunks or missed half of a multi-part question.

Stage 2: Introducing a Query Rewriter

Before changing the vector database or tweaking chunk sizes, I looked at the input. Garbage in, garbage out.

To fix this, I added a Query Rewriter step right before the retrieval call. Instead of embedding the user's raw message, an LLM first translates the messy input into a clean, search-optimized query.

  • Raw User Query: "Hey, can I bring my husky with me and how much extra will it cost?"
  • Rewritten Query: "Hotel pet policy, allowed dog breeds, pet fees and charges"

Here is a simplified look at how this step is structured using LangChain:

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# =====================================================================
# 2. QUERY TRANSFORMATION CHAINS (HYDE & QUERY REWRITER)
# =====================================================================
# Mode A: Query Rewriter Prompt (Strips conversational noise into keywords)
rewriter_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are an expert search query optimizer for a hotel policy database. "
               "Rewrite the guest's message into a short, direct, keyword-focused search query. "
               "Strip out all personal backstory, names, and conversational filler. "
               "If the question has multiple parts, include keywords for all parts."),
    ("human", "{user_query}")
])
rewriter_chain = rewriter_prompt | llm | StrOutputParser()


# Usage before hitting FAISS
optimized_query = rewriter_chain.invoke({"question": user_input})
docs = retriever.invoke(optimized_query)

The Impact

The Query Rewriter immediately stripped away conversational fluff and standardized the vocabulary. Multi-part questions and casual phrasing started hitting the right sections of the policy documents far more consistently.

Stage 3: Bridging the Semantic Gap with HyDE

Even with a Query Rewriter, there is a fundamental flaw in standard vector search: questions and answers do not look alike in vector space.

A user's query is short and interrogative ("What happens if I show up at 2 AM?"). The policy document is long and declarative ("Guests arriving after 11:00 PM must notify the front desk in advance to prevent a no-show cancellation..."). Comparing a question embedding against a document embedding is an asymmetric match.

To solve this, I implemented HyDE (Hypothetical Document Embeddings).

How HyDE Works

Take the user's question (or rewritten query).

Ask the LLM to generate a hypothetical, ideal policy excerpt that would answer the question. (It does not matter if the LLM hallucinates specific numbers hereβ€”we are only using it for semantic structure).

Embed that hypothetical document and use it to search the FAISS vector store.

Pass the real retrieved policy chunks to the final LLM to generate the factual answer.

# Mode A: HyDE Prompt (Generates a formal hypothetical policy paragraph)
hyde_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are an expert at writing formal hotel policy documents. "
               "The user will ask a question. Write a short, single-paragraph hypothetical hotel policy "
               "that perfectly answers their question. Use professional, legal hospitality terminology."),
    ("human", "{user_query}")
])
hyde_chain = hyde_prompt | llm | StrOutputParser()

# 1. Generate the hypothetical answer
hypothetical_doc = hyde_chain.invoke({"question": optimized_query})

# 2. Search FAISS using the hypothetical document instead of the raw question
retrieved_docs = retriever.invoke(hypothetical_doc)

The Impact & Trade-offs

HyDE shifted the search from question-to-document matching to document-to-document matching. For nuanced, indirect questions where the user did not know the exact terminology (like "no-show policy" or "late arrival"), HyDE pulled significantly more relevant context than standard retrieval.

The Trade-off: Latency and token cost. Adding a Query Rewriter and a HyDE step introduces extra LLM calls before you even generate the final answer. In a real-time concierge, you have to balance retrieval accuracy against how long the user is willing to wait for a response.

Key Takeaways

Fix retrieval before upgrading your LLM: When a RAG system hallucinates or gives incomplete answers, the bottleneck is almost always the context being fed into the prompt, not the reasoning power of the model.

Users do not write like your database: Adding a lightweight Query Rewriter is one of the highest-ROI improvements you can make to a conversational RAG pipeline.

HyDE shines on indirect queries: When there is a wide vocabulary gap between how users ask questions and how your documentation is written, generating a hypothetical answer bridges that gap effectively.

You can explore the full codebase for NovaDesk AI on my GitHub repository.

πŸ›ŽοΈ AI Hotel Concierge (LangGraph + FastAPI)

An enterprise-grade, stateful AI agent built to simulate a virtual hotel front desk. Unlike basic Q&A chatbots, this system bridges the gap between unstructured natural language and strict database transactions, allowing users to query policies, check room availability, book rooms, and request room service seamlessly.

This project demonstrates advanced AI engineering concepts including Agentic Orchestration (LangGraph), Advanced RAG (HyDE & Query Rewriting), State & Memory Management, and Human-in-the-Loop (HITL) security via FastAPI.

πŸ—οΈ Architecture Overview

The system is broken down into three main layers:

Frontend (Chainlit): Provides a clean, conversational chat UI.

Orchestration Brain (LangGraph): Manages conversation state, memory, guardrails, and decision routing.

Action Execution (FastAPI): Serves as the toolkit for the agent, exposing REST endpoints to interact securely with the SQLite database.

System Architecture

Agent Architecture

✨ Key Features

  • Strict Domain Guardrails: A routing node intercepts prompts. If a user asks off-topic questions (e.g…





Have you implemented HyDE or Query Rewriting in your RAG pipelines? Did you find the accuracy boost worth the extra LLM latency? Let me know below! πŸ‘‡

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.