UNDERSTANDING VECTOR RAG, GRAPH RAG, AND AGENTIC RAG

Introduction: What is RAG, and why was it needed?
Retrieval-Augmented Generation (RAG) is an Artificial Intelligence technique that combines information retrieval with Large Language Models (LLMs). It allows an LLM to search for relevant information from external knowledge sources, such as documents, databases, websites, and company records, before generating an answer.
RAG was developed to address some limitations of LLMs. First, LLMs mainly depend on information learned during their training, which means their knowledge may become outdated. Second, they may not have access to private or specialized information, such as company documents. Third, they can produce incorrect information that sounds convincing, a problem known as hallucination.
RAG improves the quality of responses by retrieving relevant and updated information and providing it to the LLM as context. This helps the model generate answers that are more relevant, accurate, and supported by external information.
PART A: VECTOR RAG

1. How does Vector RAG work?
Vector RAG is a retrieval approach that uses numerical representations called embeddings to search for information based on meaning or semantic similarity.
It works through the following steps:

1. Document collection: The system collects documents such as PDFs, reports, and text files.
2. Chunking: The documents are divided into smaller sections called chunks.
3. Embedding: Each chunk is converted into a numerical vector using an embedding model.
4. Storage: The vectors and their associated text are stored in a vector database.
5. Query embedding: When a user asks a question, the question is also converted into a vector.
6. Similarity search: The system compares the question vector with stored vectors and retrieves the most relevant chunks.
7. Generation: The retrieved information is provided to the LLM, which generates the final answer.

For example, if a student asks, "How can I pay my university fees?", Vector RAG searches university documents and retrieves information about tuition payment methods, even if the documents use different wording.

2. When was it developed, and who introduced it?

The original Retrieval-Augmented Generation framework was introduced in 2020 by Patrick Lewis and other researchers from Facebook AI Research, University College London, and New York University.
Their research paper was titled "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" and was published at NeurIPS 2020.
The researchers combined a pre-trained language model with an external knowledge source using a dense vector index of Wikipedia. This research established an important foundation for modern vector-based RAG systems.

3. What problem did it solve?
Vector RAG addressed the problem of LLMs depending only on their internal training knowledge. Traditional LLMs could not easily access newly updated information or private documents.
It also addressed the difficulty of finding relevant information using exact keyword matching. By using embeddings, Vector RAG can identify information with similar meanings even when different words are used.

4. What are its real-world applications?

1. Customer support: Companies use Vector RAG to build chatbots that retrieve answers from product manuals, FAQs, and customer service documents.
2. Education: Universities can use Vector RAG to develop student assistants that answer questions using lecture notes, academic materials, and university handbooks.
3. **Healthcare: **Healthcare organizations can use it to retrieve relevant information from medical guidelines and research documents to support information access by professionals.

5. What are its limitations?

1. Limited understanding of relationships: Vector RAG mainly retrieves similar text sections and may fail to understand complex relationships between different people, organizations, and concepts.
2. Retrieval errors: It may retrieve irrelevant or incomplete information if the question is unclear or the embeddings do not represent the meaning accurately.
3. Chunking challenges: Dividing documents into small sections may separate important information that originally belonged together.
4. Dependence on embedding quality: Poor-quality embeddings can reduce the accuracy of semantic searches.
5. Limited global understanding: It can struggle with questions that require understanding the overall themes of a large collection of documents.

PART B: GRAPH RAG

1. How does Graph RAG work?

Graph RAG is an approach that combines retrieval-augmented generation with knowledge graphs. A knowledge graph organizes information into entities and relationships.
Entities are represented as nodes, while relationships between entities are represented as edges.
For example:
• Sonia → studies → Information Technology
• Sonia → attends → University
• University → offers → Machine Learning

Graph RAG works through the following steps:

1. Document collection: The system collects information from different documents.
2. Entity extraction: It identifies important entities such as people, places, organizations, and concepts.
3. Relationship extraction: It identifies how those entities are connected.
4. Graph construction: It organizes the entities and relationships into a knowledge graph.
5. Retrieval: When a user asks a question, the system searches the graph for relevant entities and their connections.
6. Generation: The retrieved information is provided to the LLM to generate an answer. For example, a student may ask, "Which lecturer teaches Machine Learning, and which university offers the course?" Graph RAG can follow the relationships between the lecturer, course, and university to retrieve the connected information.

2. When was it developed, and who introduced it?

Knowledge graphs existed before Graph RAG and have been researched for many years.
However, Microsoft's GraphRAG approach became prominent in 2024. It was introduced by researchers at Microsoft Research, including Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson.
Their research paper, "From Local to Global: A Graph RAG Approach to Query-Focused Summarization," was published as a preprint in April 2024.
Microsoft developed GraphRAG to improve how LLMs understand and answer questions about large collections of private documents.

3. What problem did it solve?

Graph RAG was developed to address the limitations of traditional vector-based retrieval when answering complex questions involving relationships between multiple entities.
Vector RAG is effective at finding relevant text sections, but it may not capture how information across different documents is connected.
For example, when analyzing thousands of research papers, it may be difficult to understand how different drugs, genes, diseases, and researchers are related.
Graph RAG solves this problem by organizing information into connected entities and relationships. It also helps answer global questions about the main themes and connections within large document collections.

4. What are its real-world applications?

1. Healthcare and medical research: Researchers can use Graph RAG to explore relationships between drugs, genes, diseases, and medical studies.
2. Financial analysis: Financial institutions can use it to understand relationships between companies, investors, transactions, and financial risks.
3. Legal research: Lawyers can use Graph RAG to connect cases, legal documents, people, laws, and important legal relationships.
4. Business intelligence: Organizations can use it to understand connections between departments, employees, projects, customers, and business activities.

5. What are its limitations?

1. High implementation complexity: Building a knowledge graph requires identifying entities, extracting relationships, and organizing information correctly.
2. High computational cost: Extracting entities and relationships using LLMs can require significant processing resources and API costs.
3. Data quality problems: Incorrectly extracted entities or relationships can produce misleading results.
4. Difficult maintenance: Knowledge graphs must be updated when new information becomes available or existing relationships change.
5. Limited suitability for simple questions: For basic questions that require retrieving one relevant document section, Graph RAG may introduce unnecessary complexity.

PART C: AGENTIC RAG

1. How does Agentic RAG work?

Agentic RAG is an advanced retrieval approach that uses AI agents to plan, manage, and control the retrieval process.
Unlike traditional RAG, which usually follows a fixed retrieval process, Agentic RAG allows an AI agent to decide which sources to search, which tools to use, and whether additional information is needed.
It works through the following steps:

1. User query: The user submits a question or task.
2. Planning: The AI agent analyzes the question and creates a plan for finding the required information.
3. Tool selection: The agent decides which tools or knowledge sources to use, such as vector databases, knowledge graphs, APIs, or web searches.
4. Retrieval: The agent searches for relevant information.
5. Evaluation: It checks whether the retrieved information is sufficient and relevant.
6. Additional retrieval: If information is missing, the agent can perform more searches or use different tools.
7. Generation: The agent provides the collected information to the LLM to generate the final response. For example, a financial analyst may ask, "Compare the revenue growth of this company with its three main competitors and identify the risks." An Agentic RAG system can search financial reports, retrieve revenue figures, compare company information, investigate risks, and generate a final report.

2. When was it developed, and who introduced it?

Agentic RAG does not have one universally recognized invention date or a single inventor. It developed from research on AI agents, tool use, reasoning, and retrieval-augmented generation.
An important foundation was introduced in 2022 by Shunyu Yao and other researchers in their paper titled "ReAct: Synergizing Reasoning and Acting in Language Models."
ReAct combined reasoning and actions, allowing language models to interact with external tools and gather additional information.
Another important development was Self-RAG, introduced by Akari Asai and other researchers in October 2023 and published at ICLR 2024. It enabled language models to decide when retrieval was needed and evaluate their own generated responses.
These developments helped establish the foundations for modern Agentic RAG systems, which became increasingly popular in 2024 and afterward.

3. What problem did it solve?

Agentic RAG was developed to address the limitations of fixed and predefined retrieval processes.
Traditional RAG systems usually retrieve information using a predefined search method. They may fail when a question requires multiple searches, different information sources, or several reasoning steps.
Agentic RAG improves this process by allowing AI agents to make decisions during retrieval. They can plan searches, select tools, evaluate results, and retrieve additional information when necessary.
This makes it more suitable for complex tasks that require several steps before generating a final answer.

4. What are its real-world applications?

1. Financial analysis: Financial analysts can use Agentic RAG to search company reports, compare financial performance, analyze competitors, and identify business risks.
2. Customer support: AI agents can retrieve customer information, check order statuses, search company policies, and provide personalized responses.
3. Research assistance: Researchers can use Agentic RAG to search multiple academic databases, compare research findings, and summarize information from different studies.
4. Healthcare information systems: Healthcare information assistants can retrieve medical guidelines and relevant records from different sources to support professionals in finding information.

5. What are its limitations?

1. High complexity: Developing an Agentic RAG system requires designing agents, tools, workflows, and retrieval processes.
2. High cost: Multiple searches and repeated LLM calls can increase API costs and computational requirements.
3. Slower responses: Performing several retrieval and reasoning steps can take more time than traditional RAG.
4. Unreliable decisions: An AI agent may select the wrong tool, make an incorrect plan, or retrieve irrelevant information.
5. Security risks: Giving agents access to external tools, private databases, or APIs may introduce security and privacy risks if permissions are not properly controlled.
6. Difficult evaluation: It can be challenging to test and monitor every decision and retrieval step performed by an agent.

PART D: COMPARISON AND CHOOSING THE RIGHT APPROACH

1. Comparison table

2. Scenario-based recommendations

Scenario 1: A company wants a chatbot that answers questions from its 500-page employee handbook.

Recommended approach: Vector RAG
I would choose Vector RAG because it is suitable for searching large collections of documents and retrieving relevant information based on the meaning of a question. The company can divide the employee handbook into smaller chunks, convert them into embeddings, and store them in a vector database. When employees ask questions about leave policies, salaries, or working hours, the system retrieves the relevant sections and provides them to an LLM. This approach is relatively simple, cost-effective, and appropriate for a document-based chatbot.

Scenario 2: A researcher needs to understand how drugs, genes, and diseases are connected across thousands of papers.

Recommended approach: Graph RAG

I would choose Graph RAG because the research requires understanding relationships between different entities, including drugs, genes, and diseases. It can extract these entities from research papers and represent their connections in a knowledge graph. Researchers can then explore how a particular drug is related to a gene or disease across multiple studies. This makes Graph RAG more suitable than ordinary vector retrieval for complex relationship-based research.

Scenario 3: A financial analyst asks: "Compare this company's revenue growth with its top three competitors and summarize the risks."

Recommended approach: Agentic RAG
I would choose Agentic RAG because the task requires several research and analysis steps. An AI agent can search financial reports, identify the company's competitors, retrieve revenue information, and compare the results. It can also search for financial risks and additional information when the initial results are incomplete. Finally, it can organize the findings into a report that explains the comparison and identifies possible risks.
3. How each approach grew out of the limitations of the one before it
Vector RAG, Graph RAG, and Agentic RAG represent different ways of improving how LLMs retrieve and use external information.
Vector RAG provided an important foundation by allowing LLMs to retrieve relevant information using semantic similarity, but it can struggle with complex relationships and questions that require understanding an entire collection of documents. Graph RAG addresses some of these limitations by organizing information into knowledge graphs containing connected entities and relationships. However, Graph RAG can be expensive and complicated to build and maintain, and it does not automatically provide flexible decision-making for every task. Agentic RAG extends retrieval by introducing AI agents that can plan searches, select different tools, evaluate results, and retrieve additional information when necessary. Together, these approaches demonstrate how RAG systems have developed from simple semantic retrieval into more structured and adaptive information retrieval systems.
**REFERENCES
**1. Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems (NeurIPS 2020). https://arxiv.org/abs/2005.11401

1. Edge, D., et al. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. Microsoft Research. https://arxiv.org/abs/2404.16130
2. Yao, S., et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. https://arxiv.org/abs/2210.03629
3. Asai, A., et al. (2023). Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. ICLR 2024. https://arxiv.org/abs/2310.11511
4. Microsoft Research. (2024). Project GraphRAG. https://www.microsoft.com/en-us/research/project/graphrag/