Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 13 min read

Building a Resilient RAG System with .NET 10 and AWS Bedrock

Large Language Models (LLMs) are remarkable reasoning engines, but they have two well-known operational limits in production: A fixed knowledge cutoff date. Zero visibility into your private internal systems and codeba

Large Language Models (LLMs) are remarkable reasoning engines, but they have two well-known operational limits in production:

  1. A fixed knowledge cutoff date.
  2. Zero visibility into your private internal systems and codebase.

Fine-tuning a foundational model to inject internal domain knowledge is expensive, time-consuming, and immediately outdated the moment your documentation changes.

The battle-tested design pattern for solving this is Retrieval-Augmented Generation (RAG). RAG treats the LLM as an on-demand reasoning engine rather than a static database: when a user asks a question, the system queries a vector store for relevant excerpts, injects those grounded passages into the prompt context, and instructs the model to answer based strictly on the retrieved evidence.

In this guide, we'll build an end-to-end RAG service using .NET 10, Microsoft's unified Microsoft.Extensions.AI abstractions, and AWS Bedrock.

Architecture Overview 

flowchart TD
    subgraph Ingestion["1. Ingestion Pipeline (Offline / Background)"]
        Doc["Raw Document (Markdown / Text)"] --> Chunking["Sliding-Window Chunking (500 chars / 50 overlap)"]
        Chunking --> Titan["AWS Bedrock: Amazon Titan Text Embeddings G1"]
        Titan --> OSIndex[("OpenSearch: k-NN Vector Index (1536-dim)")]
    end

    subgraph Query["2. Query & Generation Pipeline (Online / Synchronous)"]
        User["Client Request"] -->|POST /api/rag/ask| API["ASP.NET Core Minimal API"]
        API -->|1. Generate query vector| BedrockEmb["AWS Bedrock (Titan Embeddings)"]
        BedrockEmb -->|2. 1536d Float Array| API
        API -->|3. k-NN similarity search| OSSearch[("OpenSearch: Cosine Distance Search")]
        OSSearch -->|4. Top-k Relevant Chunks| API
        API -->|5. Guardrail System Prompt + Grounded Context| Claude["AWS Bedrock: Claude 3.5 Sonnet"]
        Claude -->|6. Grounded Answer + Source Attributions| API
        API -->|7. JSON Response| User
    end

Core Components

  • Runtime: .NET 10 Minimal APIs.
  • AI Abstraction Layer: Microsoft.Extensions.AI (IEmbeddingGenerator and IChatClient) for decoupled, provider-agnostic interfaces.
  • Vector Embeddings: Amazon Titan Text Embeddings G1 (amazon.titan-embed-text-v1) generating 1,536-dimensional dense vectors.
  • Reasoning Model: Anthropic Claude 3.5 Sonnet hosted on AWS Bedrock Runtime.
  • Vector Store: OpenSearch (configured with an HNSW k-NN index using cosine similarity).

1. AWS Infrastructure & Console Setup

Before touching code, your AWS environment needs access to Bedrock models and a functional vector database.

Step 1.1: Request Bedrock Model Access

Foundational models in Bedrock are locked by default until requested in the console.

  1. Open the AWS Management Console and switch to us-east-1 (N. Virginia).
  2. Search for Amazon Bedrock.
  3. In the left navigation menu, scroll down to Model access and click Modify model access (or Enable specific models).
  4. Select the two models we need:
    • Amazon: Titan Text Embeddings G1 - Text (amazon.titan-embed-text-v1).
    • Anthropic: Claude 3.5 Sonnet. > [!NOTE] > Anthropic models require a brief one-time use-case declaration (company name, URL, and high-level purpose). Approval is typically granted automatically within seconds.
  5. Click Next and Submit. Confirm that the status switches to Access granted.

Step 1.2: Set Up IAM Credentials & Local AWS CLI

The official .NET AWS SDK resolves credentials via the standard AWS credential chain (environment variables $\rightarrow$ ~/.aws/credentials $\rightarrow$ IAM roles).

  1. In the AWS Console, open IAM $\rightarrow$ Users $\rightarrow$ Create user (e.g. dotnet-rag-dev).
  2. Attach the AmazonBedrockFullAccess managed policy (or a scoped policy granting bedrock:InvokeModel).
  3. Under the Security credentials tab, generate an Access Key for the CLI.
  4. Configure your local environment: 
   aws configure

Provide your AWS Access Key ID, AWS Secret Access Key, and set the default region to us-east-1.

Step 1.3: Choose Your Vector Store (The Cost Factor)

You have two viable routes for the vector database:

Option A: Local Docker Container (Recommended for Local Dev & Zero Cost)

Amazon OpenSearch Serverless has a minimum allocation of 2 OpenSearch Compute Units (OCUs) β€” 1 for search, 1 for indexing β€” which costs roughly \$175/month even when completely idle. For local prototyping and testing, run OpenSearch locally with k-NN pre-installed:

docker run -d -p 9200:9200 -p 9600:9600 \
  -e "discovery.type=single-node" \
  -e "plugins.security.disabled=true" \
  --name opensearch-rag \
  opensearchproject/opensearch:2.11.0

Option B: Amazon OpenSearch Serverless (AOSS) in AWS

If you are deploying to an AWS environment:

  1. Navigate to Amazon OpenSearch Service $\rightarrow$ Serverless $\rightarrow$ Collections $\rightarrow$ Create collection.
  2. Name it rag-knowledge-base and select Vector search.
  3. Create the three required policies:
    • Encryption policy: AWS-owned KMS key.
    • Network policy: Public access enabled for the collection endpoint.
    • Data access policy: Map your IAM user/role with index management permissions (aoss:*).
  4. Note the generated collection endpoint URL (https://<collection-id>.us-east-1.aoss.amazonaws.com). > [!WARNING] > OpenSearch Serverless strictly requires all HTTP traffic to be signed using AWS SigV4. Direct unauthenticated HTTP requests will be rejected with HTTP 403 Forbidden.

2. Project Setup & Dependencies

Initialize a new ASP.NET Core Web API project:

dotnet new webapi -n DotnetRagAws
cd DotnetRagAws

Install the AWS Bedrock SDK, the Microsoft.Extensions.AI package, and the OpenSearch client:

# Microsoft AI abstractions
dotnet add package Microsoft.Extensions.AI
dotnet add package Microsoft.Extensions.AI.Abstractions

# AWS Bedrock Runtime client
dotnet add package AWSSDK.BedrockRuntime

# OpenSearch .NET client
dotnet add package OpenSearch.Client

Ensure your appsettings.json points to your vector store:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*",
  "OpenSearch": {
    "Endpoint": "http://localhost:9200"
  }
}

3. Implementation

3.1. Document Chunk Model (DocumentChunk.cs)

This entity represents an indexed passage stored in OpenSearch:

using System.Text.Json.Serialization;

public class DocumentChunk
{
    [JsonPropertyName("id")]
    public string Id { get; set; } = Guid.NewGuid().ToString();

    [JsonPropertyName("content")]
    public string Content { get; set; } = string.Empty;

    [JsonPropertyName("source")]
    public string Source { get; set; } = string.Empty;

    [JsonPropertyName("embedding")]
    public float[] Embedding { get; set; } = Array.Empty<float>();
}

3.2. Embedding Generator Adapter (BedrockEmbeddingGenerator.cs)

We adapt Amazon Titan Text Embeddings to the standard IEmbeddingGenerator<string, Embedding<float>> interface. Titan produces a 1,536-dimensional floating-point array:

using System.Text.Json;
using System.Text.Json.Serialization;
using Amazon.BedrockRuntime;
using Amazon.BedrockRuntime.Model;
using Microsoft.Extensions.AI;

public class BedrockEmbeddingGenerator : IEmbeddingGenerator<string, Embedding<float>>
{
    private readonly IAmazonBedrockRuntime _bedrockClient;
    private readonly string _modelId;

    public BedrockEmbeddingGenerator(IAmazonBedrockRuntime bedrockClient, string modelId = "amazon.titan-embed-text-v1")
    {
        _bedrockClient = bedrockClient;
        _modelId = modelId;
    }

    public async Task<GeneratedEmbeddings<Embedding<float>>> GenerateAsync(
        IEnumerable<string> values,
        EmbeddingGenerationOptions? options = null,
        CancellationToken cancellationToken = default)
    {
        var embeddings = new List<Embedding<float>>();

        foreach (var text in values)
        {
            var payload = JsonSerializer.Serialize(new { inputText = text });

            var request = new InvokeModelRequest
            {
                ModelId = _modelId,
                ContentType = "application/json",
                Accept = "application/json",
                Body = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(payload))
            };

            var response = await _bedrockClient.InvokeModelAsync(request, cancellationToken);

            using var reader = new StreamReader(response.Body);
            var responseBody = await reader.ReadToEndAsync(cancellationToken);
            var result = JsonSerializer.Deserialize<TitanEmbeddingResponse>(responseBody);

            if (result?.Embedding != null)
            {
                embeddings.Add(new Embedding<float>(result.Embedding));
            }
        }

        return new GeneratedEmbeddings<Embedding<float>>(embeddings);
    }

    public EmbeddingGeneratorMetadata Metadata => new("BedrockEmbeddingGenerator");

    public object? GetService(Type serviceType, object? serviceKey = null) => null;

    public void Dispose() { }

    private sealed record TitanEmbeddingResponse(
        [property: JsonPropertyName("embedding")] float[] Embedding
    );
}

3.3. OpenSearch Vector Store (OpenSearchVectorStore.cs)

This service initializes an HNSW k-NN index configured for 1,536 dimensions with cosine similarity, and runs vector nearest-neighbor queries:

using System.Net.Http.Json;
using System.Text.Json;
using OpenSearch.Client;

public class OpenSearchVectorStore
{
    private readonly IOpenSearchClient _client;
    private const string IndexName = "rag-knowledge-base";

    public OpenSearchVectorStore(IOpenSearchClient client)
    {
        _client = client;
    }

    public async Task EnsureIndexExistsAsync(CancellationToken cancellationToken = default)
    {
        var existsResponse = await _client.Indices.ExistsAsync(IndexName, ct: cancellationToken);
        if (existsResponse.Exists)
        {
            return;
        }

        var createIndexResponse = await _client.Indices.CreateAsync(IndexName, c => c
            .Settings(s => s
                .Setting("index.knn", true)
            )
            .Map<DocumentChunk>(m => m
                .Properties(p => p
                    .Keyword(k => k.Name(n => n.Id))
                    .Text(t => t.Name(n => n.Content))
                    .Keyword(k => k.Name(n => n.Source))
                    .KnnVector(v => v
                        .Name(n => n.Embedding)
                        .Dimension(1536)
                        .Method(mi => mi
                            .Name("hnsw")
                            .Engine("nmslib")
                            .SpaceType("cosinesimil")
                        )
                    )
                )
            ),
            cancellationToken
        );

        if (!createIndexResponse.IsValid)
        {
            throw new InvalidOperationException($"Failed to create OpenSearch index: {createIndexResponse.DebugInformation}");
        }
    }

    public async Task IndexDocumentsAsync(IEnumerable<DocumentChunk> documents, CancellationToken cancellationToken = default)
    {
        var bulkResponse = await _client.BulkAsync(b => b
            .Index(IndexName)
            .IndexMany(documents),
            cancellationToken
        );

        if (!bulkResponse.IsValid)
        {
            throw new InvalidOperationException($"Failed to bulk index documents: {bulkResponse.DebugInformation}");
        }
    }

    public async Task<IReadOnlyList<DocumentChunk>> SearchSimilarAsync(
        float[] queryEmbedding,
        int k = 3,
        CancellationToken cancellationToken = default)
    {
        var searchResponse = await _client.SearchAsync<DocumentChunk>(s => s
            .Index(IndexName)
            .Size(k)
            .Query(q => q
                .Knn(knn => knn
                    .Field(f => f.Embedding)
                    .Vector(queryEmbedding)
                    .K(k)
                )
            ),
            cancellationToken
        );

        if (!searchResponse.IsValid)
        {
            throw new InvalidOperationException($"Vector search query failed: {searchResponse.DebugInformation}");
        }

        return searchResponse.Documents.ToList();
    }
}

3.4. Ingestion Pipeline (IngestionService.cs)

Splits raw documentation into chunks using a sliding window with overlap, generates embeddings, and bulk-loads them into the vector store:

using Microsoft.Extensions.AI;

public class IngestionService
{
    private readonly IEmbeddingGenerator<string, Embedding<float>> _embeddingGenerator;
    private readonly OpenSearchVectorStore _vectorStore;

    public IngestionService(
        IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator,
        OpenSearchVectorStore vectorStore)
    {
        _embeddingGenerator = embeddingGenerator;
        _vectorStore = vectorStore;
    }

    public async Task IngestAsync(string rawText, string sourceDocument, int chunkSize = 500, int chunkOverlap = 50, CancellationToken cancellationToken = default)
    {
        await _vectorStore.EnsureIndexExistsAsync(cancellationToken);

        var chunks = ChunkText(rawText, chunkSize, chunkOverlap);
        var documentChunks = new List<DocumentChunk>();

        foreach (var chunk in chunks)
        {
            var embeddings = await _embeddingGenerator.GenerateAsync(new[] { chunk }, cancellationToken: cancellationToken);
            var embeddingVector = embeddings.First().Vector.ToArray();

            documentChunks.Add(new DocumentChunk
            {
                Content = chunk,
                Source = sourceDocument,
                Embedding = embeddingVector
            });
        }

        await _vectorStore.IndexDocumentsAsync(documentChunks, cancellationToken);
    }

    private static List<string> ChunkText(string text, int chunkSize, int overlap)
    {
        var chunks = new List<string>();
        if (string.IsNullOrWhiteSpace(text))
        {
            return chunks;
        }

        var start = 0;
        while (start < text.Length)
        {
            var length = Math.Min(chunkSize, text.Length - start);
            var chunk = text.Substring(start, length).Trim();

            if (!string.IsNullOrEmpty(chunk))
            {
                chunks.Add(chunk);
            }

            if (start + length >= text.Length)
            {
                break;
            }

            start += chunkSize - overlap;
        }

        return chunks;
    }
}

3.5. Bedrock Claude Chat Client (BedrockChatClient.cs)

Plugs Anthropic Claude 3.5 Sonnet on Bedrock into the modern IChatClient interface (GetResponseAsync returning ChatResponse):

using System.Text.Json;
using System.Text.Json.Serialization;
using Amazon.BedrockRuntime;
using Amazon.BedrockRuntime.Model;
using Microsoft.Extensions.AI;

public class BedrockChatClient : IChatClient
{
    private readonly IAmazonBedrockRuntime _bedrockClient;
    private readonly string _modelId;

    public BedrockChatClient(IAmazonBedrockRuntime bedrockClient, string modelId = "anthropic.claude-3-5-sonnet-20241022-v2:0")
    {
        _bedrockClient = bedrockClient;
        _modelId = modelId;
    }

    public async Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> chatMessages,
        ChatOptions? options = null,
        CancellationToken cancellationToken = default)
    {
        var messagesList = chatMessages.ToList();
        var systemMessage = messagesList.FirstOrDefault(m => m.Role == ChatRole.System)?.Text ?? string.Empty;
        var conversationMessages = messagesList
            .Where(m => m.Role != ChatRole.System)
            .Select(m => new
            {
                role = m.Role == ChatRole.User ? "user" : "assistant",
                content = m.Text
            })
            .ToList();

        var payload = new
        {
            anthropic_version = "bedrock-2023-05-31",
            max_tokens = options?.MaxOutputTokens ?? 1000,
            system = systemMessage,
            messages = conversationMessages
        };

        var request = new InvokeModelRequest
        {
            ModelId = _modelId,
            ContentType = "application/json",
            Accept = "application/json",
            Body = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(JsonSerializer.Serialize(payload)))
        };

        var response = await _bedrockClient.InvokeModelAsync(request, cancellationToken);

        using var reader = new StreamReader(response.Body);
        var responseBody = await reader.ReadToEndAsync(cancellationToken);
        var result = JsonSerializer.Deserialize<ClaudeResponse>(responseBody);

        var contentText = result?.Content?.FirstOrDefault()?.Text ?? string.Empty;
        var returnMessage = new ChatMessage(ChatRole.Assistant, contentText);

        return new ChatResponse(returnMessage);
    }

    public IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
        IEnumerable<ChatMessage> chatMessages,
        ChatOptions? options = null,
        CancellationToken cancellationToken = default)
    {
        throw new NotImplementedException("Streaming can be implemented using Bedrock InvokeModelWithResponseStreamAsync.");
    }

    public ChatClientMetadata Metadata => new("BedrockChatClient");

    public object? GetService(Type serviceType, object? serviceKey = null) => null;

    public void Dispose() { }

    private sealed record ClaudeResponse(
        [property: JsonPropertyName("content")] List<ClaudeContentBlock>? Content
    );

    private sealed record ClaudeContentBlock(
        [property: JsonPropertyName("type")] string Type,
        [property: JsonPropertyName("text")] string Text
    );
}

3.6. RAG Orchestrator (RagOrchestratorService.cs)

Coordinates retrieval and generation while enforcing strict guardrail prompts:

using System.Text;
using Microsoft.Extensions.AI;

public class RagOrchestratorService
{
    private readonly IEmbeddingGenerator<string, Embedding<float>> _embeddingGenerator;
    private readonly OpenSearchVectorStore _vectorStore;
    private readonly IChatClient _chatClient;

    public RagOrchestratorService(
        IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator,
        OpenSearchVectorStore vectorStore,
        IChatClient chatClient)
    {
        _embeddingGenerator = embeddingGenerator;
        _vectorStore = vectorStore;
        _chatClient = chatClient;
    }

    public async Task<RagResult> AskQuestionAsync(string question, int topK = 3, CancellationToken cancellationToken = default)
    {
        // 1. Vectorize user question
        var questionEmbeddingResult = await _embeddingGenerator.GenerateAsync(new[] { question }, cancellationToken: cancellationToken);
        var queryVector = questionEmbeddingResult.First().Vector.ToArray();

        // 2. Fetch nearest semantically matched passages
        var relevantChunks = await _vectorStore.SearchSimilarAsync(queryVector, k: topK, cancellationToken: cancellationToken);

        // 3. Assemble grounded context
        var contextBuilder = new StringBuilder();
        var sources = new HashSet<string>();

        foreach (var chunk in relevantChunks)
        {
            contextBuilder.AppendLine($"[Source: {chunk.Source}]\n{chunk.Content}\n---");
            sources.Add(chunk.Source);
        }

        // 4. Construct prompt with anti-hallucination guardrails
        var systemPrompt = 
            "You are a factual enterprise technical assistant. Answer the user's question using ONLY the provided context snippets.\n" +
            "If the answer cannot be determined directly from the context, state clearly: 'I do not have enough information to answer that based on the provided documents.'\n" +
            "Always cite the source document names when providing answers.";

        var userPrompt = 
            $"Context:\n{contextBuilder}\n\n" +
            $"Question: {question}";

        var messages = new List<ChatMessage>
        {
            new(ChatRole.System, systemPrompt),
            new(ChatRole.User, userPrompt)
        };

        // 5. Generate grounded completion via Claude 3.5 Sonnet
        var response = await _chatClient.GetResponseAsync(messages, cancellationToken: cancellationToken);

        return new RagResult
        {
            Answer = response.Text ?? string.Empty,
            Sources = sources.ToList()
        };
    }
}

public class RagResult
{
    public string Answer { get; set; } = string.Empty;
    public List<string> Sources { get; set; } = new();
}

3.7. Wiring Dependencies & Endpoints (Program.cs) 

using Amazon.BedrockRuntime;
using Amazon.Runtime;
using Microsoft.Extensions.AI;
using OpenSearch.Client;

var builder = WebApplication.CreateBuilder(args);

// 1. Bedrock Runtime Client
builder.Services.AddSingleton<IAmazonBedrockRuntime>(sp =>
{
    var region = Amazon.RegionEndpoint.USEast1;
    return new AmazonBedrockRuntimeClient(region);
});

// 2. OpenSearch Client
builder.Services.AddSingleton<IOpenSearchClient>(sp =>
{
    var endpoint = builder.Configuration["OpenSearch:Endpoint"] 
                   ?? "http://localhost:9200";
    var settings = new ConnectionSettings(new Uri(endpoint))
        .DefaultIndex("rag-knowledge-base");

    return new OpenSearchClient(settings);
});

// 3. Microsoft.Extensions.AI Implementations
builder.Services.AddSingleton<IEmbeddingGenerator<string, Embedding<float>>, BedrockEmbeddingGenerator>();
builder.Services.AddSingleton<IChatClient, BedrockChatClient>();

// 4. Domain & Orchestration Services
builder.Services.AddSingleton<OpenSearchVectorStore>();
builder.Services.AddScoped<IngestionService>();
builder.Services.AddScoped<RagOrchestratorService>();

var app = builder.Build();

// Endpoint: Ingest technical documentation
app.MapPost("/api/rag/ingest", async (
    IngestRequest request, 
    IngestionService ingestionService, 
    CancellationToken ct) =>
{
    if (string.IsNullOrWhiteSpace(request.Text))
    {
        return Results.BadRequest(new { error = "Text payload cannot be empty." });
    }

    await ingestionService.IngestAsync(request.Text, request.Source, cancellationToken: ct);
    return Results.Ok(new { status = "Ingestion completed successfully", source = request.Source });
});

// Endpoint: Query the RAG pipeline
app.MapPost("/api/rag/ask", async (
    AskRequest request, 
    RagOrchestratorService ragService, 
    CancellationToken ct) =>
{
    if (string.IsNullOrWhiteSpace(request.Question))
    {
        return Results.BadRequest(new { error = "Question cannot be empty." });
    }

    var result = await ragService.AskQuestionAsync(request.Question, cancellationToken: ct);
    return Results.Ok(result);
});

app.Run();

// Request DTOs
public record IngestRequest(string Text, string Source);
public record AskRequest(string Question);

4. Verification & Testing

Step 1: Start the API

Run the application:

dotnet run

Check the console output for the assigned port (e.g. http://localhost:5055 as configured in Properties/launchSettings.json).

Step 2: Ingest an Architecture Document

In a separate terminal window, ingest a runbook excerpt:

curl -X POST http://localhost:5055/api/rag/ingest \
  -H "Content-Type: application/json" \
  -d '{
    "source": "PaymentService_Architecture_Runbook.md",
    "text": "PaymentService uses SQS FIFO queues with a DLQ retention period of 14 days. Batch size is limited to 10 messages with reportBatchItemFailures enabled to prevent head-of-line blocking."
  }'

Expected Response:

{
  "status": "Ingestion completed successfully",
  "source": "PaymentService_Architecture_Runbook.md"
}

Step 3: Ask a Grounded Question

Ask a specific technical question covered by the runbook:

curl -X POST http://localhost:5055/api/rag/ask \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What is the DLQ retention period and batch size for the PaymentService?"
  }'

Expected Response:

{
  "answer": "According to the PaymentService Architecture Runbook, the PaymentService uses SQS FIFO queues with a DLQ retention period of 14 days, and the batch size is limited to 10 messages with reportBatchItemFailures enabled.",
  "sources": [
    "PaymentService_Architecture_Runbook.md"
  ]
}

Step 4: Verify the Anti-Hallucination Guardrail

Now test how the system reacts to information not present in the indexed documents:

curl -X POST http://localhost:5055/api/rag/ask \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What is the primary database password and connection string?"
  }'

Expected Response:

{
  "answer": "I do not have enough information to answer that based on the provided documents.",
  "sources": [
    "PaymentService_Architecture_Runbook.md"
  ]
}

The system prompt prevents the model from speculating or fabricating secrets.

5. Production Considerations

Moving from a local prototype to production requires hardening several operational details:

  1. SigV4 Authentication for OpenSearch Serverless: If targeting AWS OpenSearch Serverless instead of self-hosted OpenSearch, plug an AWS4HttpHandler or custom signing DelegatingHandler into ConnectionSettings so every HTTP request includes an AWS Signature Version 4 header.
  2. Bedrock Throttling & Resilience: AWS Bedrock applies per-region Transactions-Per-Second (TPS) quotas on foundation models. Wrap your Bedrock calls with exponential backoff and jitter policies using Microsoft.Extensions.Resilience or Polly.
  3. Cross-Region Inference Profiles: In high-traffic US regions, AWS recommends or mandates using regional inference profiles (e.g. us.anthropic.claude-3-5-sonnet-20241022-v2:0) instead of direct model IDs for load distribution and automatic fallback.
  4. Token-Aware Chunking: For complex codebases or markdown specs, replace character-based chunking with token-aware chunking (e.g. using a BPE tokenizer) to preserve code block integrity and logical paragraph boundaries.
  5. Hybrid Search (k-NN + BM25): Pure vector distance search can overlook exact keyword matches (such as specific GUIDs, exception names, or configuration flags). Combining k-NN with BM25 keyword scoring yields significantly higher precision in engineering domains.
πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.

Share 𝕏 Twitter ✈ Telegram β†— Reddit 🟧 HN
← News πŸ€– Ai 🏠 Home