Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 9 min read

Applied AI Engineering: 5 Concepts Every Developer Needs

Introduction: Building AI Applications Is More Than Calling an API Have you ever integrated an AI API into a Python application, received an impressive response, and wondered what would happen when the model returned i

Applied AI Engineering: 5 Concepts Every Developer Needs

Introduction: Building AI Applications Is More Than Calling an API

Have you ever integrated an AI API into a Python application, received an impressive response, and wondered what would happen when the model returned incorrect JSON, exceeded its token limit, or suddenly became unavailable?

That's where the difference between using AI and engineering AI applications becomes important.

Modern developers don't necessarily need to train massive neural networks or build foundation models from scratch. They need to understand how existing models work, integrate them into software systems, validate their outputs, manage costs, and handle failures.

This is the foundation of Applied AI Engineering.

In this article, we'll explore five practical concepts every developer should understand before building production-ready applications powered by Large Language Models (LLMs).

By the end, you'll understand model selection, embeddings, tokenization, inference, reliability, and API integration through practical Python examples.

1. Understand the AI Model Landscape Before Choosing a Model

One common mistake developers make is assuming that every AI problem requires an LLM.

It doesn't.

Artificial Intelligence is a broad field, and different techniques solve different problems.

Here's a simplified breakdown:

Technology What it does Example
AI Broad intelligent behavior Rule-based assistant
Machine Learning Learns patterns from data Spam detection
Deep Learning Uses multilayer neural networks Image recognition
Generative AI Creates new content Text and image generation
LLM Processes and generates language AI chatbot

Developers should also understand three important model categories:

  • LLMs: Generate and process text.
  • Vision-Language Models (VLMs): Understand text and images.
  • Embedding Models: Convert information into numerical vectors for semantic comparison.

Practical implementation: Semantic similarity with embeddings

Imagine building a technical documentation search system.

A user searches:

"How do I restart nginx?"

Your documentation contains:

"How can I reload the nginx service?"

Traditional keyword matching might struggle to recognize that these sentences express similar meanings.

Embeddings help solve this problem.

import httpxOLLAMA_URL = "http://localhost:11434"MODEL = "nomic-embed-text"def create_embedding(text: str) -> list[float]:    response = httpx.post(        f"{OLLAMA_URL}/api/embeddings",        json={            "model": MODEL,            "prompt": text        },        timeout=30.0    )    response.raise_for_status()    return response.json()["embedding"]def cosine_similarity(a, b):    dot = sum(x * y for x, y in zip(a, b))    magnitude_a = sum(x * x for x in a) ** 0.5    magnitude_b = sum(x * x for x in b) ** 0.5    if magnitude_a == 0 or magnitude_b == 0:        return 0.0    return dot / (magnitude_a * magnitude_b)query = "How do I restart nginx?"document = "How can I reload the nginx service?"score = cosine_similarity(    create_embedding(query),

Install and prepare the model:

ollama pull nomic-embed-text
pip install httpx

Developer takeaway: Embedding models represent meaning numerically; they don't generate answers. They're particularly useful for semantic search, recommendations, duplicate detection, and retrieval-augmented generation (RAG).

2. Understand Tokens and Context Windows

Here's something many developers overlook when working with LLM APIs:

Models process tokens, not ordinary words.

A token can represent:

  • A complete word.
  • Part of a word.
  • Punctuation.
  • Whitespace.
  • A special symbol.

Why does this matter?

Because token consumption directly affects API costs, context limits, and application performance.

A context window defines how many tokens a model can consider during a request.

It may contain:

  1. System instructions.
  2. Conversation history.
  3. Retrieved documents.
  4. User messages.
  5. Generated output, depending on the provider's limits.

Practical implementation: Count tokens using Python

import tiktokendef analyze_text(text: str):    encoder = tiktoken.get_encoding("cl100k_base")    tokens = encoder.encode(text)    print("Characters:", len(text))    print("Words:", len(text.split()))    print("Tokens:", len(tokens))text = """Applied AI Engineering focuses on buildingreliable applications using existing AI models."""analyze_text(text)

Install the dependency:

pip install tiktoken

Note that cl100k_base is an example encoding. Actual token counts depend on the model's tokenizer.

Why developers should care

Suppose your application sends an entire 100-page document to an LLM every time a user asks a question.

That approach can increase:

  • Token consumption.
  • Request latency.
  • API costs.
  • Context overflow risk.

A better architecture retrieves only relevant information before sending it to the model.

This is one of the fundamental ideas behind RAG.

Developer takeaway: Tokenization isn't just an AI concept. It's an important application-design and cost-management consideration.

3. Hosted vs. Local LLMs: Choose the Right Inference Architecture

Where should your AI model run?

This is an important architectural decision when building AI-powered applications.

There are two common approaches.

Hosted inference

Your application sends requests to an external AI provider through an API.

Advantages:

  • Access to powerful foundation models.
  • No local GPU requirement.
  • Provider-managed infrastructure.
  • Easier scaling.

Disadvantages:

  • Recurring API costs.
  • Network dependency.
  • Rate limits.
  • Data privacy considerations.

Local inference

Your application runs an open-weight model on your own computer or server using tools such as Ollama.

Advantages:

  • Greater control over deployment.
  • Data can remain on your machine.
  • No per-token API charges.
  • Offline experimentation after downloading models.

Disadvantages:

  • Hardware limitations.
  • Model management and maintenance.
  • Potentially slower inference.
  • Smaller models may provide lower-quality responses.

Practical implementation: Run a local LLM using Ollama

First, install Ollama and download a small model:

ollama pull llama3.2:1b
pip install httpx python-dotenv

Create local_llm.py:

import httpximport timeOLLAMA_URL = "http://localhost:11434"MODEL = "llama3.2:1b"def generate_response(prompt: str):    start = time.perf_counter()    response = httpx.post(        f"{OLLAMA_URL}/api/generate",        json={            "model": MODEL,            "prompt": prompt,            "stream": False,            "options": {                "temperature": 0.2            }        },        timeout=120.0    )    response.raise_for_status()    elapsed = (time.perf_counter() - start) * 1000    result = response.json()    return {        "response": result["response"],        "latency_ms": round(elapsed, 2),        "input_tokens": result.get("prompt_eval_count"),        "output_tokens": result.get("eval_count")    }

You now have a basic local inference client.

Which approach should you choose?

Requirement Suitable approach
Powerful models Hosted
Local experimentation Local
Sensitive local data Local, with appropriate controls
Minimal infrastructure management Hosted
Offline operation Local
Provider-managed scaling Hosted

Don't assume that a local model and a hosted model with similar names provide equivalent capabilities.

Benchmark both using identical prompts and evaluate quality, latency, token usage, privacy, and operational cost.

Developer takeaway: Model deployment is an architectural decision, not simply a matter of choosing an AI provider.

4. Build Reliability: Handle Hallucinations and Unpredictable Outputs

Here's an important production lesson:

An LLM response is not automatically a trustworthy application result.

LLMs can produce hallucinations: confident responses containing incorrect or unsupported information.

For example, an AI-powered IT support application might:

  • Invent a Linux command.
  • Recommend an incorrect configuration.
  • Generate invalid JSON.
  • Provide an outdated API method.
  • Misinterpret an ambiguous support ticket.

Another challenge is nondeterminism.

The same prompt may produce different responses across repeated requests. Temperature influences output variation, but setting it to zero does not guarantee identical responses across every provider and deployment.

Practical implementation: Validate AI-generated JSON

Imagine an application that extracts technical information from server error messages.

You want a predictable structure:

{
  "service": "nginx",
  "port": 80,
  "event": "bind_failure"
}

Instead of blindly trusting the model, validate its output using Pydantic.

import jsonfrom pydantic import BaseModel, Fieldclass ExtractionResult(BaseModel):    service: str | None = None    port: int | None = Field(default=None, ge=1, le=65535)    event: str | None = Nonedef validate_ai_output(raw_response: str):    try:        parsed = json.loads(raw_response)        result = ExtractionResult.model_validate(parsed)        return result    except (json.JSONDecodeError, ValueError) as error:        raise ValueError(            f"Invalid AI response: {error}"        ) from errorsample = """{    "service": "nginx",    "port": 80,    "event": "bind_failure"}"""validated = validate_ai_output(sample)

Install:

pip install pydantic

This implementation ensures that extracted information follows your application's expected structure and constraints.

However, schema validation doesn't prove that the extracted facts are correct. You still need grounding, domain-specific checks, and appropriate human review.

Additional reliability practices

  1. Test known-answer questions.
  2. Test ambiguous prompts.
  3. Test incorrect assumptions.
  4. Test whether the model can abstain when information is unavailable.
  5. Repeat identical prompts to measure output variation.
  6. Treat user-provided documents as untrusted input.
  7. Restrict tool permissions and require approval for sensitive actions.

Developer takeaway: Production AI engineering requires testing, validation, observability, and failure handlingβ€”not just prompt engineering.

5. Design AI Applications with Structured Workflows and Cost Controls

A common mistake in AI application development is putting all the logic inside one large prompt.

A better approach is to separate application responsibilities into smaller components.

Consider an AI-powered IT support ticket system.

A user submits:

The VPN connection stopped working after the latest update.

Your application might perform the following operations:

  1. Classify the support ticket.
  2. Extract relevant technical information.
  3. Generate a summary.
  4. Retrieve relevant troubleshooting documentation.
  5. Prepare a response for human review.

These are different AI application patterns.

Pattern Purpose
Generation Create new content
Classification Assign predefined categories
Extraction Convert text into structured data
Summarization Condense information
Embeddings Represent semantic meaning
RAG Retrieve relevant information before generation
Tool calling Connect model decisions to external tools

Practical implementation: Build a validated classification workflow

import httpxOLLAMA_URL = "http://localhost:11434"MODEL = "llama3.2:1b"ALLOWED_LABELS = {    "Network",    "Hardware",    "Software",    "Security"}def classify_ticket(ticket: str) -> str:    prompt = f"""    Classify this IT support ticket into exactly    one category:    Network, Hardware, Software, Security.    Return only the category name.    Ticket: {ticket}    """    response = httpx.post(        f"{OLLAMA_URL}/api/generate",        json={            "model": MODEL,            "prompt": prompt,            "stream": False        },        timeout=120.0    )

Expected output:

Network

The important engineering decision here is that the model proposes a classification, but application code enforces the permitted output.

Don't use an LLM for everything

Sometimes conventional programming is the better solution.

Problem Recommended solution
Addition and arithmetic Python
Database filtering SQL
Authentication Security libraries
Input validation Pydantic
Semantic search Embeddings
Document summarization LLM
Natural-language classification LLM or ML classifier

For example, calculating an invoice total with Python is more reliable than asking an LLM to perform arithmetic.

The objective isn't to introduce AI everywhere. It's to use AI where it provides meaningful value.

Monitor token usage and operational costs

Every production AI application should measure:

  • Input tokens.
  • Output tokens.
  • Request latency.
  • Model and provider.
  • Failure rate.
  • Estimated request cost.

A simple cost-estimation function:

def estimate_cost(    input_tokens: int,    output_tokens: int,    input_price: float,    output_price: float) -> float:    input_cost = (        input_tokens * input_price / 1_000_000    )    output_cost = (        output_tokens * output_price / 1_000_000    )    return input_cost + output_cost

The prices should come from the provider's current pricing documentation and be expressed in the same currency and pricing units.

Developer takeaway: Build AI applications as structured software systems with validation, deterministic logic, monitoring, and cost controls.

Putting Everything Together: Your Applied AI Engineering Checklist

Before moving toward advanced topics such as RAG, AI agents, and production LLMOps, make sure you understand these fundamentals.

  • Understand AI, ML, deep learning, and Generative AI.
  • Identify foundation models, LLMs, VLMs, and embedding models.
  • Understand tokenization and context windows.
  • Distinguish training from inference.
  • Compare hosted and local inference.
  • Understand hallucinations and nondeterminism.
  • Evaluate models using quality, latency, and cost.
  • Implement classification, extraction, and summarization.
  • Validate model outputs using Pydantic.
  • Understand when conventional programming is preferable.
  • Build reusable API integrations.
  • Monitor token consumption and estimated costs.

What's Next?

Understanding AI and LLM fundamentals is only the beginning.

Once you understand these concepts, you can progress toward more advanced AI engineering architectures:

  1. Prompt Engineering: Design better instructions and structured prompts.
  2. Retrieval-Augmented Generation (RAG): Connect LLMs with external knowledge sources.
  3. Vector Databases: Store and retrieve semantic embeddings.
  4. AI Agents: Build systems that interact with tools and execute controlled workflows.
  5. LLMOps: Evaluate, monitor, deploy, and maintain AI applications in production.

These topics build on the fundamentals covered in this article.

Final Thoughts

Building AI-powered applications isn't about sending a prompt to an API and displaying the response.

It's about understanding the underlying models, selecting appropriate inference architectures, managing context and costs, validating generated outputs, and designing systems that remain useful when things go wrong.

The developers who understand these engineering principles will be better equipped to build practical AI applications rather than just experiment with AI tools.

Explore More with DevSpireHub

Want to develop your AI engineering skills step by step?

Explore the complete learning resource:

AI and LLM Foundations: A Complete Guide to Artificial Intelligence and Large Language Models

Bookmark this article

If you found this useful:

  • Bookmark this guide so you can revisit the examples.
  • Comment below: Which AI concept are you currently exploringβ€”embeddings, RAG, local LLMs, or AI agents?
  • Follow DevSpireHub for practical Python tutorials, open-source developer tools, AI engineering guides, and hands-on technical resources.

Explore More Resources

🌐 Website: https://devspirehub.com/

πŸš€ DevSpire Nexus: https://nexus.devspirehub.com/

πŸ’» Portfolio: https://portfolio.devspirehub.com/

πŸ™ GitHub: https://github.com/kumaradoss16

πŸ“Έ Instagram: https://www.instagram.com/devspirehub/

πŸ’Ό LinkedIn: https://www.linkedin.com/in/kumaradoss-s/

Let's learn, build, and engineer better AI applications together!

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.