100: AI Engineering

Chapter Overview

AI Engineering is the discipline of building robust, scalable, and valuable applications on top of pre-existing Foundation Models. This represents a fundamental shift from traditional Machine Learning, where the primary focus was on building and training models from scratch.

AI Engineers focus on the adaptation and application of massive, pre-trained models rather than creating new models from the ground up. This paradigm shift has revolutionized how we approach AI problem-solving.

---

The Evolution: From Traditional ML to AI Engineering

The rise of AI Engineering is driven by two transformative factors:

1. Dramatic leap in model capability - Foundation models can now handle diverse tasks with unprecedented accuracy
2. Simultaneous drop in barriers to entry - Building with AI has become more accessible than ever

Key Distinction

Traditional ML Engineer: "How can I build the best possible model for this specific task (e.g., sentiment analysis)?"

AI Engineer: "Given this incredibly powerful general-purpose model, how can I best adapt it to solve multiple business problems efficiently?"

This distinction is crucial because the focus shifts from model creation to system design, data flow optimization, and user interaction enhancement.

---

The AI Engineer's Decision Framework

When approaching a new challenge, AI Engineers must strategically choose the most appropriate adaptation technique. The key is to start with the simplest, most effective solution and scale complexity only when necessary.

Core Assessment Questions

The framework centers on two fundamental dimensions of model performance:

Behavior Assessment

Does the model fail because it doesn't understand the required style, format, or reasoning process?

Examples: Providing correct answers in prose instead of JSON format, failing to follow complex multi-step instructions, or not maintaining consistent tone.

Information Assessment

Does the model fail because it lacks the necessary knowledge or context?

Examples: Missing company-specific policies, recent events post-training, or private document contents.

The Strategic Decision Path

This assessment leads to a clear, actionable decision tree:

%%{init: {
  'theme': 'base',
  'themeVariables': {
    'primaryColor': '#f4f4f4',
    'primaryTextColor': '#2d3748',
    'primaryBorderColor': '#e2e8f0',
    'lineColor': '#4a5568',
    'secondaryColor': '#edf2f7',
    'tertiaryColor': '#f7fafc',
    'background': '#ffffff',
    'mainBkg': '#ffffff',
    'secondBkg': '#f8fafc',
    'tertiaryBkg': '#edf2f7'
  }
}}%%

flowchart TD
    A[🚀 New AI Challenge] --> B{🎯 Behavior & Style<br/>Assessment}

    B -->|❌ Incorrect behavior| C[🔧 Fine-Tuning<br/>Teach new skills & behaviors]
    B -->|✅ Correct behavior| D{📚 Knowledge & Information<br/>Assessment}

    D -->|❌ Missing information| E[🔍 RAG System<br/>Provide contextual knowledge]
    D -->|✅ Has information| F[💡 Prompt Engineering<br/>Guide with existing capabilities]

    C --> G[🏗️ Production System]
    E --> G
    F --> G

    subgraph toolkit[" 🛠️ The AI Engineer's Toolkit"]
        direction TB
        C
        E  
        F
    end

    classDef startNode fill:#e6f3ff,stroke:#0066cc,stroke-width:3px,color:#003d7a
    classDef decisionNode fill:#fff7e6,stroke:#cc7a00,stroke-width:2px,color:#804d00
    classDef solutionNode fill:#e6ffe6,stroke:#00b359,stroke-width:2px,color:#006633
    classDef endNode fill:#f0e6ff,stroke:#8000ff,stroke-width:2px,color:#4d0080
    classDef toolkitBox fill:#f8f9fa,stroke:#6c757d,stroke-width:2px,stroke-dasharray: 5 5

    class A startNode
    class B,D decisionNode
    class C,E,F solutionNode
    class G endNode
    class toolkit toolkitBox

Solution Strategies Explained

🔧 Fine-Tuning🔍 RAG (Retrieval-Augmented Generation)💡 Prompt Engineering

When to use: The model has the right information but wrong behavior

Purpose: Teach the model new skills, response patterns, or reasoning approaches

Examples: - Training a model to respond in a specific professional tone - Teaching domain-specific reasoning patterns - Adapting output format preferences

Trade-offs: Higher cost and complexity, but provides consistent behavioral changes

When to use: The model has the right behavior but lacks information

Purpose: Supplement the model with external, up-to-date, or private knowledge

Examples: - Accessing company documentation - Incorporating recent news or events - Querying private databases

Trade-offs: Moderate complexity, excellent for dynamic information needs

When to use: The model has both the right behavior and information

Purpose: Guide the model to use its existing capabilities optimally

Examples: - Crafting effective instructions - Providing helpful examples - Structuring context effectively

Trade-offs: Lowest cost and complexity, fastest to implement and iterate

---

Best Practices for AI Engineers

Golden Rules

1. Start Simple: Always begin with prompt engineering before considering more complex solutions
2. Measure Impact: Each technique should solve a specific, measurable problem
3. Consider Costs: Balance solution complexity with business value
4. Think Systems: Design for scalability, maintainability, and monitoring from day one

The framework presented here provides a systematic approach to AI engineering decisions, ensuring that you choose the most appropriate tool for each challenge while maintaining efficiency and effectiveness.

---

Next Steps

Ready to dive deeper? Explore the foundational concepts that make AI Engineering possible: 101-Foundation-Models.