AI Engineer Roadmap: Building LLM Apps

Learn to build reliable LLM-powered applications: prompting, RAG, agents and evaluation, step by step.

10 milestones · ~170 hours total

1. 1

   Python and API fundamentals

   ~20h

   Comfortable Python, virtual environments, JSON, HTTP clients, async basics and handling API keys safely.

   Practice this →
2. 2

   How LLMs work, at a practical level

   ~10h

   Tokens, context windows, temperature, embeddings and why models hallucinate. Enough theory to debug behaviour.

   Practice this →
3. 3

   Prompt engineering and structured output

   ~12h

   System vs user prompts, few-shot examples, JSON schema outputs and validating responses with Pydantic.

   Practice this →
4. 4

   Calling LLM APIs well

   ~12h

   Streaming, retries, rate limits, timeouts and token cost tracking. Build a small CLI chat tool.

   Practice this →
5. 5

   Embeddings and vector search

   ~14h

   Create embeddings, store them in pgvector or Chroma, and compare cosine similarity and keyword search.

   Practice this →
6. 6

   Retrieval-Augmented Generation (RAG)

   ~25h

   Chunking strategies, retrieval, reranking and grounded answers with citations over your own documents.

   Practice this →
7. 7

   Tool use and agents

   ~22h

   Function calling, a simple agent loop, tool errors and when a plain workflow beats an agent.

   Practice this →
8. 8

   Evaluation and testing

   ~15h

   Build a test set, check retrieval quality and answer correctness, use LLM-as-judge carefully and run regression checks.

   Practice this →
9. 9

   Safety, guardrails and prompt injection

   ~10h

   Input and output validation, treating retrieved text as untrusted, PII handling and permission limits for tools.

   Practice this →
10. 10

    Ship a portfolio project

    ~30h

    Deploy a RAG app with a web UI, logging, an eval suite and a README explaining design trade-offs.

    Practice this →