The GenAI Engineer Workbook

An 18-week workbook that takes a working software engineer to shipping GenAI systems they can defend — evaluated, guarded, deployed, and measured.

Most AI courses show you a system being built. You watch someone wire up a RAG demo, it answers a question, and the video ends. Then an interviewer asks how you know it works, what happens when a user tells your agent to ignore its instructions, or what it costs per thousand queries — and the demo has taught you nothing that helps.

This is a workbook instead. You build one assistant across nine phases and nine workshops, and each phase adds the thing the demos skip: a golden set and a judge you calibrated yourself, a leash on the agent that lives in code rather than in the prompt, a memory you can invalidate, a red-team suite in CI, a deploy with a rollback path, and a P99 budget you can defend. You finish with one system you can explain end to end, and numbers to quote about it.

---

Before you start: what you need

Skills. This is not a first programming course, and it is not an introduction to Python — it assumes you can build software and teaches you to build this kind of software. You do not need a machine-learning background, and there is no maths derivation anywhere in here.

Required — assumed on day one.

• Python (comfortable) — type hints, async/await, Pydantic, uv or poetry, pytest
• APIs & HTTP — verbs, status codes, API-key auth, JSON, SSE/streaming, retry with backoff
• Git/GitHub — branching, PRs, code review
• Docker basics — Dockerfile, docker compose, multi-stage builds

Helpful, not required — each is either taught here or has a stated way around it.

• A cloud — any of AWS/GCP/Azure. Phase 8 deploys with compose on one box; a cloud makes the last mile familiar rather than possible
• SQL — joins and indexes make pgvector feel familiar, but every query the course writes is shown in full
• Design patterns — adapter/strategy and dependency injection carry this whole course; you can also just read them off the lessons that use them
• Hardware — any 16GB+ machine runs the local-model lessons; more RAM unlocks stronger models (sizing table in Phase 1). No GPU at all? Hosted budget tiers cover everything.
• Light math — vectors, cosine similarity, probability intuition. No PhD required, promise

The workbook's opening screen carries the same two lists as a self-check, and pnpm check-claims fails if this copy drifts from app/src/data/intro.ts.

Scope — what this course does not teach. It is about building systems on top of models. Six areas are deliberately out of scope, and knowing that now beats inferring it from an absence in week seventeen:

Not covered	Why not	Where to go instead
Transformer mathematics and model architecture	You will use attention daily and never implement it. A model here is a component with a latency, a price and a failure mode — the right abstraction for building on one, the wrong one for changing one	Karpathy's Let's build GPT, The Illustrated Transformer
Pretraining and distributed training	Nothing here trains a base model. Trillion-token pipelines, sharding, and keeping a thousand GPUs busy are a specialty with their own hiring loop	The Llama and OLMo technical reports; the Ultra-Scale Playbook
Fine-tuning and alignment research	The electives cover when to fine-tune and how LoRA fits a workflow — not RLHF, DPO, reward modelling, or what alignment means. The default here is that retrieval and prompting solve most of what people fine-tune to fix	The InstructGPT and DPO papers; the HF alignment handbook
GPU kernels, quantization, serving at scale	You will run local models and reason about tokens per second and RAM. You will not write CUDA or tune a vLLM cluster; the elective runs vLLM, it does not operate it under load	The vLLM and FlashAttention papers; llama.cpp quantization docs
Multimodal depth	Vision and audio appear as electives and as capabilities you call, not as things you train or evaluate deeply	The CLIP and Whisper papers
Research methodology	You learn to measure a system you built — not to design a study, pick a defensible baseline, or reason about significance across seeds	How to Read a Paper (Keshav), then reproduce one result end to end

None of the above is a prerequisite for this job. All of it is a prerequisite for a different one, and if that is the job you want, the right-hand column is a better use of your next eighteen weeks than this workbook.

Hardware. Four honest tiers — pick yours and nothing you learn changes:

Tier	What runs	What to know
Any machine	Every lesson's fast test suite (make test)	Offline, deterministic, no models, no keys — the whole course can be completed here
16 GB	The course's working models locally: qwen3.5:9b, gemma4:e4b, embeddings, guard models	The recommended local path. The 30B eval judge does not fit here — swap in a smaller judge or a hosted one
32–64 GB	+ qwen3-coder:30b as a free local Phase 3 judge	The comfortable path; bigger judges are measurably better
No GPU / older laptop	Everything, against a hosted budget tier by changing one base_url	Needs an account, an API key, and network; costs real (small) money

Phase 1 carries the full sizing table, and both come from one place: app/src/data/reference.ts. pnpm check-claims fails if this copy drifts from it.

Money. The fast test tier of every lesson runs offline with zero API keys — that is a design constraint of this course, not an accident. Running models locally is free. The hosted fallback and the optional "see the difference" comparisons against frontier providers need an account and cost real money — small, metered, and entirely optional.

Software. Python 3.11 through 3.14, uv, Ollama installed on your own machine, and Docker for the Phase 8 deployment lessons. Install Ollama on the host rather than in a container: Docker Desktop gives containers no GPU, and the difference on the course's own 9B is 0.52 tokens per second against 81. Phase 8's stack runs its infrastructure in compose and reaches your Ollama at host.docker.internal:11434. That range is a range, not a floor: every lesson declares requires-python = ">=3.11,<3.15" and CI runs the whole set at both ends on every push, because an unbounded 3.11+ is a claim about versions that did not exist when it was written. One lesson — 4.4, the framework bakeoff — pins 3.12 exactly, because CrewAI's dependency tree does not build on anything newer. It declares that itself and uv fetches the interpreter; you do not need 3.12 for anything else.

---

Start here

# 1. Open the workbook — the course.html that came with the release. It is one
#    self-contained file: no server, no build, no network.
open course.html

# 2. Install the toolchain and pull the models the course uses.
#    Ollama goes on this machine, not in a container — it needs the GPU.
curl -LsSf https://astral.sh/uv/install.sh | sh
ollama pull qwen3.5:9b        # chat + tool calling
ollama pull nomic-embed-text  # embeddings

# 3. Do the first lesson.
cd src/phase1-foundations/01-universal-client/before
make setup && make test       # red: the TODOs are yours to fill in
$EDITOR src/client.py
make check                    # green: lint, types and tests

Two more models (qwen3-coder:30b for the Phase 3 judge, llama-guard3:8b for the Phase 6 guardrails) are worth pulling when you reach those phases rather than now.

The workbook is a build output, not a file in this repo. If you cloned instead of downloading the release, build it yourself — that needs Node, which nothing else in the course does: cd app && pnpm install, then ./package.sh from the repo root. That builds all three artifacts from one commit and stamps dist/BUILD.json with it. dist/ is deliberately not in the repository — it is reproducible from any commit, and committing a megabyte of minified output per release would grow every clone forever. If you are the one publishing, run ./verify-dist.sh first: it is what catches a dist/ you built, kept working past, and were about to upload anyway. It checks the archive member by member — every file's own sha256, recorded at package time and compared against both the zip and git archive HEAD — so a lesson that is present under the right name but carries the wrong bytes fails there rather than in a student's terminal. It also reads the stamp's own metadata: schema version, release version and where that number came from, build time, and the toolchain that produced the artifact.

Publishing itself is a tag: pushing v* runs .github/workflows/release.yml, which builds with ./package.sh, gates on ./verify-dist.sh, checks the stamp names the tagged commit, and attaches the three artifacts plus BUILD.json to the GitHub release. Running it by hand still works and is the same code; the workflow exists because the step people skip under time pressure is the verification, not the build.

Your progress in course.html is saved in the browser's localStorage, so tick things off as you go — but keep in mind it lives in that one browser on that one machine. Three buttons at the bottom of the sidebar are your insurance against that: Export writes a small JSON file, Import reads one back on another machine, and Reset clears everything after a confirmation. Export before you clear a cache, change laptops, or try a different browser; nothing else in the workbook can recover a wiped localStorage.

The dashboard also has a completion manifest: it reconciles what you have ticked with evidence/manifest.json from make evidence, and writes a COMPLETION.md that says which of the two it is. Ticked boxes alone come out as self-reported — deliberately, and no amount of ticking changes that.

---

How the lessons work

Every exercise ships twice. before/ is a runnable scaffold with the judgement removed and TODOs where your work goes; its tests fail on purpose, and turning them green is the exercise. after/ is a working reference.

Attempt before you read. The order is the method, not a suggestion: write your attempt, run the tests, and open after/ only once your version passes or you are genuinely stuck — then diff it against what you wrote, because the diff is where the lesson is. Reading a working solution feels like learning and mostly is not. The struggle you skip is the part that makes it stick, and once you have read a solution you can no longer tell, from the inside, whether you could have written it.

That pairing is the whole pedagogical spine: each phase moves you along a worked → faded → independent ladder, and every phase ends with at least one task that hands you a blank editor and no scaffold at all. If you only ever fill in blanks, you will learn to recognise good code rather than write it, which is the failure mode this course is built to avoid.

Tests come in two tiers: make test is fast, offline and deterministic, and make test-integration runs the same code against real models. Full mechanics — the make targets, the library stack, version pinning — are in src/README.md.

---

The nine phases

Each phase runs about two weeks and ends in a workshop that adds a layer to the same assistant. Workshops 1 and 9 bookend it: the tool you measure with, and the loop that turns everything into offers.

#	Phase	You build	Afterwards you can prove
1	Speak Fluent LLM	One client across every provider; a RAG pipeline by hand	You bill from usage, not estimates, and every later stage has a name you know
2	Retrieval That Actually Works	Hybrid search + reranker, behind an offline gate	Retrieval choices backed by numbers, and stage-by-stage debugging under pressure
3	Prove It Works: Evals & Judges	Golden set, calibrated judge, CI merge gate	"It works" — with a κ against your own labels and a gate that blocks your merges
4	Agents on a Leash	The reason–act–observe loop from scratch, then frameworks	An agent contained in code: step caps, deadlines, approval on anything irreversible
5	Agents That Remember & Collaborate	Memory with provenance and TTL; a delegating crew	A fact recalled, cited, and correctly forgotten — with the cost of delegation measured
6	Whiteboard It & Defend It	The 8-step design script; layered guardrails	A system designed out loud, and an injection that lands but cannot fire a gated tool
7	MCP: The Universal Tool Port	Your own MCP server, consumed by your assistant	Tools that any agent can use, with the right auth for each deployment
8	Run It in Production	Containers, CI, OpenTelemetry, cache and routing	A deployed stack, gated on evals and red-team, defended on P99 rather than an average
9	The GenAI Mindset	Drill deck, metric-mined resume, funnel tracker	Every claim traced to a file you wrote, and a job search you can debug

The full lesson-by-lesson map is in src/README.md, and there is an electives shelf in the app — fine-tuning, multimodal, GraphRAG, GPU serving — for topics you should only pick up once they appear in three job descriptions you actually want.

---

Gates, not dates

The week numbers are a shape, not a schedule. What decides whether you move on is the milestone at the end of each phase, and the milestones are written as things that either happen or do not: the service answers with citations and abstains when it cannot; faithfulness clears 0.85 on a 50-question golden set; no landed injection fires a gated tool.

Miss one and the honest move is to stay — the fix-it playbook for each is part of the syllabus rather than an appendix. The workbook tracks all of this for you; the dashboard in course.html is the checklist.

Reading the phase takes minutes and the app tells you how many. Doing it takes the fortnight. Those are very different numbers and the course is careful never to blur them.

---

Where to get unstuck

1. Re-read the failing test. The before/ tests are written to describe the shape of the answer, not just to fail.
2. Check the phase's VERIFIED.md. Each phase carries a dated stamp saying when its lessons last passed, and — where it recorded one — the exact version that run resolved to. Most record the date and the declared ranges instead: the lessons are version-bounded, not locked, and only the capstone ships a lockfile. GenAI dependencies break fast; if that date is old, expect drift, and upgrade one dependency at a time.
3. Open after/. It is a reference, not a cheat — but read it, then close it and write your own.
4. Run ./src/verify-lessons.sh if something looks broken in the repo rather than in your code. It checks every lesson in the course and tells you which one is unhappy.

---

What is in this repo

• src/ — the companion code: every before/+after/ lesson pair and the nine workshop briefs. This is the part you work in.
• app/ — the source of the course app. Only interesting if you want to change the course; students never need it. React and TypeScript, building to the single course.html, with three gates (alignment, integrity, density) that content has to pass before it can ship.
• release/ — what goes in the release next to the workbook. Currently the student-facing README that ./package.sh copies into dist/.

License

MIT — use it, fork it, teach from it.