rohitg00/ai-engineering-from-scratch

Name: rohitg00/ai-engineering-from-scratch
Author: rohitg00

Learn it. Build it. Ship it for others.

51,8278,976Pythonagentsaiai-agentsai-engineeringcomputer-visioncoursedeep-learningfrom-scratchgenerative-aillmmachine-learningmcpnlppythonreinforcement-learningrustswarm-intelligencetransformerstutorialtypescriptUpdated 3d ago

README

From the creator of Agent Memory - #1 Persistent memory ⭐ which naturally works with any agents or chat assistants.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

84% of students already use AI tools. Only 18% feel prepared to use them professionally. This curriculum closes that gap.

523 lessons. 20 phases. ~342 hours. Python, TypeScript, Rust, Julia. Every lesson ships a reusable artifact: a prompt, a skill, an agent, an MCP server. Free, open source, MIT.

You don't just learn AI. You build it. End-to-end. By hand.

Start here: choose what you want to build

You do not need to scan 523 lessons before beginning. Pick one goal. Each link opens the same curriculum on GitHub or the website, and both versions use the same lesson code.

| Your goal | Learn on GitHub | Learn on the website | |---|---|---| | I am new and want the complete foundation | Phase 0: Setup and Tooling | Dev Environment | | I know Python and want math plus ML foundations | Phase 1: Math Foundations | Linear Algebra Intuition | | I want to build production LLM applications | Phase 11: LLM Engineering | Prompt Engineering | | I want to build agents | Phase 14: Agent Engineering | The Agent Loop | | I want to use coding agents on real repositories | Agent-Assisted Engineering path | Agent-Assisted Engineering | | I want to shape the right build before implementation | Product Judgment and Delivery path | Product Judgment and Delivery | | I want to build with Model Context Protocol (MCP) | Model Context Protocol (MCP) route | Model Context Protocol (MCP) path | | I want to write and ship Agent Skills | Focused Agent Skills route | Agent Skills path | | I want to prepare for a Claude certification | Certification onboarding | Certification Academy |

Not sure where you fit? Use the start-learning placement tutor or the website prerequisites guide.

Compare four core domains and six career routes in the AI Engineering Learning Paths.

Use every lesson the same way

Read docs/en.md and explain the core idea in your own words.
Type and build the important code instead of treating the code block as decoration.
Run the lesson command from the repository root, the directory containing README.md and phases/.
Keep evidence: the command, working directory, exit code, meaningful output, and the artifact you changed or produced.
Continue only when you can explain the output and make one small change without guessing.

Commands in lesson pages are paths from the repository root unless the lesson explicitly says to change directories. If a lesson offers several languages, run the implementation for the language you are learning.

Clone it and produce your first evidence

git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python3 phases/00-setup-and-tooling/01-dev-environment/code/verify.py --route beginner
python3 phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py

The preflight separates requirements needed now from tools needed later. Every required failure includes the detected reason and a corrective command. The second command is a dependency-free lesson and ends by showing that a matrix times a vector is the operation inside a neural network layer. Save that terminal output as your first evidence.

Add the AI tutor in 30 seconds

If Node.js, npx, and a skill-capable coding agent are already installed, your coding agent can become your tutor in two commands. A repository clone is not needed to install or read the tutor. Runnable focused-path labs need python3. Agent Skills host labs also need a selected host and a writable user or project skill scope.

Check the local requirements first:

node --version
npx --version
python3 --version

Then install the curriculum skills and choose the host and scope you intend to use when the installer asks:

npx skills add rohitg00/ai-engineering-from-scratch

Invocation syntax belongs to the host, not to the portable SKILL.md format:

| Host | Start the course | Start Model Context Protocol (MCP) | Start Agent Skills | Run a phase quiz | |---|---|---|---|---| | Codex | start-learning, or choose it from /skills | learn-mcp, or choose it from /skills | learn-agent-skills, or choose it from /skills | check-understanding 13, or choose it from /skills | | Claude Code | /start-learning | /learn-mcp | /learn-agent-skills | /check-understanding 13 | | Other compatible hosts | Use start-learning to begin the course. | Use learn-mcp to start the Model Context Protocol (MCP) path. | Use learn-agent-skills to start the Agent Skills Engineering path. | Use check-understanding to quiz me on Phase 13. |

A ten-question placement quiz maps what you already know to a starting phase and saves a personalized study plan to LEARNING.md. From there, the learn skill teaches one lesson per session: concept, math, code, quiz. It streams lessons straight from this repo, and the course-guide skill jumps you to the exact lesson that covers anything you are stuck on. In Codex, invoke these skills with learn and course-guide; in Claude Code, use /learn and /course-guide; in other compatible hosts, ask to use the skill by name.

Only want Model Context Protocol (MCP)? Use the MCP invocation for your host. It creates MCP-LEARNING.md and follows one 17-lesson route through stateless requests, transports, bidirectional work, security, reliability, registry governance, and conformance evidence. The exact order and checkpoints live in the Model Context Protocol (MCP) manifest.

Only want Agent Skills? Use the Agent Skills invocation for your host. It creates AGENT-SKILLS-LEARNING.md and follows one coherent five-lesson route: contract, discovery, invocation, sandbox boundaries, then release evals and real-host portability. Start on the web with the Agent Skills path.

The installer lists the hosts it can configure and asks where to install. If you do not have Node.js, npx, python3, a supported host, or a writable scope yet, use the website or read docs/en.md manually. That path teaches the concepts, but real-host discovery, invocation, script, and uninstall evidence remains pending until the preflight is available. Read the lessons at aiengineeringfromscratch.com.

How this works

Most AI material teaches in scattered pieces. A paper here, a fine-tuning post there, a flashy agent demo somewhere else. The pieces rarely line up. You ship a chatbot but can't explain its loss curve. You hook a function to an agent but can't say what attention does inside the model that's calling it.

This curriculum is the spine. 20 phases, 523 lessons, four languages: Python, TypeScript, Rust, Julia. Linear algebra at one end, autonomous swarms at the other. Every algorithm gets built from raw math first. Backprop. Tokenizer. Attention. Agent loop. By the time PyTorch shows up, you already know what it's doing under the hood.

Each lesson runs the same loop: read the problem, derive the math, write the code, run the test, keep the artifact. No five-minute videos, no copy-paste deploys, no hand-holding. Free, open source, and built to run on your own laptop.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

The shape of the curriculum

Twenty phases stack on top of each other. Math is the floor. Agents and production are the roof. Skip ahead if you already know the lower layers, but don't skip and then wonder why something at the top is breaking.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#fafaf5','primaryTextColor':'#1a1a1a','primaryBorderColor':'#3553ff','lineColor':'#3553ff','fontFamily':'JetBrains Mono','fontSize':'12px'}}}%%
flowchart TB
  P0["Phase 0 — Setup &amp; Tooling"] --> P1["Phase 1 — Math Foundations"]
  P1 --> P2["Phase 2 — ML Fundamentals"]
  P2 --> P3["Phase 3 — Deep Learning Core"]
  P3 --> P4["Phase 4 — Vision"]
  P3 --> P5["Phase 5 — NLP"]
  P3 --> P6["Phase 6 — Speech &amp; Audio"]
  P3 --> P9["Phase 9 — RL"]
  P5 --> P7["Phase 7 — Transformers"]
  P7 --> P8["Phase 8 — GenAI"]
  P7 --> P10["Phase 10 — LLMs from Scratch"]
  P10 --> P11["Phase 11 — LLM Engineering"]
  P10 --> P12["Phase 12 — Multimodal"]
  P11 --> P13["Phase 13 — Tools &amp; Protocols"]
  P13 --> P14["Phase 14 — Agent Engineering"]
  P14 --> P15["Phase 15 — Autonomous Systems"]
  P15 --> P16["Phase 16 — Multi-Agent &amp; Swarms"]
  P14 --> P17["Phase 17 — Infrastructure &amp; Production"]
  P15 --> P18["Phase 18 — Ethics &amp; Alignment"]
  P16 --> P19["Phase 19 — Capstone Projects"]
  P17 --> P19
  P18 --> P19

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

The shape of a lesson

Each lesson lives in its own folder, with the same structure across the entire curriculum:

phases/<NN>-<phase-name>/<NN>-<lesson-name>/
├── code/      runnable implementations (Python, TypeScript, Rust, Julia)
├── docs/
│   └── en.md  lesson narrative
└── outputs/   prompts, skills, agents, or MCP servers this lesson produces

Every lesson follows six beats. The Build It / Use It split is the spine — you implement the algorithm from scratch first, then run the same thing through the production library. You understand what the framework is doing because you wrote the smaller version yourself.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#fafaf5','primaryTextColor':'#1a1a1a','primaryBorderColor':'#3553ff','lineColor':'#3553ff','fontFamily':'JetBrains Mono','fontSize':'13px'}}}%%
flowchart LR
  M["MOTTO<br/><sub>one-line core idea</sub>"] --> Pr["PROBLEM<br/><sub>concrete pain</sub>"]
  Pr --> C["CONCEPT<br/><sub>diagrams &amp; intuition</sub>"]
  C --> B["BUILD IT<br/><sub>raw math, no frameworks</sub>"]
  B --> U["USE IT<br/><sub>same thing in PyTorch / sklearn</sub>"]
  U --> S["SHIP IT<br/><sub>prompt · skill · agent · MCP</sub>"]

Getting started

Three ways in. Pick one.

Option A — learn in your terminal (recommended). After the Node.js, npx, host, and scope preflight above, install the learning skills into a compatible agent and let the course drive itself:

npx skills add rohitg00/ai-engineering-from-scratch

Use the host-specific invocation table above. The installed skills provide start-learning, learn, course-guide, and the focused learn-mcp and learn-agent-skills routes. Lesson prose can stream from this repository without a clone. A local clone is required for copied repository code commands and executable MCP or Agent Skills labs. Progress lives in LEARNING.md, MCP-LEARNING.md, or AGENT-SKILLS-LEARNING.md in your project, so every session can resume.

Option B — read. Open any completed lesson on aiengineeringfromscratch.com or expand a phase under Contents. No setup, no cloning.

Option C — clone and run.

git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python3 phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py

Cloning also auto-loads the learning skills in Claude Code, and gives every lesson's code to the learn tutor for real execution instead of read-along.

Prerequisites

You can write code (any language; Python helps).
You want to understand how AI actually works, not just call APIs.

Prepare for Claude certifications

The Claude Certification Academy is a free, open-source preparation program for all four official Claude certification tracks: Associate Foundations, Developer Foundations, Architect Foundations, and Architect Professional. Each route combines blueprint-mapped lessons, runnable labs, a diagnostic, capstone work, and a full-length original practice exam.

Use the AI-native GitHub onboarding guide with Claude Code, Codex, ChatGPT, Cursor, or another agent. Run claude-certification in Codex, /claude-certification in Claude Code, or ask another host to use claude-certification. It chooses a track, creates a persistent route in CLAUDE-CERTIFICATION.md, teaches one step at a time, runs the real labs, and gives artifact-based feedback. The same curriculum remains available on the certification website.

The academy is independent study material based on public exam objectives. It is not affiliated with Anthropic, does not reproduce live exam questions, and cannot guarantee a passing score.

The learning skills

| Skill | What it does | |---|---| | start-learning | One-time onboarding: why you're learning, placement quiz, personalized plan saved to LEARNING.md. | | learn | The tutor loop. Warm-up recall, then the next lesson taught interactively, then its quiz; records progress and a review queue. | | course-guide | Topic router. "Where do I learn attention?" or "my loss is NaN" → the exact lessons, with links. | | learn-mcp | Focused Model Context Protocol (MCP) tutor. Creates MCP-LEARNING.md, follows the 17-lesson manifest, and records wire, security, reliability, and conformance evidence. | | learn-agent-skills | Focused Agent Skills tutor. Creates AGENT-SKILLS-LEARNING.md, teaches lessons 22, 24, 25, 26, and 27, and records real-host evidence. | | claude-certification | Certification tutor. Chooses CCAO-F, CCDV-F, CCAR-F, or CCAR-P; teaches each lesson; runs labs; reviews artifacts; administers diagnostics and mocks; saves progress. | | find-your-level | Ten-question placement quiz. Maps your knowledge to a starting phase and produces a personalized path with hour estimates. | | check-understanding <phase> | Per-phase quiz, eight questions, with feedback and specific lessons to review. Use the Codex, Claude Code, or natural-language form in the invocation table above. |

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Read the core curriculum as a book

The 20-phase core curriculum under phases/ compiles into a six-volume book series. EPUB and PDF are built by CI from the same core lesson sources and attached to every GitHub release; the links below always resolve to the newest release. Volume numbers index the series, not versions: each copy carries a dated edition stamp, and older editions stay downloadable from their release.

Certification curricula are intentionally not converted into the books. Their AI tutor state, runnable labs, interactive figures, diagnostics, and timed mocks remain first-class on GitHub and the website.

| Vol | Title | Phases | Download | |-----|-------|--------|----------| | 1 | Foundations · Math, Tooling, and Classical Machine Learning | 00-02 | EPUB · PDF | | 2 | Deep Learning · Networks, Vision, and Speech | 03, 04, 06 | EPUB · PDF | | 3 | Language · NLP Foundations and the Transformer | 05, 07 | EPUB · PDF | | 4 | Large Language Models · Generation, Reinforcement, Pretraining, and Engineering | 08-11 | EPUB · PDF | | 5 | Agents · Multimodality, Protocols, Autonomy, and Swarms | 12-16 | EPUB · PDF | | 6 | Production · Infrastructure, Safety, and Capstones | 17-19 | EPUB · PDF |

The book is the snapshot; this repository is the living edition. Every chapter ends with links back to the lesson's animated figures, quiz, and runnable code. Build locally with python3 scripts/build_book.py (pandoc required); pipeline details in book/README.md.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Every lesson ships something

Other curricula end with "congratulations, you learned X." Each lesson here ends with a reusable tool you can install or paste into your daily workflow.

Install the lot with python3 scripts/install_skills.py <target>. Real tools, not homework. By the end of the curriculum, you have a portfolio of 523 artifacts you actually understand because you built them.

FIG_002 · A worked sample

Phase 14, lesson 1: the agent loop. ~120 lines of pure Python, no dependencies.

code/agent_loop.py build it

def run(query, tools):
    history = [user(query)]
    for step in range(MAX_STEPS):
        msg = llm(history)
        if msg.tool_calls:
            for call in msg.tool_calls:
                result = tools[call.name](**call.args)
                history.append(tool_result(call.id, result))
            continue
        return msg.content
    raise StepLimitExceeded

outputs/skill-agent-loop.md ship it

---
name: agent-loop
description: ReAct-style loop for any tool list
phase: 14
lesson: 01
---

Implement a minimal agent loop that...

outputs/prompt-debug-agent.md

You are an agent debugger. Given the trace
of an agent run, identify the step where
the agent went wrong and explain why...

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Twenty phases. Click any phase to expand its lesson list.

Phase 0: Setup & Tooling `12 lessons`

Get your environment ready for everything that follows.

| # | Lesson | Type | Lang | |:---:|--------|:----:|------| | 01 | Dev Environment | Build | Python | | 02 | Git & Collaboration | Learn | — | | 03 | GPU Setup & Cloud | Build | Python | | 04 | APIs & Keys | Build | Python | | 05 | Jupyter Notebooks | Build | Python | | 06 | Python Environments | Build | Shell | | 07 | Docker for AI | Build | Docker | | 08 | Editor Setup | Build | — | | 09 | Data Management | Build | Python | | 10 | Terminal & Shell | Learn | — | | 11 | Linux for AI | Learn | — | | 12 | Debugging & Profiling | Build | Python |

| # | Lesson | Type | Lang | |:---:|--------|:----:|------| | 01 | Linear Algebra Intuition | Learn | Python, Julia | | 02 | Vectors, Matrices & Operations | Build | Python, Julia | | 03 | Matrix Transformations & Eigenvalues | Build | Python, Julia | | 04 | Calculus for ML: Derivatives & Gradients | Learn | Python | | 05 | Chain Rule & Automatic Differentiation | Build | Python | | 06 | Probability & Distributions | Learn | Python | | 07 | Bayes' Theorem & Statistical Thinking | Build | Python | | 08 | Optimization: Gradient Descent Family | Build | Python | | 09 | Information Theory: Entropy, KL Divergence | Learn | Python | | 10 | Dimensionality Reduction: PCA, t-SNE, UMAP | Build | Python | | 11 | Singular Value Decomposition | Build | Python, Julia | | 12 | Tensor Operations | Build | Python | | 13 | Numerical Stability | Build | Python | | 14 | Norms & Distances | Build | Python | | 15 | Statistics for ML | Build | Python | | 16 | Sampling Methods | Build | Python | | 17 | Linear Systems | Build | Python | | 18 | Convex Optimization | Build | Python | | 19 | Complex Numbers for AI | Learn | Python | | 20 | The Fourier Transform | Build | Python | | 21 | Graph Theory for ML | Build | Python | | 22 | Stochastic Processes | Learn | Python |

| # | Lesson | Type | Lang | |:---:|--------|:----:|------| | 01 | What Is Machine Learning | Learn | Python | | 02 | Linear Regression from Scratch | Build | Python | | 03 | Logistic Regression & Classification | Build | Python | | 04 | Decision Trees & Random Forests | Build | Python | | 05 | Support Vector Machines | Build | Python | | 06 | KNN & Distance Metrics | Build | Python | | 07 | Unsupervised Learning: K-Means, DBSCAN | Build | Python | | 08 | Feature Engineering & Selection | Build | Python | | 09 | Model Evaluation: Metrics, Cross-Validation | Build | Python | | 10 | Bias, Variance & the Learning Curve | Learn | Python | | 11 | Ensemble Methods: Boosting, Bagging, Stacking | Build | Python | | 12 | Hyperparameter Tuning | Build | Python | | 13 | ML Pipelines & Experiment Tracking | Build | Python | | 14 | Naive Bayes | Build | Python | | 15 | Time Series Fundamentals | Build | Python | | 16 | Anomaly Detection | Build | Python | | 17 | Handling Imbalanced Data | Build | Python | | 18 | Feature Selection | Build | Python |

| # | Lesson | Type | Lang | |:---:|--------|:----:|------| | 01 | Generative Models: Taxonomy & History | Learn | Python | | 02 | Autoencoders & VAE | Build | Python | | 03 | GANs: Generator vs Discriminator | Build | Python | | 04 | Conditional GANs & Pix2Pix | Build | Python | | 05 | StyleGAN | Build | Python | | 06 | Diffusion Models — DDPM from Scratch | Build | Python | | 07 | Latent Diffusion & Stable Diffusion | Build | Python | | 08 | ControlNet, LoRA & Conditioning | Build | Python | | 09 | Inpainting, Outpainting & Editing | Build | Python | | 10 | Video Generation | Build | Python | | 11 | Audio Generation | Build | Python | | 12 | 3D Generation | Build | Python | | 13 | Flow Matching & Rectified Flows | Build | Python | | 14 | Evaluation: FID, CLIP Score | Build | Python | | 19 | Visual Autoregressive Modeling (VAR): Next-Scale Prediction | Build | Python |

| # | Lesson | Type | Lang | |:---:|--------|:----:|------| | 01 | MDPs, States, Actions & Rewards | Learn | Python | | 02 | Dynamic Programming | Build | Python | | 03 | Monte Carlo Methods | Build | Python | | 04 | Q-Learning, SARSA | Build | Python | | 05 | Deep Q-Networks (DQN) | Build | Python | | 06 | Policy Gradients — REINFORCE | Build | Python | | 07 | Actor-Critic — A2C, A3C | Build | Python | | 08 | PPO | Build | Python | | 09 | Reward Modeling & RLHF | Build | Python | | 10 | Multi-Agent RL | Build | Python | | 11 | Sim-to-Real Transfer | Build | Python | | 12 | RL for Games | Build | Python |

| # | Lesson | Type | Lang | |:---:|--------|:----:|------| | 01 | Prompt Engineering: Techniques & Patterns | Build | Python | | 02 | Few-Shot, CoT, Tree-of-Thought | Build | Python | | 03 | Structured Outputs | Build | Python | | 04 | Embeddings & Vector Representations | Build | Python | | 05 | Context Engineering | Build | Python | | 06 | RAG: Retrieval-Augmented Generation | Build | Python | | 07 | Advanced RAG: Chunking, Reranking | Build | Python | | 08 | Fine-Tuning with LoRA & QLoRA | Build | Python | | 09 | Function Calling & Tool Use | Build | Python | | 10 | Evaluation & Testing | Build | Python | | 11 | Caching, Rate Limiting & Cost | Build | Python | | 12 | Guardrails & Safety | Build | Python | | 13 | Building a Production LLM App | Build | Python | | 14 | Model Context Protocol (MCP) | Build | Python | | 15 | Prompt Caching & Context Caching | Build | Python | | 16 | Agent State Machines — Graphs, Nodes, Checkpoints | Build | Python | | 17 | Agent Framework Tradeoffs | Learn | Python |

Lessons 06-18 and 28-31 form the focused Model Context Protocol (MCP) path. Its manifest order is 06, 07, 08, 09, 10, 11, 12, 13, 14, 15, 16, 18, 17, 28, 29, 30, 31. Start it with the host-specific learn-mcp invocation above. Lesson 23 is its only optional capstone and also requires Lessons 19 and 20.

Lessons 22 and 24-27 form the focused Agent Skills learning path, from package contract through real-host release gates. Start it with the host-specific learn-agent-skills invocation shown above; do not follow numeric next navigation from 22 to 23.

Each Phase 14 workbench lesson (31-42) ships a mission.md briefing the agent before it opens the full lesson docs.

Lessons 31-46 form the Agent-Assisted Engineering path. Its manifest order combines the workbench foundation with task framing, planning, delegation, and durable feedback. Lessons 47-54 form the Product Judgment and Delivery path, from outcome framing through evidence, risk, scope, measurement, staged release, and feedback ownership.

| # | Project | Combines | Lang | |:---:|---------|----------|------| | 01 | Terminal-Native Coding Agent | P0 P5 P7 P10 P11 P13 P14 P15 P17 P18 | Python | | 02 | RAG over Codebase (Cross-Repo Semantic Search) | P5 P7 P11 P13 P17 | Python | | 03 | Real-Time Voice Assistant (ASR → LLM → TTS) | P6 P7 P11 P13 P14 P17 | Python | | 04 | Multimodal Document QA (Vision-First) | P4 P5 P7 P11 P12 P17 | Python | | 05 | Autonomous Research Agent (AI-Scientist Class) | P0 P2 P3 P7 P10 P14 P15 P16 P18 | Python | | 06 | DevOps Troubleshooting Agent for Kubernetes | P11 P13 P14 P15 P17 P18 | Python | | 07 | End-to-End Fine-Tuning Pipeline | P2 P3 P7 P10 P11 P17 P18 | Python | | 08 | Production RAG Chatbot (Regulated Vertical) | P5 P7 P11 P12 P17 P18 | Python | | 09 | Code Migration Agent (Repo-Level Upgrade) | P5 P7 P11 P13 P14 P15 P17 | Python | | 10 | Multi-Agent Software Engineering Team | P11 P13 P14 P15 P16 P17 | Python | | 11 | LLM Observability & Eval Dashboard | P11 P13 P17 P18 | Python | | 12 | Video Understanding Pipeline (Scene → QA) | P4 P6 P7 P11 P12 P17 | Python | | 13 | Stateless MCP Server with Registry and Governance | P11 P13 P14 P17 P18 | Python | | 14 | Speculative-Decoding Inference Server | P3 P7 P10 P17 | Python | | 15 | Constitutional Safety Harness + Red-Team Range | P10 P11 P13 P14 P18 | Python | | 16 | GitHub Issue-to-PR Autonomous Agent | P11 P13 P14 P15 P17 | Python | | 17 | Personal AI Tutor (Adaptive, Multimodal) | P5 P6 P11 P12 P14 P17 P18 | Python |

Deep-build tracks — multi-lesson series that build a complete subsystem from scratch.

| # | Project | Combines | Lang | |:---:|---------|----------|------| | 20 | Agent Harness Loop Contract | A. Agent harness | Python | | 21 | Tool Registry with Schema Validation | A. Agent harness | Python | | 22 | JSON-RPC 2.0 Over Newline-Delimited Stdio | A. Agent harness | Python | | 23 | Function Call Dispatcher | A. Agent harness | Python | | 24 | Plan-Execute Control Flow | A. Agent harness | Python | | 25 | Verification Gates and Observation Budget | A. Agent harness | Python | | 26 | Sandbox Runner with Denylist and Path Jail | A. Agent harness | Python | | 27 | Eval Harness with Fixture Tasks | A. Agent harness | Python | | 28 | Observability with OTel GenAI Spans and Prometheus Metrics | A. Agent harness | Python | | 29 | End-to-End Coding Agent on the Harness | A. Agent harness | Python | | 30 | BPE Tokenizer From Scratch | B. NLP LLM | Python | | 31 | Tokenized Dataset with Sliding Window | B. NLP LLM | Python | | 32 | Token and Positional Embeddings | B. NLP LLM | Python | | 33 | Multi-Head Self-Attention | B. NLP LLM | Python | | 34 | Transformer Block from Scratch | B. NLP LLM | Python | | 35 | GPT Model Assembly | B. NLP LLM | Python | | 36 | Training Loop and Evaluation | B. NLP LLM | Python | | 37 | Loading Pretrained Weights | B. NLP LLM | Python | | 38 | Classifier Fine-Tuning by Head Swap | B. NLP LLM | Python | | 39 | Instruction Tuning by Supervised Fine-Tuning | B. NLP LLM | Python | | 40 | Direct Preference Optimization from Scratch | B. NLP LLM | Python | | 41 | Full Evaluation Pipeline | B. NLP LLM | Python | | 42 | Large Corpus Downloader | C. Train end-to-end | Python | | 43 | HDF5 Tokenized Corpus | C. Train end-to-end | Python | | 44 | Cosine LR with Linear Warmup | C. Train end-to-end | Python | | 45 | Gradient Clipping and Mixed Precision | C. Train end-to-end | Python | | 46 | Gradient Accumulation | C. Train end-to-end | Python | | 47 | Checkpoint Save and Resume | C. Train end-to-end | Python | | 48 | Distributed Data Parallel and FSDP from Scratch | C. Train end-to-end | Python | | 49 | Language Model Evaluation Harness | C. Train end-to-end | Python | | 50 | Hypothesis Generator | D. Auto research | Python | | 51 | Literature Retrieval | D. Auto research | Python | | 52 | Experiment Runner | D. Auto research | Python | | 53 | Result Evaluator | D. Auto research | Python | | 54 | Paper Writer | D. Auto research | Python | | 55 | Critic Loop | D. Auto research | Python | | 56 | Iteration Scheduler | D. Auto research | Python | | 57 | End-to-End Research Demo | D. Auto research | Python | | 58 | Vision Encoder Patches | E. Multimodal VLM | Python | | 59 | Vision Transformer Encoder | E. Multimodal VLM | Python | | 60 | Projection Layer for Modality Alignment | E. Multimodal VLM | Python | | 61 | Cross-Attention Fusion | E. Multimodal VLM | Python | | 62 | Vision-Language Pretraining | E. Multimodal VLM | Python | | 63 | Multimodal Evaluation | E. Multimodal VLM | Python | | 64 | Chunking Strategies, Compared | F. Advanced RAG | Python | | 65 | Hybrid Retrieval with BM25 and Dense Embeddings | F. Advanced RAG | Python | | 66 | Cross-Encoder Reranker | F. Advanced RAG | Python | | 67 | Query Rewriting: HyDE, Multi-Query, and Decomposition | F. Advanced RAG | Python | | 68 | RAG Evaluation: Precision, Recall, MRR, nDCG, Faithfulness, Answer Relevance | F. Advanced RAG | Python | | 69 | End-to-End RAG System | F. Advanced RAG | Python | | 70 | Task Spec Format | G. Eval framework | Python | | 71 | Classical Metrics | G. Eval framework | Python | | 72 | Code Exec Metric | G. Eval framework | Python | | 73 | Perplexity and Calibration | G. Eval framework | Python | | 74 | Leaderboard Aggregation | G. Eval framework | Python | | 75 | End-to-End Eval Runner | G. Eval framework | Python | | 76 | Collective Ops From Scratch | H. Distributed train | Python | | 77 | Data Parallel DDP From Scratch | H. Distributed train | Python | | 78 | ZeRO Optimizer State Sharding | H. Distributed train | Python | | 79 | Pipeline Parallel and Bubble Analysis | H. Distributed train | Python | | 80 | Sharded Checkpoint and Atomic Resume | H. Distributed train | Python | | 81 | End-to-End Distributed Training | H. Distributed train | Python | | 82 | Jailbreak Taxonomy | I. Safety harness | Python | | 83 | Prompt Injection Detector | I. Safety harness | Python | | 84 | Refusal Evaluation | I. Safety harness | Python | | 85 | Content Classifier Integration | I. Safety harness | Python | | 86 | Constitutional Rules Engine | I. Safety harness | Python, YAML | | 87 | End-to-End Safety Gate | I. Safety harness | Python |

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

The toolkit

Every lesson produces a reusable artifact. By the end you have:

outputs/
├── prompts/      prompt templates for every AI task
└── skills/       SKILL.md files for AI coding agents

Plug them into Claude, Cursor, Codex, OpenClaw, Hermes, or any agent that reads a SKILL.md / AGENTS.md directory. Real tools, not homework.

Install course skills into your agent

Two skill sets, two installers:

The learning skills (start-learning, learn, course-guide, learn-mcp, learn-agent-skills, claude-certification, find-your-level, and check-understanding) live under skills/ and install into a supported skill-capable host with one command. Installation needs Node.js and npx, but not a repository clone or Python:

npx skills add rohitg00/ai-engineering-from-scratch

skills writes to the host and scope selected during installation, such as .claude/skills/, .cursor/skills/, .codex/skills/, or another supported skills folder. Verify that the selected host discovers that exact destination.

The lesson artifacts. The repo ships 396 skills and 99 prompts under phases/**/outputs/; install them via scripts/install_skills.py. Requires cloning the repo. Supports tag filters, dry-runs, and per-agent layouts:

python3 scripts/install_skills.py <target>                                 # every skill, default --layout skills (nested)
python3 scripts/install_skills.py <target> --layout skills                 # same as above, explicit
python3 scripts/install_skills.py <target> --type all                      # skills + prompts + agents
python3 scripts/install_skills.py <target> --phase 14                      # one phase only
python3 scripts/install_skills.py <target> --tag rag                       # filter by tag
python3 scripts/install_skills.py <target> --layout flat                   # flat files
python3 scripts/install_skills.py <target> --dry-run                       # preview without writing
python3 scripts/install_skills.py <target> --force                         # overwrite existing files

<target> is the skills directory for your agent (examples: ~/.claude/skills/, ~/.cursor/skills/, ~/.config/openclaw/skills/, .skills/, or any path your agent reads).

By default the script refuses to overwrite an existing destination and exits with code 1 after listing every colliding path. Use --dry-run to preview collisions or --force to overwrite. Every non-dry-run run writes a manifest.json in the target with the full inventory grouped by type and phase. Pick the layout your agent reads:

| --layout | Path written | |---|---| | skills | <target>/<name>/SKILL.md (nested convention, supported by Claude / Cursor / Codex / OpenClaw / Hermes) | | by-phase | <target>/phase-NN/<name>.md | | flat | <target>/<name>.md |

Drop the agent workbench into your own repo

The Phase 14 capstone ships a reusable Agent Workbench pack (AGENTS.md, schemas, init / verify / handoff scripts). Scaffold it into any repo with:

python3 scripts/scaffold_workbench.py path/to/your-repo            # full pack + seeds
python3 scripts/scaffold_workbench.py path/to/your-repo --minimal  # skip docs/
python3 scripts/scaffold_workbench.py path/to/your-repo --dry-run  # preview only
python3 scripts/scaffold_workbench.py path/to/your-repo --force    # overwrite

You get the seven workbench surfaces wired up, a starter task_board.json, and a fresh agent_state.json at schema_version: 1. From there: edit the task, edit AGENTS.md, run scripts/init_agent.py, hand the contract to your agent. The pack source lives at phases/14-agent-engineering/42-agent-workbench-capstone/outputs/agent-workbench-pack/.

Browse the entire course as JSON

scripts/build_catalog.py walks every phase, every lesson, every artifact on disk and writes catalog.json at the repo root. One file, every course truth.

python3 scripts/build_catalog.py               # writes <repo>/catalog.json
python3 scripts/build_catalog.py --stdout      # to stdout, do not touch repo
python3 scripts/build_catalog.py --out path/to/file.json

The catalog is filesystem-derived, not README-derived, so counts always match what is actually on disk. Use it for site builds, downstream tooling, or to verify the README counts have not drifted. Schema is documented at the top of the script.

A GitHub Action (.github/workflows/curriculum.yml) rebuilds catalog.json on every PR and fails the build if the committed file is stale. After editing any lesson, run python3 scripts/build_catalog.py and commit the result, or CI will reject the PR. The same workflow runs audit_lessons.py in warn-only mode (so existing drift does not block contributors).

Smoke-check every lesson's Python code

scripts/lesson_run.py byte-compiles every .py file under each lesson's code/ directory. Default mode is syntax-check only — no execution, no API keys, no heavy ML deps required. Catches the regressions contributors introduce most often (bad indentation, broken f-strings, stray edits).

python3 scripts/lesson_run.py                  # syntax-check the whole curriculum
python3 scripts/lesson_run.py --phase 14       # one phase only
python3 scripts/lesson_run.py --json           # JSON report on stdout
python3 scripts/lesson_run.py --strict         # exit 1 if any lesson fails
python3 scripts/lesson_run.py --execute        # actually run, 10s timeout per lesson

--execute runs each lesson's code/main.py (or the first .py file) with a 10-second timeout. Lessons whose entry file starts with a # requires: pkg1, pkg2 comment listing non-stdlib deps are skipped with reason needs <deps>. The script is opt-in and not wired into CI.

Stdlib only, Python 3.10+. Set LINK_CHECK_SKIP=domain1,domain2 to override the default skip-list (twitter.com, x.com, linkedin.com, instagram.com, medium.com — domains that aggressively block automated HEAD/GET).

Where to start

| Background | Start at | Estimated time | |---|---|---| | New to programming and AI | Phase 0 — Setup | ~306 hours | | Know Python, new to ML | Phase 1 — Math Foundations | ~270 hours | | Know ML, new to deep learning | Phase 3 — Deep Learning Core | ~200 hours | | Know deep learning, want LLMs and agents | Phase 10 — LLMs from Scratch | ~100 hours | | Senior engineer, only want agent engineering | Phase 14 — Agent Engineering | ~60 hours | | Only want to build production MCP systems | Model Context Protocol (MCP) path | ~23 hours 15 min | | Only want to build production Agent Skills | Agent Skills Engineering path | ~9.5 hours |

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Why this matters now

"The hottest new programming language is English." — Andrej Karpathy (tweet)

"Software engineering is being remade in front of our eyes." — Boris Cherny, creator of Claude Code

"Models will keep getting better. The skill that compounds is knowing what to build." — Industry consensus, 2026

Attention Is All You Need — Vaswani et al., 2017 → Phase 7
Language Models are Few-Shot Learners (GPT-3) → Phase 10
Denoising Diffusion Probabilistic Models → Phase 8
InstructGPT / RLHF → Phase 10
Direct Preference Optimization → Phase 10
Chain-of-Thought Prompting → Phase 11
ReAct: Reasoning + Acting in LLMs → Phase 14
Model Context Protocol — Anthropic → Phase 13

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Contributing

| Goal | Read | |---|---| | Contribute a lesson or fix | CONTRIBUTING.md | | Fork for your team or school | FORKING.md | | Lesson template | LESSON_TEMPLATE.md | | Track progress | ROADMAP.md | | Glossary | glossary/terms.md | | Code of conduct | CODE_OF_CONDUCT.md |

Before submitting a lesson, run the invariant check:

python3 scripts/audit_lessons.py           # full curriculum
python3 scripts/audit_lessons.py --phase 14  # single phase
python3 scripts/audit_lessons.py --json    # CI-friendly output

Exit code is non-zero when any rule fails. Rules (L001–L010) validate directory shape, docs/en.md presence + H1, code/ non-emptiness, quiz.json schema (rejects the legacy q/choices/answer keys that caused issue #102), and relative links inside lesson docs.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Sponsor the work

Free, MIT-licensed, 523 lessons. The curriculum is maintained through cash sponsorships and approved Hardware Lab Partners.

Reach: 114,584 monthly visitors · 181,995 page views (Vercel Web Analytics verified 2026-08-29) · 50,728 GitHub stars (verified 2026-08-29).

Current sponsors

| Sponsor | What they support | |---|---| | CodeRabbit · iii | Current sponsors supporting curriculum maintenance and open-source development. | | | Open-source infrastructure support through the Vercel Open Source Program. |

| Tier | $/mo | What you get | |------|------|---| | Backer | $25 | Name in BACKERS.md | | Bronze | $250 | Text-only row in README sponsor block + launch-day tweet | | Silver | $750 | Small logo in README + listed as one supported provider in API lessons | | Gold | $2,000 | Medium logo in README + sponsor page + quarterly X / LinkedIn co-feature | | Platinum | $5,000 | Hero logo above the fold + one dedicated integration lesson, max 1 partner | | Diamond / Title Partner | $10,000 | Sole title-partner placement + hero logo + quarterly reporting and co-feature, max 1 partner |

Hardware manufacturers can support the course through the separate Hardware Lab Partner policy. Equipment is in-kind support and does not automatically grant a cash sponsorship tier.

Full rate card, hard rules, pricing anchors, and reach data: SPONSORS.md. Sign up via GitHub Sponsors.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

If this manual helped you, star the repo. It keeps the project alive.

License

MIT. Use it however you want — fork it, teach it, sell it, ship it. Attribution appreciated, not required.

Maintained by Rohit Ghumare and the community.

Comments0

No comments yet. Set the tone — say what you would want to know.