Learning LLMs

Mostly notes to my future self. The plan is to go deep enough on the fundamentals that the rest stops feeling like magic.

Phase 1 · Pre-training

The architectural foundations of a base model. Tokens, embeddings, associations, attention, the transformer.

  1. 1 Pre-training: An Overview A flyover of the pre-training pipeline. Text in, trained model out. Each stage gets its own deep dive in the rest of the series.
  2. 2 Tokens and Tokenization How LLMs split text into tokens, the BPE algorithm, and why 'strawberry' has 3 r's the model can't see.

Phase 2 · Training

How a base model's parameters actually get learned. Loss, gradient descent, backprop, scaling laws, training at scale.

    Phase 3 · Post-training

    How a base model becomes a useful assistant. SFT, RLHF, DPO, constitutional methods, reasoning-model RL.

      Phase 4 · Leveraging the LLM

      Using a trained model as a developer. Sampling, system prompts, tool use, structured output, context management.

        Phase 5 · MCP

        The protocol connecting LLMs to the world. What it is, how it works, building it, where it goes.

          Phase 6 · Workflows

          AI-native software development. The agentic landscape, sandboxing, niche tools, autonomous workflows.