# How Models Learn

**Track:** Model Foundations — AI for Entrepreneurship — model foundations (5)
**Framework / surface:** venture strategy
**Level:** Beginner
**Prerequisites:** none

**In one line:** Pre-training predicts the next token at web scale; post-training — instruction tuning and RLHF — turns the raw predictor into an assistant with someone's chosen taste.

## Theory, aesthetics & inspiration

Every model you build on was made twice. Pre-training is the first act: self-supervised next-token prediction over web-scale text, from which grammar, facts, and style emerge as by-products of a single objective (Brown et al., 2020). The result is a base model — fluent, unruly, and indifferent to instruction. Post-training is the second act: supervised instruction-tuning followed by preference optimization — reinforcement learning from human feedback, proposed by Christiano and colleagues in 2017 and industrialized in OpenAI's InstructGPT (Ouyang et al., 2022) — shapes the raw predictor into an assistant. For a creative founder the lesson is that a model's default taste is not nature but curation: human raters chose the behavior that made it agreeable, cautious, and generic. Knowing where behavior comes from tells you which layer — prompt, fine-tune, or post-training — you must change to make it yours.

**Founder question:** Which of your product's behaviors come from the base model, and which from someone else's post-training choices?