# Metrics That Matter

**Track:** Scaling & Stewardship — AI for Entrepreneurship — complete (29)
**Framework / surface:** venture strategy
**Level:** Advanced
**Prerequisites:** Data Flywheels & Cold Starts

**In one line:** Retention over signups, margin over revenue, evals over vibes — measuring an AI product honestly.

## Theory, aesthetics & inspiration

Sequoia's "Generative AI's Act Two" (2023) said the quiet part with data: AI apps were acquiring users at historic speed and retaining them far below the mobile-era benchmarks, because novelty drives signups and only embedded value drives return visits. The honest dashboard for an AI-native product starts there — cohort retention over cumulative signups, and a north-star metric denominated in completed jobs, not sessions. Around it sit the metrics this architecture makes newly necessary: margin per successful outcome (tokens, retries, and human review included), eval pass rates trended across model versions, escalation rate — how often the human behind the curtain still catches the work — and correction rate feeding the data flywheel. Vanity has new costumes: "AI interactions" is the new page views, and services-shaped revenue that builds no reusable asset shows up only in margin — the Service First test applies: measure the residue, not the shape. The exponential adds one composite worth watching — capability leverage: does each model upgrade raise your completed-jobs-per-user while costs fall? A durable product answers yes reflexively.

**Founder question:** When the models improve, do your completed-jobs-per-user rise without you shipping anything?