Chapter 05
The Six Practices
I have simplified the discipline to six practices. To be candid, there was some curve fitting to reach a tidy number, and I could have merged or split a few without losing much. Two things hold under all of them. The work starts with clarity about what I actually want from the agent. And every piece of it is gated by verification, with the strength of the gate set by how much the output can afford to be wrong. A forgiving, probabilistic output needs only a light check. An unforgiving one needs a hard gate.
Specs
Clarity of Intent
Before any of what follows does useful work, take a moment to know where you are.
There are two ends to a spectrum. On one end, you have already made the decisions. You know what you want, you know why, and you need execution. The right posture toward Claude Code in that case is prescriptive. Strip the rationale and keep the instructions. The agent does not need to be convinced of the decision, only told what it is.
On the other end, you want the agent to do the diagnosis. You want a fresh read of the repo, or the requirement doc, or the artifacts you have collected, looking for issues, proposing fixes, and surfacing things you missed. The right posture in that case is to say so. Ask for an artifact you can read and review. Iterate with the agent until the artifact is sharp enough to act on. Only then ask for execution. And make sure you have ways to validate that execution, preferably automated.
Most situations fall somewhere between the two ends. That is fine. It stops being fine when you are between the ends without realizing it, because then the ambiguity ends up in your prompts. The agent will respond to that ambiguity by either making too many decisions, in directions you did not intend, or by making too few, asking for clarification when it should be moving. Either way wastes a turn.
The recurring task is to be conscious of where you sit on this spectrum on any given prompt. Clarity of your own thought has to come before you ask the agent to think. What follows is written assuming you have done that thinking, and it describes what to do once you know which direction you are pushing.
Requirements
For any work that is not trivial, I write a requirement document first. The pattern I use is a docs/ folder that holds everything relevant to the project: design notes, meeting notes, architecture sketches, user stories, whatever helps. I have the agent draft the requirement from that material, then I read it, disagree with parts, and refine it by hand. Reviewing it is itself an exercise in getting clear, which is the point. When the requirement gets large, I have it broken into smaller files like docs/requirements/feature-name.md.
Manage Agents
Think Like a Manager
The most common mistake is to assume you will read every line the agent writes.
When you work in retail conversation with the agent, reading every diff line by line, holding every decision in your head, and approving every change by hand, you are doing something closer to typing than to engineering management. It does not scale past one piece of work at a time. A good engineering manager of ten developers would not work this way: there are not enough hours in the day, so the manager does something else.
The right frame is that you are managing ten developers, and the manager’s job is to make the conditions right. Each developer has a clear specification, which in our case is the requirement doc. Each one knows the conventions, which live in CLAUDE.md. A class of mistakes gets caught by tests, for free, another class gets caught by CI, another by the security-reviewer subagent at the moment of decision, and another by structured logs that surface what production cares about. Most of the verification happens through these mechanisms, not through you personally inspecting each change. The mechanisms are the lever a manager has that a typist does not.
As that manager, you calibrate where to dive deep. Sometimes the call is instinct, the feeling that a change in the auth code deserves your own eyes. Sometimes it is mechanism, the security-reviewer subagent set to run on the diff before merge. Sometimes the right move is a scan, where you skim the diff for anything that jumps out and trust the tests for the rest. And sometimes it is a full read, because the migration touches every customer’s data and you intend to look at every line. The choice depends on the risk profile of the change, on which mechanisms already cover the surface, and on what experience has told you about where this kind of work tends to go wrong. Calibrating that choice is the work.
The question itself changes. It stops being whether you read every diff, and becomes whether you have the right regime and are diving deep at the right moments. The first does not scale. The second lets the rest of the discipline pay off.
| Old job (type) | New job (manage) |
|---|---|
| Write each line | Review diffs from multiple streams |
| Read your own code | Read code you didn’t write |
| Catch your own mistakes | Catch invisible mistakes |
| Speed = typing speed | Speed = where you look |
Automated Tests and Deployments
Target 100 Percent Automated Test Coverage
I aim for close to full automated test coverage. How close depends on the project, but the target is to automate the whole pyramid: unit tests, integration tests, performance and endurance tests, and end-to-end tests. The number matters less than the habit of writing the test alongside the code, so almost nothing reaches production unchecked.
My own bias is a strong suite of unmocked end-to-end tests. For a web app that means something like Playwright driving a fully provisioned environment, with the real dependent systems connected wherever that is reasonable, so the test exercises the same paths a user will.
Evals for the AI Your Software Calls
Most modern applications have LLM dependencies. You need automated evals for the model-dependent part of the system against a fixed set of inputs that scores the results. There are two kinds, and a durable system that calls an LLM usually needs both.
The first is the deterministic eval. It applies whenever the model’s output has a checkable property, even when you cannot predict the output itself. If you ask the model to extract fields from a document, you cannot predict the phrasing, but you can assert that the result is valid JSON, carries the required keys, and that the dates parse. If you ask it to choose a tool, you cannot predict its reasoning, but you can assert it called the tool you expected with arguments in range. If you ask it to classify, you can hold a labeled set of examples and measure how often it agrees with the labels. Deterministic evals are cheap, they pass or fail, and they belong in CI beside the unit tests. They catch the regressions that hurt most, like the prompt edit that quietly breaks JSON output, or the model upgrade that starts reaching for the wrong tool.
The second is the model-based eval, often called LLM-as-judge. It applies when the output is open-ended and there is no single right answer, like a summary, an explanation, the reply in a support chat, or the tone of a generated message. A second model scores the first against a rubric that asks whether the output is faithful to the source, whether it is relevant, whether it is safe, and whether it is in the right shape. This is how you verify the parts of an AI feature that have no golden string to compare against.
The model-based eval has to be handled with more care than the deterministic one, because the judge is itself a fallible model. A judge has biases. It tends to prefer longer answers, it can be swayed by confident phrasing, and it drifts when the model behind it is updated. Keep the judge honest the way you would a human grader. Calibrate the judge against examples a human has already scored, and trust it only as far as it tracks them. Pin the judge’s model and version, so a score means the same thing next month that it meant today. Treat the number as directional, watched over time, rather than as a precise grade. A model-based eval that has never been checked against human judgment gives you one model’s opinion of another, which is not verification.
These two kinds of eval are the nudge-to-gate spectrum, seen from the verification side. Where a behavior is deterministic, you check it with an exact assertion that passes or fails, the gate’s equivalent in a test. Where it is probabilistic, an exact assertion is the wrong instrument, so you verify the properties that must hold across every valid output and, for the open-ended part, fall back to a judge whose score you watch over time, the self-check band doing verification rather than enforcement. The more of the output you can pull toward the deterministic end, by constraining its shape as it is generated so well-formedness is guaranteed rather than checked, the more the eval can spend its attention on the part no constraint can supply, which is whether the values are right. Format can be enforced; correctness still has to be verified. Matching the rigor of the check to where the behavior sits is the whole game.
Evals sit at a different point in the workflow than unit tests. Unit tests gate code changes. Evals gate the things that change the model’s behavior, including a new prompt, a new model version, and a change to the context you retrieve and feed in. When you would otherwise have no way to tell whether last week’s prompt edit made the feature better or worse, the eval set is the thing that can tell you.
# Deterministic eval: the output is unpredictable, the shape is not.
GOLDEN = load_cases("evals/extraction/*.json") # inputs + expected fields
def test_extraction_is_well_formed():
for case in GOLDEN:
result = extract_invoice(case.input) # real model call
assert is_valid_schema(result, InvoiceSchema)
assert result["total"] == case.expected["total"]
assert result["date"].isoformat() # parses
# Model-based eval: no golden string, so a judge scores against a rubric.
def eval_summary_faithfulness(source, summary):
verdict = judge( # pinned judge model
rubric="Is every claim in the summary supported by the source? "
"Score 1-5 and cite any unsupported claim.",
source=source, candidate=summary,
)
return verdict.score # tracked over time, calibrated against human labels
| Deterministic eval | Model-based eval | |
|---|---|---|
| Use when | Output has a checkable property | Output is open-ended; no single right answer |
| Checks | Schema, fields, tool calls, labels | Faithfulness, relevance, tone, safety |
| Cost | Cheap; runs in CI | A model call per case; slower |
| Verdict | Pass / fail | A score, tracked over time |
| Watch out for | Brittleness if the property is too strict | The judge’s own bias; must be calibrated |
The part of the system that used to be unverifiable, the model’s behavior that was mocked away in every test, gets a way to be checked without faith. A prompt change and a model upgrade stop being leaps of faith, because you can measure whether they helped. The keystone of the discipline, verifiability, reaches the one component deterministic testing could never touch.
Idempotent Deployment That Provisions Everything
A full CI/CD pipeline is standard practice, and for me it starts at the code level. Whenever the agent generates code, I have it keep every piece of infrastructure setup in idempotent scripts, with nothing done by hand. The scripts stand up a complete environment from scratch, on demand: the code, the services, the schema, the IAM, and the seed data. I do this for two reasons. Reliable deploys come from running the same script every time, and automated testing only works when a clean environment is cheap to create.
New Developer Onboarding
Onboarding a new developer rests on those same scripts and a good README.md. I make sure the README carries enough for someone to get the app running quickly, and I keep the distance from clone to running app short. That distance is a direct reading on maintainability: the longer it takes a new person, or a fresh agent, to get going, the less maintainable the project is.
Agent Context
The agent only knows what is in its context, so much of the work is keeping that context clear and current. With Claude Code, a few parts make it up. There is memory, which Claude keeps either because I told it to or because it judged the detail worth holding; I review and prune it so it stays accurate. There is CLAUDE.md, loaded into every request, which is why it has to carry the right rules and stay small; Anthropic recommends keeping it under a modest size for exactly that reason. There are the other features, skills, hooks, and rules among them, that shape behavior quietly when you are not watching them. And there is the docs/ folder, which Claude does not load on its own but which I can point it at when a task needs it.
The mechanics will change as Claude Code evolves. The principle does not: give the agent the right information, at the right level, to make the decisions the work needs. That is what keeps its output aligned with the project instead of with whatever it happened to assume.
Same Thing, the Same Way
Across the codebase, I keep the same kind of thing looking the same way. A card is one CSS class, used everywhere a card appears. A form field is one component, used everywhere a form field appears. A page header is one shape, used in every page that has a header. When the agent generates a new page, it pulls from the existing vocabulary; it does not invent fresh styling for each new screen.
In the Flask app the example uses, the styling layer is Tailwind utility classes with daisyUI components on top. The rules are simple. No inline styles. No per-page stylesheets. If a daisyUI component fits, use the daisyUI component. If you need to deviate, deviate in one place and have every page that needs that deviation use the same shared class. The override lives in app/static/css/site.css or its equivalent, and the override has a name. The agent is told, in CLAUDE.md, that this is the rule. When the agent suggests an inline style or a new ad-hoc class for a card that already has a card class, the suggestion is rejected and the existing class is reused.
The principle generalizes past CSS. Routes have one shape, service functions have one shape, and Firestore documents of a given kind have one shape. The codebase has a vocabulary, and adding a new word to the vocabulary is a deliberate act, not something that happens by accident because the agent did not know the existing word.
The visual and structural coherence I called the architecture’s shape stops eroding feature by feature. New contributors find one place to look for how something is done, and the answer is the same one they will find next time. The codebase drifts more slowly than it otherwise would.
Probabilistic → Deterministic
A model’s output is a guess, and most of this discipline is about not betting more on a guess than it can bear. For each behavior I want from the agent or its code, I ask one question: how wrong can this be before it matters? The answer sets how hard I work to pin it down.
The controls I have run from soft to hard. A line in the prompt is the softest: the model usually honors it and sometimes does not. A standing rule in CLAUDE.md is firmer, read on every turn, though still advice the model can drift from. A hook is firmer still, because it runs on every change and can refuse one. A test or a CI check is the hard end: it blocks the merge, and the model cannot talk its way past it.
A prompt is a nudge; a gate is a guarantee.
The work is putting each behavior at the right point on that range. A tidy log message can ride on a nudge in the prompt; if the model phrases it oddly now and then, nobody is hurt. A function that must never write a raw card number to the logs is the other extreme, and it earns a test that fails loudly, because the cost of being wrong once is too high to leave to a request. Most things sit in between, and the judgment is matching the control to the cost of the mistake.
Both directions cost you. Lean too soft, and something you could not afford to get wrong is riding on a sentence the model is free to ignore. Reach for a gate everywhere, and you have built, and now have to maintain, machinery around behaviors that never needed it. I have done both. The same question sorts it out each time: how wrong can this be before it matters, and is the control I have strong enough for that answer?
None of this makes the model deterministic. It shrinks the part left to chance. I pin the output that has a checkable shape, enforce the rules that have to hold, and gate the few things that must never break, so the genuinely probabilistic part that remains is small, known, and watched.
Failing Loud and Early
When AI is generating code quickly, I lean hard on making the system fail loudly and early, with enough detail to act on. I find it the cheapest way I have to keep a system debuggable. A failure that shows up at startup, next to its cause, costs me minutes to find. Hidden behind a default, the same failure surfaces three layers downstream and costs me an afternoon.
There is a temptation, when reading a list like this, to treat it as a checklist, as if covering most of the items means you are doing well. That is the wrong frame. None of this is something to complete. It is a set of facets of one discipline, which you pursue continuously. The discipline is continuous in the sense that the CLAUDE.md needs trimming, the calibration needs adjustment, the docs need sweeping, etc.
The discipline is a pursuit that begins to make the project durable.