The scariest thing about a genuinely capable coding agent is how quickly it commits. You type two sentences, and forty seconds later there are changes across nine files, half of which you did not want and one of which quietly changed a query you spent a week hardening. The agent was not wrong about how to implement the thing. It was wrong about what the thing was, and it never stopped to check.
I have watched this happen enough times that I stopped treating it as a prompting problem and started treating it as a workflow problem. A better one-shot prompt is not the fix. A gate is: the agent is not allowed to edit files until it has asked what it needs to ask, shown me a plan, and gotten a yes.
Why speed is the problem
Human engineers have a built-in pause. Before a senior developer touches your codebase they ask a couple of questions, sketch the approach, maybe drop a comment on the ticket. That pause is where the wrong-work gets caught, cheaply, in a sentence, instead of expensively, in a diff you have to read and reject.
Agents removed the pause. That is most of their value and most of their danger in the same motion. An agent that implements immediately optimizes for the wrong thing: it treats “produce a diff” as the goal, when the goal was “produce the diff we agreed on.” The gap between those two is where the deleted work and the silent scope creep live.
That gap is not a knowledge problem. The agent knows how to write the code. It just never checked that it was writing the right code.
The same task, gated and ungated. Skipping the pause trades a small, predictable cost for a large, unpredictable one.
So I gave the pause back, deliberately, as a standing instruction on every agent in the InterlinedList repo.
The three beats
The workflow is written down in .claude/workflows/plan-first.md and every agent links to it. It is three beats, in order, and implementation is gated behind all three.
The three beats, in order. Nothing is edited until all three pass, and work that grows past the approved plan loops back to re-plan rather than quietly expanding.
Ask. Surface what the prompt left open before committing to an approach. Ambiguous scope, unstated edge cases, a product decision hiding inside a technical request, whether a feature should be tier-gated. The migrations agent asks about column types and nullability and whether anything destructive is implied. The Next.js agent asks which surfaces are in and out. The rule has an escape hatch, because asking three questions about a one-line copy fix is its own kind of annoying: skip the questions only when the request is genuinely unambiguous and low-risk. When in doubt, ask. A pointed question is cheaper than a wrong build every single time.
Plan. Before touching files, lay out the shape of the change: the files and routes and components you will touch, the ones you will deliberately leave alone, the approach, any migration (additive, always), the tests the change needs, and anything risky. In this repo “risky” has a specific meaning: auth, IDOR, subscription gating, SSRF, secret handling, anything destructive or hard to reverse. The plan is a decision aid, not a document. It should be short enough to read in one breath and specific enough that approving it means something.
Confirm. Implement only after an explicit yes. If the plan changes in the back-and-forth, restate the revised version and get the yes again. And the part that actually matters over a long session: approval is scoped to the plan that was approved. If the work grows past it, the agent stops and re-plans instead of quietly expanding. That last clause is what keeps a “small fix” from turning into an afternoon of changes I never signed off on.
What it looks like per agent
I did not want one generic paragraph pasted eight times. The gate is the same, but what you ask about depends on the job, so each agent got the beats written for its lane.
One gate, written for each lane. The two read-only reviewers pick the full gate back up the moment they move from finding to fixing.
The migrations agent plans the exact idempotent migration.sql and confirms it is purely additive before it applies anything. The unit-testing agent asks which behaviors to lock in and which boundaries to mock, then lists the cases each test file will assert. The e2e agent names the flows, the auth and seed prerequisites, and the breakpoints that matter. The docs agent confirms which of the three docs is in scope and whether a new page is needed. The blog agent (yes, this one) settles the angle and the section arc and which real code it will verify claims against before drafting a word.
The two read-only reviewers are the interesting edge. Security and UX do not implement, so there is no edit to gate. For them the gate degrades to its first beat: confirm the review scope if it is ambiguous (which routes, how deep, which breakpoints), then produce findings. But the moment the user says “now fix what you found,” they are implementers, and the full ask-plan-confirm gate snaps back on before they touch code. The reviewer does not get to slide from “here is a finding” into “and I fixed it” without crossing the same line everyone else crosses.
The obvious objection
This is slower. That is the point, and it is also not as true as it sounds. The plan step costs you a few seconds and one read. Rejecting a forty-second nine-file diff that went the wrong direction costs you the read plus the reject plus the re-prompt plus the nagging worry about what it touched that you did not catch. The gate front-loads a small, predictable cost to avoid a larger, unpredictable one. Over a day of handoffs it is not close.
It also composes with the other habit I built into these agents: every one of them does its work in an isolated git worktree, on its own branch, torn down when the task lands. Plan first, then do the approved work in a sandbox that cannot collide with anyone else. The worktree contains the blast radius. The plan makes sure there is not supposed to be a blast in the first place.
The pause was always the expensive part of good engineering. Worth teaching the machines to keep it.
Run two coding agents against a single working tree and they will fight. One is halfway through editing app/api/messages/route.ts while the other checks out a different branch underneath it. The index lock flickers. A git stash from one session swallows the other’s uncommitted work. I have lost real edits this way, and every time the root cause was the same: one working tree, one HEAD, two writers.
The InterlinedList repo already had matching agents for the jobs I hand off most: a Next.js implementer, a migrations specialist, unit and e2e testers, a docs writer, security and UX reviewers. They are good at their lanes. What they were missing was a lane in the literal sense. They all drove on one road.
I found the pattern I wanted written up in Augment’s guide to git worktrees for parallel AI execution, and it maps almost one to one onto how I already think about agents. This is the writeup of what I built on top of it for this repo: the directory convention, the seven scripts that manage the lifecycle, and the wiring that makes every agent use them without being reminded.
The one-tree problem
A normal clone gives you a single working directory backed by one .git. That is fine for one person doing one thing. The moment you parallelize, the shared mutable state (the working files, the index, the current branch) becomes the bottleneck. You cannot have the migrations agent on agent/add-webhooks and the docs agent on agent/help-refresh at the same instant, because “the branch” is a property of the whole checkout.
Two writers, one mutable checkout. The collisions have nothing to do with the work: an index lock flickers, a git stash swallows the other session’s edits, HEAD thrashes between branches.
Git solved this in 2015 with git worktree. A worktree is a second (third, fourth) working directory attached to the same repository. Each one has its own files, its own index, and its own checked-out branch, while sharing one object store on disk. The object store is the expensive part, so you share it. A directory of files is cheap, so you duplicate it per task. That is exactly the tradeoff you want for parallel agents.
Share the expensive part, duplicate the cheap part. The object store lives on disk once; the working files, the index, the checked-out branch, and even the dev-server port are private to each worktree.
The convention
Every agent works in .trees/<task-id> on a branch named agent/<task-id>, cut from origin/develop (this repo integrates on develop, not main). So a task called add-list-webhooks lives at .trees/add-list-webhooks on branch agent/add-list-webhooks. The directory name and the branch name always agree because both are derived from the same sanitized slug.
The .trees/ container is gitignored. Worktrees are workspace, not history:
# .gitignore # Agent worktrees (see scripts/worktrees/) .trees/
That single ignore line is the whole footprint the pattern leaves in the tracked tree. Everything else is scripts and instructions.
The scripts
I did not want agents (or me) typing raw git worktree incantations and getting the branch name wrong, or forgetting to copy .env.local, or leaving stale metadata behind. So the lifecycle lives in scripts/worktrees/ as a small set of focused shell scripts. Here is every one of them.
The whole lifecycle before the per-script detail. Create, inspect while working, remove when it lands, sweep the merged trees on a schedule.
_lib.sh is the shared library the others source. It holds the functions that keep conventions consistent: wt_repo_root resolves the primary checkout even when you call it from inside a linked worktree (it reads git rev-parse --git-common-dir and walks up), wt_sanitize lowercases a task id and strips it to [a-z0-9._-], wt_resolve_base fetches and prefers origin/<base> over a local branch, wt_port_for hashes a branch name into a stable dev port, and wt_is_locked reads the porcelain worktree list to check lock state. Nothing in it is clever. It exists so the clever bits are written once.
wt-create.sh <task-id> [base] is the one agents call first. It creates the worktree and makes it ready to work in, in one shot:
scripts/worktrees/wt-create.sh add-list-webhooks
Under the hood it resolves the base ref (default develop), ensures .trees/ is in .gitignore, turns on git rerere so repeated conflict resolutions replay across parallel merges, runs git worktree add -b agent/<slug> .trees/<slug> origin/<base>, copies the root .env.local into the worktree, appends a deterministic DEV_PORT derived from the branch name, runs npm ci --prefer-offline, and finally locks the worktree so other sessions can see it is in use. Flags let you opt out where it makes sense: --no-install skips the dependency install for a quick plumbing check, --no-lock leaves it unlocked, and --baseline runs the test suite right after setup so a green baseline proves any later failure came from the agent’s change rather than a pre-existing break.
The port assignment is worth a sentence. Two agents both running next dev on 3000 is another collision, a quieter one. wt_port_for runs the branch name through cksum and maps it into the 3100 to 9998 range, so each worktree gets a stable, distinct port written into its own .env.local. Start the server with npm run dev -- -p "$DEV_PORT" and two dev servers coexist.
wt-list.sh answers “what is running right now.” It prunes stale metadata, then prints one row per worktree with path, branch, lock state, and short HEAD:
Before an agent touches a shared file, it can look here and see who else is holding what. Git will not warn you about two branches editing the same file, so this list plus discipline about non-overlapping file domains is the actual safety mechanism.
wt-lock.sh <task-id> [reason] and wt-unlock.sh <task-id> are thin wrappers over git worktree lock/unlock. A lock is advisory: it resists prune and move, and it is the signal in wt-list.sh that says “an agent is live in here, do not reap this.”
wt-remove.sh <task-id> is the teardown. It unlocks if needed, runs git worktree remove, and prunes. This matters more than it looks: deleting a worktree with rm -rf leaves dangling metadata in .git/worktrees/ that haunts you until the next prune. The script never does that. --force discards uncommitted changes on purpose, and --delete-branch drops agent/<task-id> in the same step when the work has landed.
wt-cleanup.sh [base] is the bulk sweep. It walks every worktree physically under .trees/, and for each one whose branch is already an ancestor of origin/develop (in other words, merged), it removes the worktree and deletes the branch. It deliberately skips anything outside .trees/, so the primary checkout and any sibling worktrees I keep elsewhere on disk are never touched. This is the script you point a daily cron or a post-merge hook at.
Each of these is also exposed as an npm script, so npm run wt:create -- add-list-webhooks, npm run wt:list, and npm run wt:remove -- add-list-webhooks all work if you prefer that entry point. There is a scripts/worktrees/README.md documenting the whole set alongside the code.
Wiring it into the agents
Scripts nobody runs are decoration. The point was to make every agent reach for a worktree by default, so I added the instruction in three places at three levels of specificity.
At the top, CLAUDE.md now states the standing rule: every agent works in an isolated worktree, torn down with the lifecycle scripts. That is the repo-wide contract.
In the middle, two shared protocol docs under .claude/workflows/ hold the full detail: worktrees.md spells out the create-work-remove lifecycle, the boundaries (shared object store, non-overlapping files, and the important caveat that the database is shared even though the files are not), and plan-first.md covers the companion habit I wrote about separately. Every agent and skill links to these rather than repeating them.
At the leaf, each of the eight agent definitions in .claude/agents/ got a “Work in an isolated git worktree (required)” section written for its job. The implementers (Next.js, migrations, tests, docs, blog) get the full create-work-remove flow. The two read-only reviewers (security, UX) get a variant that tells them to cd into the worktree under review and read its diff, and explicitly not to create, lock, or remove anything. The migrations agent gets an extra warning in bold, because the worktree isolates schema.prisma and the migration files but not the Postgres instance: db:migrate still hits localhost and db:migrate:deploy still hits production from any worktree. That is the one place the isolation is a lie, and the agent needs to know it.
The five paired skills in .claude/skills/ (the ones that back the implementer agents) got a short “Worktree-first, plan-first” block near the top pointing at the same protocol docs, so whether the work comes in through the agent or the skill, the instruction is there.
Proving it works
I ran the whole lifecycle before committing any of it. Create a worktree from develop, confirm it is locked and has its port, unlock and re-lock it, remove it with the branch, and verify the cleanup pass leaves the sibling worktrees alone:
Every step did what it said, the .gitignore guard refused to duplicate the .trees/ line it found already present, and the sibling worktree I keep for feed-perf work was never in scope for cleanup. That last part was the thing I most wanted to confirm, because a cleanup script that reaches outside its sandbox is worse than no cleanup script.
Where agents still get confused
Isolating the filesystem fixes the filesystem. It does nothing about the fact that the work itself overlaps, and there are a handful of ways an agent still gets lost.
The database is one instance, and every worktree writes to it. I flagged this to the migrations agent in bold, but it deserves more than a warning. If the migrations agent adds a column on agent/add-webhooks, that column now exists in the same localhost Postgres every other worktree points at. The docs agent three trees over never sees the changed schema.prisma, yet its queries hit the mutated database anyway. Additive-only migrations keep this survivable most of the time, since an extra column nobody reads is harmless. But the moment two agents touch the same table, or one runs db:migrate:deploy and reaches production from what looked like a sandbox, the isolation is a fiction. The files are private. The database is not.
Green in isolation, red on merge. This is the one that bites. Agent A changes a function signature in lib/lists/queries.ts. Agent B, on its own branch, calls that function from a route it owns. Their files never overlap, so wt-list.sh shows no conflict and git stays quiet. Both test suites pass, because A’s worktree still holds B’s old caller and B’s worktree still holds A’s old signature. It’s all green right up until both branches land on develop, and then the integration is broken in a way neither agent could see from inside its own tree. Worktrees convert a loud, immediate collision into a quiet one that surfaces later. Often a fair trade, but a trade.
Stale base drift. Every worktree is cut from origin/develop the moment it’s created. Agents aren’t always short-lived. Let one run for a few hours while three others merge back, and it’s now building on a develop that no longer exists. It will reintroduce a helper that got deleted upstream, or write against an API another agent already reshaped. The worktree has no idea the ground moved under it. Nothing in the create-work-remove loop forces a re-fetch, so a long-running agent drifts out of date without noticing.
“Does this already exist?” stops having one answer. With five branches in flight, whether feature X is “already built” depends on which tree you grep. An agent that checks the primary checkout won’t see work-in-progress on another branch, and it will happily build a second copy. I’ve watched two sessions independently implement overlapping halves of the same feature, each sure it was first, because neither branch was visible to the other and neither said up front what it was about to touch. wt-list.sh tells you which branches exist. It says nothing about what each one intends to change.
The lock is a suggestion.wt-lock resists prune and move, and it flags a tree as live in the list, but it will not stop another agent from opening the same file on its own branch and editing away. The real guard against two agents clobbering one file is the up-front decomposition plus the discipline to run wt-list.sh and honor what it shows. An agent that skips the check has no seatbelt, just the shape of one.
Abandoned trees pile up.wt-cleanup.sh only sweeps branches already merged into develop. A worktree from a crashed or cancelled session is unmerged, still locked, and invisible to the sweep. It sits on disk with a full node_modules until someone removes it by hand. And an agent resuming a task can trip over a half-finished tree from an earlier run and read its stale state as current work.
Losing the current directory. The workflow says cd .trees/<task-id> and do everything there. Shell state doesn’t always survive between tool calls, and an absolute path into the primary checkout looks identical to one into a worktree. An agent that loses track of where it is can read the main checkout’s copy of a file, reason about it as though it were its branch’s version, and edit the wrong tree. The isolation holds only as long as the agent keeps its bearings.
What this buys
The honest version: worktrees do not make independent tasks independent. If two agents both need to edit the same route, isolating their filesystems just delays the merge conflict, it does not prevent it. The decomposition still has to be real. What the pattern removes is the accidental collision, the kind that has nothing to do with the work and everything to do with sharing one mutable checkout. Those were most of my pain, and now they are gone by construction.
Advantages and disadvantages
Advantages
Accidental collisions disappear. Each agent gets its own files, index, and HEAD, so index-lock flicker, a stray git stash eating another session’s edits, and checkout thrash stop happening.
Every task lands on its own agent/<task-id> branch, which keeps review and merge clean and stops one task’s half-finished mess from bleeding into another’s diff.
The object store is shared, so the costly part of the repo lives on disk once and spinning up another tree is cheap.
Each worktree gets a deterministic DEV_PORT, so several next dev servers run side by side instead of fighting over 3000.
The lifecycle is scripted and wired into every agent definition, so the right setup and teardown happen without anyone remembering the incantation.
git rerere is on by default, so a conflict you resolve once replays across the parallel merges that hit it again.
Cleanup is fenced to .trees/, so the bulk sweep never reaches the primary checkout or the sibling worktrees I keep elsewhere.
Disadvantages
The database isn’t isolated, and neither is anything else global (production through db:migrate:deploy, OAuth apps, third-party rate limits). File isolation quietly implies an isolation that isn’t there.
Overlapping edits on separate branches turn into silent, deferred conflicts and semantic breakage that pass every isolated test and only show up at integration.
A long-running worktree drifts from a moving develop, and nothing in the loop forces the re-fetch that would catch it up.
Locks are advisory, so the actual protection against two agents editing one file is decomposition plus discipline, not anything git enforces.
With several branches live, “does this already exist” has no single answer, and agents duplicate each other’s work when branches can’t see one another.
Every worktree carries a full node_modules, so disk use and npm ci time multiply with each active task.
Crashed or abandoned sessions leave locked, unmerged trees that the merged-only cleanup won’t reap, so someone clears them by hand.
Coordination is still manual: the tooling shows which branches exist, not which files each agent means to touch.
The next habit I want in every agent sits upstream of all of this: stop and plan before touching a single file. That one is worth its own post.
More solutions to the confusion above are coming in the next few posts. Subscribe so you don’t miss ’em: drop your email into the box just below this post, or grab the RSS feed.
Back in Part 1, I said the whole series was one argument dressed up as eight posts. LLM tokens are the expensive thing, Temporal is not, and Temporal’s mechanics structurally push the token bill down. I also said the way I’d build it was to brainstorm against the real code first, then hand the agent structure instead of hope. This is where those two threads tie together.
The capstone I keep circling back to is the one that sounds the most like science fiction and is actually the most dangerous to ship carelessly: “research topic X from my saved links and draft a document with citations.” The agent retrieves. It reasons. It drafts. And if I’m not careful, it also burns through a month of my inference budget in one runaway loop while I’m asleep.
So this post is two things. The agent itself, and the playbook for building everything that came before it. The agent is the thing you build last, and the order it sits on top of is the part I actually want to argue for.
The Capstone Is Just the Earlier Pieces, Composed
This part should feel earned by now. The agentic-research-to-draft phase, the last of the proposals, doesn’t invent anything. It composes.
The link content from Part 5 (the crawl-and-cache that turns your saved URLs into readable text) is the agent’s source material. The retrieval from Part 5’s second half, the pgvector search over your own docs, lists, and messages, is how the agent finds the relevant three paragraphs instead of stuffing forty links into context. And the durable generation from Part 3, the workflow that survives a timeout and asks before it writes, is the drafting step.
Compose those three and you get an agent loop: retrieve, reason, retrieve again, draft, cite. Each of those steps is already an activity I’d have built and tested for its own feature. The agent is a workflow that calls them in a loop the model steers.
Which is exactly where it gets scary. A loop the model steers has no natural stopping point. It’ll keep deciding it needs one more search, one more fetch, one more pass at the draft. That’s not a bug in the model. That’s what an agent is. The bug would be letting it run unsupervised.
Temporal Supervising an LLM Agent Is a Nice Recursion
I like this framing because it’s a little bit funny. Temporal is an orchestrator whose entire job is supervising unreliable, long-running, retry-heavy work. An LLM agent is unreliable, long-running, retry-heavy work. So the agent runs inside a Temporal workflow, and the workflow is the adult in the room.
Three guardrails, all first-class workflow logic, none of them prompt-hope:
A hard token budget. Before every model call, the workflow calls assertBudget, the same primitive from Part 7 that reads the AiGeneration ledger and throws a non-retryable BudgetExceeded when the user’s daily or monthly ceiling is hit. The agent can want another search all it likes. If the budget’s gone, the call never fires.
A max-step cap. The loop counts its own iterations. Twelve steps, or whatever I tune it to, and it’s done. It drafts with what it has rather than spiraling.
A wall-clock timeout. Temporal’s workflow timeout means even a wedged agent eventually stops, cleanly, with whatever it produced so far.
And here’s the piece that makes the whole thing cost-sane instead of terrifying: every step is memoized. If the agent fails at step 9 of a 12-step loop (provider hiccup, worker restart, whatever) the workflow resumes at step 9. It does not re-run steps 1 through 8. It does not re-pay for the eight model calls it already made. In a serverless handler, a mid-loop failure means starting the whole expensive agent over. Here it means retrying one step.
Sketching the loop, this is the shape I’d hand Claude to fill in:
// Proposed: the supervised agent loop as a Temporal workflow.
export async function researchAgentWorkflow(input: ResearchInput): Promise<Draft> {
const MAX_STEPS = 12;
const context: Snippet[] = [];
for (let step = 0; step < MAX_STEPS; step++) {
// Governor first: a hard ceiling the agent cannot talk past.
await assertBudget(input.userId);
// Cheap model plans the next move over the small retrieved context.
const move = await planNextStep({
model: "claude-haiku-4-5",
goal: input.topic,
context,
});
if (move.type === "done") break;
// Retrieval is mechanical and nearly free. No expensive tier here.
const hits = await retrieveFromLinks(input.userId, move.query);
context.push(...hits); // stays small: top-k snippets, not whole pages
}
// Expensive tier runs exactly once, on the final synthesis.
return await draftWithCitations({
model: "claude-opus-4-8",
topic: input.topic,
context,
});
}
The cost story lives in the model choices, not the prose. Retrieval and planning run on a cheap tier: Haiku for the “what should I look for next” decisions, which are frequent and small. Opus 4.8, the expensive tier, runs once, on the final synthesis, over a retrieval-augmented context that stayed deliberately small the whole way through. I’m not paying Opus prices to decide which link to read next. I pay them for the draft, and that’s the one thing in the loop I think is worth it.
No clever prompt is doing that. The structure is.
How I’d Sequence the Whole Build
If you’ve read the series straight through, you might be tempted to build the shiniest thing first. Don’t. The agent ships last, and the ordering underneath it is the real deliverable of this post, because every step ships value on its own and nothing is a big-bang.
Here’s the order I’d hand myself:
1. The foundation phase: the Temporal worker and the cost primitives. The worker, the client wiring, the llmCall activity with the model cascade, assertBudget, the response cache. This is the groundwork everything else stands on, and nothing works without it. Mostly infra, not code, and it’s a week.
2. The durable publish and idempotency fix, pure win, no LLM. This is the one from Part 4: the every-minute cron that can double-publish because a partial failure re-runs on the next tick. Rebuild it as a Temporal Schedule with the message id as the workflow id, so a given message can never be published twice. Zero model tokens. It fixes a current production hazard. If I could only ship one thing from this whole series, it’d be this.
3. Durable Generate plus confirm. Part 3. Long docs stop blowing the function budget, and a wrong list schema never gets silently written because the workflow pauses on a signal and waits for a human. This is the in-product generate button, on solid footing.
4. Link crawl, then retrieval. Part 5. Crawl the links into a cache (the cheapest, highest-cache-value piece in the set) then layer pgvector retrieval on top. This is “where did I save that article?”
5. Batch enrichment, then prompt-to-list. Part 6. The fan-out that tags everything and expands anything without re-billing completed calls, then the “build a list from a sentence” flow composed on top of it.
6. The AI calendar and digests. Part 4’s second half. Now that publishing is durable, the calendar planner and the scheduled digests have solid ground to schedule onto.
7. The cost dashboard, then the agent. Part 7’s dashboard makes “cheap” visible: tokens, cache-hit rate, budget remaining. I want that governor on-screen before I ship the most token-hungry feature in the whole set. Then, and only then, the agentic capstone.
The rule underneath the whole ordering is that value ships at every step. The publish fix helps users who never touch AI. The link cache is useful before any retrieval sits on it. The dashboard is useful before the agent exists. I’m never in a state where I’ve spent three weeks and have nothing a user can hold.
The Verification Is Where the Trust Actually Lives
The meta-point of the whole series is this one, so I’ll say it flat. You do not trust an autonomous agent because the prompt was good. You trust it because it runs inside guardrails, and because tests prove the properties you care about are there.
So the verification for the agentic capstone, and everything under it, is not an afterthought. It’s the thing that makes any of this shippable:
Vitest unit tests for the activity logic and the cost math. Does the cascade pick the cheap model for planning? Does assertBudget throw at the right ledger sum? Pure functions, no DB, fast.
Temporal replay tests that prove a workflow resumes correctly after a simulated worker crash. This is how I prove, not hope, that a retry doesn’t restart the expensive agent from step zero.
An idempotency test that starts two publish workflows with the same message id and asserts exactly one publish. That’s the double-post race, closed and locked.
Cost-assertion tests, the ones I care about most for the agent. Fail an activity mid-pipeline and assert the already-completed model calls are not re-invoked on retry. Memoization is money, and this test is how I know the money’s saved.
Playwright E2E for the generate to confirm to write flow, so the human-in-the-loop gate works in a real browser and not just in my head.
A security pass before any PR: SSRF on the crawler (everything through safeFetch), IDOR on every workflow write (where: { id, userId }), prompt-injection defaults (artifacts private by default so a poisoned link can’t publish on your behalf), and budget-bypass attempts.
And every schema change (the LinkContent table, the pgvector column, the response cache) goes through the strict additive migration workflow. Hand-written, idempotent, applied to both databases. No prisma db push, no shortcuts. That workflow has broken production before, precisely when someone skipped it.
What I Actually Learned Working With Claude
The lesson closes the loop the series opened. Supervise the agent with structure, let the tests carry the trust, and it turns out that’s also how I’ve been building with Claude the whole time.
You don’t hand an agent a vague goal and hope. You hand it constraints: a budget it can’t exceed, a step count it can’t blow past, a timeout it can’t outlast. Then you make it write the tests that prove those constraints hold, the replay test, the idempotency test, the cost-assertion test. What makes the agent trustworthy is the wall around it, not how nicely I phrased the ask.
And that’s true one level up too, which I didn’t expect going in. Sequencing the build so each step is independently verifiable (the publish fix before the calendar, the cache before retrieval, the dashboard before the agent) is the same move as capping the agent’s steps. Give the work boundaries, make the boundaries checkable, prove them before moving on. Whether the thing on the leash is a research agent or my own plan to build one, the leash is what I trust.
I’ve been brainstorming this out loud for eight posts. The pieces are grounded in code that already exists: the safeFetch guards, the DSL, the crosspost fan-out, the cron that really does risk a double-post. None of it is shipped. All of it is buildable, in the order above, with value at every step. So I’m going to start at step one and build the durable publish fix, because it’s a pure win with no model tokens and it closes a real hazard. I’d love for you to follow along, as I am going to move forward now and implement this. If you’re interested, subscribe!
Six posts in, and I’ve been dancing around a number. Every proposal I’ve sketched (durable generate, the calendar planner, the link crawler, batch enrichment) carries a cost, and the expensive part is never Temporal. It’s the tokens. A worker box and a managed Temporal namespace run me on the order of tens of dollars a month (obviously at this time), flat, whether they process ten jobs or ten thousand. The model bill is the one that scales with usage. It’s also the one that decides whether an AI feature is worth shipping or is a slow-motion budget fire.
So this post is the payoff. Durability isn’t just nice, durability is money. The mechanics I keep reaching for aren’t in the plan because they’re clever. They’re there because each one structurally drives the token bill down, and does it in a way I can measure instead of squint at and hope for. Keeping this cheap relative to the Claude and ChatGPT APIs was never a footnote I tacked onto the series. It’s been the point the whole way through.
None of this is live yet. AI assistance in the product is still marked Coming Soon. This is me brainstorming the cost layer I’d build, and how I’d hand it to Claude to build with me.
The Single Biggest Saver Is Not Re-Paying for Work You Already Did
Start with the one that matters most. Durable memoization.
When a Temporal workflow runs an activity (say, an llmCall that generates a document section) the result gets persisted the moment it completes. If the workflow later fails and retries, it does not re-run that activity. It replays the history, sees the completed result, and moves on.
Think about what that means on a six-step generation pipeline that dies at step five. Without durability, a retry starts over: steps one through four re-invoke the model, and you pay for all four again just to get back to where you already were. With Temporal, the retry re-runs step five and only step five. Steps one through four are already-billed calls that never happen twice.
That’s the difference between retries as a cost multiplier and retries that are free. On a batch enrichment job that fails at item 47 of 50, you re-bill item 47, not items 1 through 46. I wrote a whole post about that in Part 6, and it’s the same mechanic every proposal here leans on. Memoization is why a flaky provider or a worker restart doesn’t torch the budget.
Cheapest Model First, Escalate Only on Demand
The next lever is refusing to reach for the expensive tier by default. The llmCall activity would run a model cascade: cheapest capable tier first (Haiku-class, Flash-class), escalating only when the task actually needs it.
Most of what these features do is mechanical. Tagging a message. Summarizing a row. Extracting a field. That work does not need a frontier model, and paying frontier rates for it is just lighting money on fire. So the cascade defaults to cheap, and it reserves the Opus-class tier for the thing the user explicitly asked for: long-form synthesis, a research draft, a document they told me they want written well.
Paired with that: per-target max_tokens ceilings, so a runaway generation can’t balloon. A message caps around 1k, a list around 4k, a document around 16k. The ceiling isn’t a polite suggestion the model is vaguely aware of. The activity enforces it as a hard cap. So you get the cheap tier by default, a small ceiling per target, and the expensive model only shows up when you asked for it.
A Gate That Throws Before It Spends
Cascades and ceilings keep individual calls cheap. But the scenario that scares me is unbounded spend: a runaway agent loop, or a leaked user key someone else is now happily burning. So the plan puts a budget gate before every call, not after.
An assertBudget activity reads accumulated spend from the AiGeneration ledger and throws a non-retryableBudgetExceeded when the user is over their ceiling. Non-retryable is the important word. A normal failure retries; this one halts the workflow cold, because retrying a budget breach just tries again to spend money you’ve already said no to.
Roughly what I’d hand to Claude as the shape:
// activity: runs on the worker, before any billable llmCall
export async function assertBudget(userId: string, estimate: TokenEstimate): Promise<void> {
const spent = await getSpendThisPeriod(userId); // counts only, from AiGeneration
const remaining = budgetFor(userId) - spent;
if (estimate.maxTokens > remaining) {
// non-retryable: a budget breach must not loop and re-attempt the spend
throw ApplicationFailure.nonRetryable(
`Budget exceeded: need ${estimate.maxTokens}, have ${remaining}`,
"BudgetExceeded",
);
}
}
A workflow calls assertBudget and then llmCall, in that order, every time. Worst case is a bounded overspend of one call, never an open tap.
Never Bill the Same Request Twice
Then there’s the plain-dumb-obvious saver: caching. A response cache keyed by hash(system + prompt + model). If an identical request comes through (same system prompt, same user prompt, same model) it’s served from cache and never re-billed. Regenerate the same summary twice, pay once.
Same idea one layer down for link work: URL-hash dedup. When the crawler from Part 5 fetches and embeds a link, it keys on a hash of the URL. A link shared across ten messages, three docs, and a list row gets fetched, extracted, and embedded exactly once, ever. Embeddings are already orders of magnitude cheaper than generation, so computing them once and caching forever makes the cheapest part of the pipeline round down to nothing.
And to keep from wasting tokens on retries you caused yourself: worker concurrency and task-queue rate caps tuned to stay under provider limits. Blow past a rate limit and you get a 429 storm, and every retry in that storm is wall-clock and, on some providers, tokens down the drain. Obeying the limit is cheaper than fighting it.
One more, because it’s the most satisfying: confirm-before-spend. For the prompt-to-list flow, the workflow generates the schema, then pauses on a Signal and waits. It does not spend a single row-generation token until the user confirms the schema is right. Get the schema wrong and you’ve spent one cheap schema call, not fifty row calls against a shape nobody wanted. That pause is impossible in a serverless handler and native in Temporal.
The Ledger Counts, It Doesn’t Read
Every one of these levers needs a source of truth, and that’s the AiGeneration ledger. Critical design decision: it records counts and status only. Tokens in, tokens out, which model, succeeded or failed. It never stores the prompt text or the model output.
That’s not laziness. One table does three jobs. Privacy, because I’m not warehousing what people wrote or what the model said back. The quota counter, because assertBudget reads its sums off it. And the cost dashboard, because those same rows drive a per-user “AI usage and spend” view: tokens, calls, cache-hit rate, budget remaining. When something looks expensive, the ledger doubles as the debugging trail. You can see that a workflow made forty calls without ever seeing what it said.
Build the Guardrails With Claude, Then Make It Prove Them
Now the part I’ve been wanting to get to, because this is where working with an LLM changed how I’d approach any of it.
Memoization and caching only save money if they’re in the right places. Put a cache breakpoint one step too early and you re-bill everything after it. Miss a memoization boundary and a retry quietly re-runs a paid call. The savings are entirely a function of where the boundaries land, and eyeballing a six-step pipeline for those boundaries is the kind of thing I get wrong.
So the first move is a diagnostic question, not a code request. I’d point Claude at the actual workflow and ask: “walk this pipeline and tell me exactly where a retry would re-bill the model.” Then let its answer drive placement. It reads the activity boundaries, traces the retry path, and tells me which calls are already durably memoized and which ones a mid-pipeline failure would re-invoke. That answer is the map. The caching and memoization go where the map says, not where I guessed.
The second move matters more: make it prove the savings. Memoization you can’t prove is just a hope with good intentions. So I’d have Claude write a Temporal replay test (using TestWorkflowEnvironment) that fails an activity partway through the pipeline and then asserts the already-completed LLM activities are not invoked again on retry.
it("does not re-bill completed LLM activities on retry", async () => {
const llmCall = vi.fn()
.mockResolvedValueOnce("step-1 result")
.mockResolvedValueOnce("step-2 result")
.mockRejectedValueOnce(new Error("provider blip at step 3")) // fail mid-pipeline
.mockResolvedValue("step-3 result (retry)");
await worker.runUntil(client.workflow.execute(generatePipeline, { /* ... */ }));
// steps 1 and 2 completed before the failure, so Temporal replays their
// memoized results and MUST NOT call the model for them again.
const stepsCalled = llmCall.mock.calls.map((c) => c[0].step);
expect(stepsCalled.filter((s) => s === 1)).toHaveLength(1);
expect(stepsCalled.filter((s) => s === 2)).toHaveLength(1);
expect(stepsCalled.filter((s) => s === 3)).toHaveLength(2); // failed once, retried
});
That test is the proof. It fails loudly the day someone refactors the pipeline in a way that breaks memoization and starts silently re-billing steps one and two. The savings stop being a story I tell about the architecture and start being a property CI enforces. Same loop for the response cache: a test that fires two identical requests and asserts the second one hits cache and never reaches the model.
Ask the agent where the money leaks. Let its answer place the guardrails. Then make it write the test that locks them in. That loop works because a cost regression is invisible until the bill shows up weeks later, and a test drags it into the light on the exact commit that caused it, while you can still git blame your way back to it.
Where This Lands
Part 8 is the capstone: the agentic research-to-draft flow, the single most token-hungry thing in the whole set. It’s the reason every governor in this post has to exist before it ships. I’ll also lay out how I’d sequence the entire build, starting from the foundation phase (the Temporal wiring and the shared cost primitives everything else reuses) and working all the way up to the agent, and how I’d verify each piece as it lands.
Every serverless AI feature I’ve built hits the same wall the second someone wants it at scale. “Can it tag all my untagged messages?” Sure, one message. “All of them?” Now we’re talking about a job that runs for minutes, touches hundreds of items, and will absolutely fall over partway through. And when it falls over, the naive version starts from zero and re-bills you for every item it already finished. Nobody wrote that behavior on purpose. It’s just what you get by default, and it’s expensive.
This is the post in the Durable AI series where durability pays off the most. Not “nice to have.” It’s the biggest cost saver in the whole series, and I’m putting that up front so you don’t skim past it.
The Jobs That Don’t Fit in a Request
There’s a whole class of work InterlinedList can’t do today and won’t be able to do inside a Vercel function, no matter how the AI plan lands. It’s the bulk stuff:
“Add 20 rows to this list matching X.”
“Tag every untagged message.”
“Summarize each finished book in my reading list into its own document.”
“Backfill link previews across all my docs.”
What these share is that they’re multi-minute, many-item, and retry-heavy. A serverless handler has a hard duration ceiling and no durable state between invocations. If item 34 of 50 throws a 429 from the model provider, the whole request dies and there’s no clean way to pick up at 35. So you either don’t offer the feature, or you ship a fragile version that quietly loses work and double-charges for the rest.
Today’s ground truth matters here, so let me be precise. Lists are a Subscriber feature. A list’s row data is stored as JSONB in ListDataRow.rowData, validated against a user-defined schema. The validator that gates every row lives in lib/lists/dsl-validator.ts. None of the batch AI machinery below exists yet: this is a brainstorm I’d hand to Claude to build, and AI assistance in the product is still “Coming Soon.” I’m designing it out loud.
The Shape: Fan Out, Cap Concurrency, Track Progress
The design I keep landing on is a generic fan-out workflow. One workflow supervises the whole batch; one item equals one activity. Each activity is independent and idempotent. It does its work, writes its result, and doesn’t care whether its neighbors succeeded or failed.
Temporal is the durable engine underneath, for reasons that are specific and not hand-wavy:
Partial progress is preserved. A completed activity’s result is persisted. When the workflow retries after a failure at item 47 of 50, it re-runs item 47 and only item 47. Items 1 through 46 already succeeded and their results are memoized, so it doesn’t touch them. On a batch of 50 LLM calls where one flakes, you re-bill one item, not fifty. This is the money argument, and I like it because it’s structural. You get it from the shape of the thing, not from remembering to code it.
Concurrency is capped. You don’t fire 50 model calls at once and trigger a 429 storm. A 429 storm is expensive twice over: the retries burn wall-clock, and on some providers they burn tokens too. Cap concurrency to a small number and you respect the provider’s rate limit (and your own BYO-key limit), so the retries never happen at all.
The job survives a worker restart. Deploy mid-batch, crash the worker, whatever. The workflow resumes from where it was, not from zero.
Here’s the constraint made concrete: a generic batchEnrich workflow that fans out over items with a bounded concurrency window.
// Workflow: supervises the batch, never touches the model or the DB directly.
export async function batchEnrich(input: BatchEnrichInput): Promise<void> {
const { items, concurrency } = input; // e.g. concurrency = 5
const running: Promise<void>[] = [];
for (const item of items) {
// enrichItem is an ACTIVITY: its result is durably memoized.
// On a workflow retry, already-completed items are NOT re-invoked.
// Temporal replays their recorded results instead of re-billing them.
const p = enrichItemActivity(item).then(() => {
recordProgress(item.id); // drives the live progress UI
});
running.push(p);
// Cap in-flight work so we never exceed the provider / BYO-key rate limit.
if (running.length >= concurrency) {
await Promise.race(running);
// prune settled promises so the window stays at `concurrency`
}
}
await Promise.all(running);
}
The per-item activity is where the real work (and the idempotency) lives. It reuses code that already ships in the repo:
// Activity: one item. Independent, idempotent, individually retryable.
export async function enrichItemActivity(item: EnrichItem): Promise<void> {
// 1. Cheap-model call for the mechanical part (tag / summarize).
const enriched = await llmCall(item.prompt, { model: "cheap" });
// 2. Validate against the list's real schema before writing anything.
// validateFormData lives in lib/lists/dsl-validator.ts, the same gate
// the app uses for every human-entered row.
const result = validateFormData(item.fields, enriched.rowData);
if (!result.isValid) throw new NonRetryableError(result.errors);
// 3. Write through the existing materialize helpers, not raw Prisma.
// lib/materialize/build-list.ts / build-doc.ts already know how to
// turn structured data into rows and documents with ownership intact.
await writeThroughMaterialize(item.target, enriched);
}
Nothing here reinvents validation or writing. validateFormData is the existing row validator. lib/materialize/build-list.ts and build-doc.ts are the existing builders that turn structured content into lists and documents. The persistence writers I sketched back in the foundation phase, the groundwork everything in this series stands on, handle the actual writes with IDOR checks intact. The workflow’s only job is orchestration: fan out, cap, track, resume.
The Cost Levers, Stacked
Cost is the spine running through this whole series, so let me connect the dots. Three levers push the LLM bill down, and this one feature stacks all three:
Batching. Tagging is mechanical and small. Instead of N calls to tag N messages, one call tags a batch of them: you collapse the per-request overhead and the token duplication of repeating the same system prompt N times over. Fan-out doesn’t have to mean one-model-call-per-item. The fan-out unit can itself be a batch.
A cheap-model default. Tagging and summarizing don’t need a frontier model. They need a competent, fast, cheap one. Claude Haiku 4.5 runs at $1 per million input tokens and $5 per million output; Opus 4.8 is $5 and $25. That’s a 5x swing for mechanical work where the cheap tier does the job fine. The default model for a batch-tag job should be the cheap one, and you escalate only for the rare task that needs it.
No redo on retry. Memoization again: the durable engine never re-invokes an already-billed call on a retry. This is the lever that costs you nothing to pull, because it’s baked into how the workflow resumes.
Put those together and a “tag everything” run over a few hundred messages costs cents instead of dollars, and a mid-run failure costs you one retry instead of a full re-run.
What I’d Tell Claude Before It Writes Anything
There’s a “working with Claude” lesson buried in all this, and honestly it’s why I wanted to write this post at all.
Resumability is a design constraint, not a feature you bolt on later. Ask an agent to “write a batch tagging job” and say nothing else, and you get a for loop that calls the model N times and stores results at the end. It works in the demo. Then it falls over on the first real batch, loses partial work, and re-bills everything on retry. Now you’re spending a day retrofitting durability into code that was never shaped for it, and durability doesn’t retrofit cleanly, because it changes what a “unit of work” even is.
So I state the constraints before the code, right in the prompt:
Each item is an independent, idempotent activity. The workflow must resume mid-batch without re-running finished items: a failure at item 47 of 50 re-processes only item 47. Cap concurrency at N to stay under the provider rate limit. Validate every item against the list schema before writing, and write through the existing materialize helpers, not raw Prisma.
Hand it that, and Claude generates the right thing: the generic batchEnrich workflow, the per-target activity adapters (list rows go through build-list, documents through build-doc, message tags through the tag writer), and the batched prompt that tags many items in one call. The constraints do the architectural work; the agent fills in the wiring. I own the seam (what’s an activity, what’s idempotent, where the concurrency cap lives) and let it own the code.
The failure mode I’m avoiding is the one where the constraint shows up after the code. State “don’t re-bill completed items” up front and it shapes the whole structure. State it after, and you’re rewriting.
The Surface
On the product side this becomes a “run a bulk action” panel: pick a target (a list, your messages, a set of docs), describe the enrichment, and watch live progress as items complete. Cancel is a Temporal Signal, a real mid-flight stop rather than a “please ignore the results” fiction. The workflow receives the signal, stops fanning out new items, lets the in-flight ones finish or abort, and reports what it got done. Progress and cancel both fall out of a workflow that already tracks per-item state.
Part 7 is the payoff post: the cost governor, where all of these levers get formalized into a budget ledger, a model cascade, and the response cache that make “cheap enough to ship” measurable instead of hoped-for.
You must be logged in to post a comment.