Tag: programming

Where Does Temporal Even Run When You’re All-In on Vercel?

Part 2 of 8: Durable AI for InterlinedList

In Part 1 I talked myself into Temporal as the orchestration layer for the AI features I want to build. Durable memoization, a model cascade, a hard budget gate: all the mechanics that push the LLM bill down. Good story. Then I hit the question that stops most “let’s just add Temporal” plans cold, and it stopped me too.

Where does it actually run?

InterlinedList is 100% serverless on Vercel. Every route is a function that spins up, does its thing, and gets torn down. That model is wonderful right up until you need something that never stops running. And Temporal needs exactly that.

The Two Parts That Don’t Fit

Temporal has two moving pieces, and neither of them is a serverless function.

The first is the Temporal Service, the durable-state backend. It’s the thing that remembers where every workflow is, what’s completed, what’s pending, and what to retry. It’s stateful infrastructure. You don’t invoke it and let it die.

The second is the Worker. A worker is a long-lived process that polls a task queue, picks up work, and runs your activities. Polls it continuously. As in, forever. A Vercel function that shuts down after a request cannot be a Temporal worker, because the polling loop dies the instant the function returns. You can’t flag your way out of this in a config file. It’s a straight-up mismatch between “poll a queue for hours” and “respond in seconds then vanish.”

So for a while this felt like a dead end. Adding Temporal seemed to mean un-serverless-ing the whole app, and I did not build InterlinedList on Vercel just to go stand up a fleet of always-on boxes.

One New Always-On Thing. That’s It.

The resolution I kept circling back to is smaller than the fear suggested. I introduce exactly one new always-on component: a small @temporalio/worker process running somewhere cheap and boring, like Fly.io, Railway, or a small VM. Something in the five-to-twenty-dollars-a-month range. Alongside it, either self-hosted Temporal OSS or a low-tier Temporal Cloud namespace for the service.

That’s the entire footprint. One worker, one service. Vercel doesn’t change shape at all.

What does change is what the Vercel routes do. They stop doing work and become thin clients. Authenticate with the getCurrentUser / getCurrentUserOrSyncToken helpers I already have, gate with isSubscriber, start a workflow, then hand the UI a workflow id to poll. The route never does the long-running work itself. It just points at it.

The route sketch is almost embarrassingly short:

export async function POST(request: NextRequest) {
  const user = await getCurrentUserOrSyncToken(request);
  if (!user) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
  if (!isSubscriber(user.customerStatus)) return NextResponse.json({ error: "Forbidden" }, { status: 403 });

  const handle = await client.workflow.start(generateArtifactWorkflow, {
    taskQueue: "ai",
    workflowId: `gen-${user.id}-${crypto.randomUUID()}`,
    args: [{ userId: user.id, prompt }],
  });
  return NextResponse.json({ workflowId: handle.workflowId });
}

Auth, gate, start the workflow, return the id. That 16k-token document generation that would blow right past the function’s duration budget? It never runs in the function. The function kicks it off and gets out of the way.

The Reuse Win: No Second Codebase

This is the part that made me stop worrying. The worker is not a new codebase. It imports the same lib/ code the app already runs.

The activities (the actual units of work Temporal executes) call into modules that already exist or are already planned. safeFetch from lib/security/ssrf.ts for any link crawling, so the SSRF guards apply exactly as they do today. detectLinks from lib/messages/link-detector.ts. The post-status.ts modules under lib/bluesky/ and lib/mastodon/ for cross-post fan-out. The DSL parser and validator in lib/lists/dsl-*.ts for anything list-shaped. And the AI writer library from the companion genai plan, lib/ai/, which doesn’t exist yet but will be wrapped as activities rather than rewritten.

Zero business-logic duplication. That matters more than it sounds. It means the IDOR checks and subscription gating I already trust stay exactly where they live. A worker activity that writes a list row runs through the same where: { id, userId } ownership filter as the web route. I’m not maintaining two versions of “can this user do this” and praying they stay in sync. There’s one version, and the worker just calls it.

What the Foundation Actually Contains

Before any feature ships, there’s a foundation phase, the groundwork everything else in the plan stands on. It’s the worker, the service, the client wiring, and then a small set of cost-control primitives that every later proposal reuses:

  • An llmCall activity with a model-cascade policy (cheap model first, escalate only when the work demands it) and per-target max_tokens ceilings: roughly 1k for a message, 4k for a list, 16k for a document.
  • An assertBudget activity that reads spend before it lets a call through, so a runaway job throws instead of burning credits.
  • A response cache keyed by hash(system + prompt + model), so identical requests never re-bill.
  • Task queues with rate limits tuned to whatever the provider or the user’s BYO key allows.

Net-new dependencies are modest: @temporalio/client on the Vercel side, the @temporalio/worker / workflow / activity trio on the worker host, and one LLM SDK. That’s the whole shopping list. (AI assistance in the product is still Coming Soon. This is me brainstorming the machine that would power it, not shipping it.)

Scaffold the Unfamiliar, Own the Verification

I’ve been doing a lot of this alongside Claude lately, and one lesson keeps repeating.

A durable-execution worker is new territory for me. When something’s unfamiliar the instinct is either to avoid it or to throw the whole thing at the model and hope. Both are wrong. What works is to spec the seams and delegate the boilerplate.

I give Claude the hard constraints, not vibes. “The app is serverless on Vercel and stays that way. Reuse this lib/ code, here are the exact modules. Here are the auth and gating seams. The worker imports, it does not reimplement.” With those rails, Claude is great at generating the worker scaffold and the client wiring, the boilerplate I’d get subtly wrong on the first pass because the framework is new to me.

But I do not let it hand me generated infrastructure and call it done. I own correctness, and for a durable workflow, correctness has a specific proof: a Temporal replay test using TestWorkflowEnvironment that simulates a worker crash mid-workflow and asserts the workflow resumes deterministically, picking up from the last completed activity instead of re-running (and re-billing) the ones already done.

That test does the real work. It’s how “a retry doesn’t re-pay for the LLM call” stops being a claim in a blog post and turns into something I’ve watched go green in CI. The model writes the code I don’t know how to write yet, and I write the test that proves it’s right.

With the foundation standing and provably resumable, Part 3 turns it into the first real feature: a Generate button that survives a timeout and asks before it writes a single thing.

Adron brainstorming and working on InterlinedList.

I’ve Been Brainstorming Durable AI for InterlinedList, and It Kept Pointing at Temporal

Part 1 of 8: Durable AI for InterlinedList

I’ve been chewing on a question for weeks. If InterlinedList is going to have real AI features (the “Coming Soon” ones on the pricing page, the ones I actually want to use myself) what does the plumbing behind them look like? Not the prompts. The plumbing. And every time I sketched it out, I hit the same wall in the same four places.

So I did the thing I’d tell anyone else to do. I stopped sketching in a vacuum and pointed Claude at the actual repository.

The Wall Is Serverless, and I Put It There

InterlinedList is 100% serverless on Vercel. That was a deliberate choice and mostly a great one. It’s also the thing shaping every AI idea I have, whether I like it or not.

A serverless function has a duration ceiling. It has no durable memory between requests, it can’t pause and wait for you to click “yes,” and when it fails partway through expensive work, it has no idea what it already paid for. For a request/response app that serves pages, none of that matters. For anything that calls a language model, all of it matters. A lot.

Here are the four places I kept getting stopped.

Scheduling is already fragile, and this one isn’t hypothetical. InterlinedList has scheduled publishing today (a Subscriber feature). The cron at app/api/cron/publish-scheduled-messages/route.ts runs every minute. I checked vercel.json, it’s a literal * * * * *. It fans a due post out to Bluesky, Mastodon, LinkedIn, and X, then flips the message out of “scheduled” state. There’s no lock and no idempotency key. So if the cross-posts succeed but that final UPDATE fails (a reaped Neon connection, a cold-start hiccup) the message stays “due” and the next tick republishes it. To every platform. Again. That’s a slow-moving double-post waiting for a bad minute, a bug I already own, sitting in production, with zero AI anywhere near it.

Long generation doesn’t fit. A 16k-token document, streamed out of a model, will blow past a serverless function’s budget before it finishes. There’s no clean way to say “keep going, I’ll be back.”

Batch and agentic work has nowhere to live. “Tag every untagged message.” “Add twenty rows to this list matching these criteria.” “Research my saved links and draft a doc.” Those are multi-minute, multi-step, retry-heavy jobs. A function that dies at ninety seconds is the wrong shape for all of them.

And there’s no cost governor. This is the one that kept me up. Every retry re-bills the model. There’s no budget ceiling, no cheap-model-first policy, no response cache, no dedup. One runaway loop and you’ve torched real money.

The Expensive Line Item Is Never the Infrastructure

Once I reframed the money question, the design mostly wrote itself. This reframe runs under the whole series, so it’s worth stating plainly.

The expensive thing about AI features is tokens, not servers. Basically never servers.

A durable-workflow engine (Temporal, in this case) costs on the order of tens of dollars a month, flat. One small always-on worker, maybe a managed namespace. That number does not move when your users get busy. The model bill does, and it moves a lot.

So the question isn’t “can I afford Temporal.” It’s “does Temporal make the model bill smaller.” And the reason it kept showing up in my brainstorm is that its mechanics do that, structurally:

  • Durable memoization. A completed step is persisted. When a six-step pipeline fails at step five, the retry re-runs step five, not the four already-billed model calls before it. A retry never re-invokes an LLM call you already paid for. This is the single biggest saver in the whole list.
  • Model cascade. Cheap model first (Haiku, Flash) and escalate to an expensive tier only when the work demands it, or the user explicitly asks for long-form.
  • Batching. Tag fifty messages in one call instead of fifty calls. The per-request overhead and the duplicated context collapse.
  • Rate-limit obedience. Cap concurrency to stay under provider limits, so you don’t trip a 429 storm whose retries waste both wall-clock and, on some providers, tokens.
  • Response and URL dedup. Identical prompts get served from cache. A link that’s already been fetched isn’t fetched again. An embedding is computed once, ever.
  • A hard token-budget gate. Check spend before each call and refuse to exceed a per-user ceiling. A leaked key or a runaway agent can’t burn unbounded credits.
  • Confirm-before-spend. Pause and wait for the user to approve the plan before generating the expensive part. You don’t pay to build fifty list rows against a schema the model guessed wrong.

Every one of those pushes the bill down. The infrastructure that enables them is a rounding error against what it saves. And I want to be clear that “cheap as possible, relative to running raw Claude or ChatGPT calls” is why the architecture looks the way it does in the first place. I didn’t bolt cost control on at the end. I designed around it, and everything else grew out of that constraint.

Brainstorm Against Your Real Code, Not a Whiteboard

Now the part about working with an LLM to build this, because that’s what this series is modeling.

I didn’t design this on a whiteboard. I opened Claude, gave it the real repo, and asked it to map the idea onto the code that already exists. It found the every-minute cron and traced the exact failure path where the double-post lives. It pointed at lib/security/ssrf.ts and the safeFetch guard that any link-crawling feature has to route through, and at the link detector and metadata fetcher already sitting in lib/messages/. It also noticed there’s no lib/ai/ directory yet. Nothing generative is built, which lines up with AI being “Coming Soon” rather than shipped.

The output of that session was a grounded proposal (temporal-solutions.md) that complements an earlier BYO-key and MCP plan I’d written. That earlier plan covered what the AI produces and who pays for the tokens. It said nothing about orchestration, durability, or cost. This new one fills that gap, and it fills it against real file paths, not imagined ones.

Steal this if it’s useful. Don’t design architecture in a vacuum and then go hunting for where it fits. Let the agent read the actual repository first. Have it map your idea onto the code that exists, name the real hazards, and pressure-test it into something concrete before a single line of implementation. Explore first. The proposal you get back is grounded because the agent actually went and looked.

There’s a catch I want to name up front, because I try not to write brochure copy. Temporal has parts that fundamentally cannot run on Vercel serverless: a durable-state backend and long-lived worker processes that poll continuously. That’s a real new piece of always-on infrastructure in an app I deliberately built to have none. It’s the scariest unknown in the whole plan, and I’m not going to pretend it away.

Next post, I go straight at it: where Temporal actually runs when you’re all-in on serverless, and how the Vercel side stays a three-line client that just kicks off a workflow.

Adron brainstorming and working on InterlinedList.

That Gap Is Still Widening, But The Bottleneck Was Never Engineering

In my previous post, The Gap Is Widening and It’s Not Slowing Down, I focused on the growing divide between individuals and teams that have embraced Generative AI and those that have not. That divide is real, measurable, and growing faster than many people seem willing to acknowledge. I’m not walking any of that back.

But after watching organizations over the last year attempt to integrate AI into their engineering practices, I’m increasingly convinced the widening gap isn’t actually about AI – at least not entirely. The technology is only exposing something that has existed for decades. The bottleneck was never engineering. It was never software development. It was never the people building things.

The bottleneck has almost always been the machinery *surrounding* the people building things: the approvals, the reporting structures, the committees, the prioritization processes, the disconnected leadership layers, the bureaucracy, the politics – the endless collection of organizational systems that somehow manage to consume vast amounts of energy while producing remarkably little forward progress. Generative AI didn’t create this problem. It simply made it impossible to hide.

Individual Engineers Are Experiencing a Massive Productivity Expansion

There is very little debate left about whether Generative AI increases the productivity of individual contributors. We’ve moved well past that question. The debate today isn’t whether productivity gains exist – it’s about how *much* gain exists and, more importantly, who is actually capable of capturing it.

Engineers today can create prototypes in hours that once required days or weeks. Architectural alternatives can be explored in an afternoon instead of consuming entire sprints of spike work. Documentation can be generated, revised, and maintained at speeds that would have seemed unrealistic even five years ago. Testing frameworks, infrastructure automation, deployment pipelines, and application scaffolding can all be stood up dramatically faster than before.

Even beyond code generation, AI has become an accelerator for thinking itself. It provides rapid feedback loops, architectural critiques, alternative approaches, and research capabilities that allow engineers to move through uncertainty faster than they ever have before. An engineer operating effectively with modern tooling can often accomplish what previously required several engineers. A small team can frequently deliver what once demanded a much larger one. This isn’t speculation anymore – we’re watching it happen every day.

Yet despite these gains, many organizations report only marginal improvements in overall delivery speed. Why? Because software development was never the slowest part of the process.

The Bottleneck Was Hiding Somewhere Else

For years, organizations convinced themselves that software engineers were the constraint. The logic seemed simple enough: if projects are late, development must be slow; if features take too long, engineering capacity must be insufficient; if delivery struggles, more developers must be needed. So organizations hired more engineers, added process around them, and watched delivery timelines stay roughly the same.

Then AI dramatically accelerated the development side of the equation – and something interesting happened. The overall system didn’t accelerate proportionally. Instead, the delays became easier to identify. Features completed quickly sat for weeks waiting for approval. Product decisions that took months were backed by implementation work that took days. Architecture reviews became calendar-management exercises instead of engineering exercises. Governance processes consumed more time than development itself, and procurement delays stalled technology adoption before it could even begin.

The moment development became faster, every other inefficiency suddenly became visible. The tide went out and revealed the rocks – and there were far more rocks than most organizations were prepared to acknowledge.

We’ve Been Trying To Solve This Problem For Over A Century

One of the most frustrating aspects of this situation is that the underlying problem isn’t new. Some of the most influential thinkers in management spent their entire careers attempting to solve exactly these issues, and we largely ignored them.

Chief among them was W. Edwards Deming. His work fundamentally reshaped manufacturing, quality management, and organizational thinking throughout the twentieth century. His influence helped transform post-war Japanese manufacturing and directly shaped the practices that would eventually become Lean methodology and the Toyota Production System. One of Deming’s most important observations was that organizations routinely blame individuals for failures that are actually caused by systems — the worker gets blamed, the engineer gets blamed, the frontline employee gets blamed, while the actual system producing poor outcomes remains untouched and unexamined.

Deming repeatedly argued that management’s primary responsibility was improving the system itself – not creating more reports, not creating more oversight, not creating more bureaucracy, but actually improving the system. This remains one of the most ignored lessons in modern business. When productivity stalls, organizations add process. When communication breaks down, they add meetings. When delivery slows, they add approvals. When uncertainty increases, they add governance layers. The response is almost always additional complexity, rarely simplification – yet simplification is often exactly what’s needed.

The Toyota Way Was Never About Manufacturing

One of the most persistently misunderstood management concepts in business is the Toyota Production System and the principles described in *The Toyota Way*. Organizations study Toyota and immediately focus on manufacturing techniques, kanban boards, and production flow. That completely misses the point.

The true innovation was not manufacturing. It was the relentless pursuit of removing waste from systems. Toyota became exceptional because it continuously questioned every activity that consumed effort without creating value – every unnecessary handoff, every unnecessary delay, every unnecessary approval, every unnecessary process step. Everything was examined through the lens of whether it actually moved something of value forward. If it didn’t, it was a candidate for elimination.

Modern software organizations often claim to embrace these ideas while operating with approval chains that require six or seven layers of sign-off, organizational structures where decisions travel further than the code itself, and workflows designed to optimize reporting while actively damaging delivery speed. The language of Lean has become extremely popular in tech. The discipline required to actually implement it remains genuinely rare. There’s a meaningful difference between saying “we practice continuous improvement” and operating a system that systematically identifies and eliminates its own waste – most organizations are doing the former and calling it the latter.

AI Is Exposing Management Debt

The software industry talks constantly about technical debt, and rightly so – I’ve spent decades fighting it. But I’m increasingly convinced that many organizations suffer more from *management debt* than technical debt, and management debt is considerably harder to see from the inside.

Management debt accumulates when organizations create layers of process that never get removed. It accumulates when reporting structures expand indefinitely, when approval chains grow with every re-org, when every novel problem gets solved by introducing another committee, another meeting, another workflow, or another governance layer. Over time these accumulate into a dense friction system that surrounds every team trying to build and ship something. Unlike technical debt, management debt is often invisible to leadership because leadership frequently created it – and organizations don’t typically build mechanisms for evaluating whether the management decisions made five years ago are still earning their overhead.

Generative AI is now exposing these accumulated liabilities with uncomfortable clarity. If engineers can produce ten times more output and delivery only improves ten percent, leadership should not be asking what’s wrong with the engineers. They should be asking what’s wrong with the system. The answer may be uncomfortable. It may involve examining years of accumulated organizational decisions, questioning structures that have become politically entrenched, and acknowledging that the bureaucracy itself is the liability. But that’s where the actual solution lives.

Systemic Thinking Matters More Than Ever

One of the most valuable disciplines organizations can adopt today is genuine systemic thinking – not as a framework to be installed and presented to the board, but as an actual way of seeing how the organization produces its outputs.

The reason it matters comes down to a simple but uncomfortable idea: organizations are systems, and systems produce exactly what they are designed to produce. Not what leadership intends. Not what the org chart implies. What the actual system, with its real incentives and real workflows, is built to produce. Many organizations want innovation while designing systems optimized for risk avoidance. They want speed while designing systems that optimize for approval coverage. They want accountability while designing systems optimized for blame diffusion. They want creativity while designing systems that reward conformity and punish variance.

The outputs shouldn’t be surprising – the system is behaving exactly as designed. Generative AI doesn’t alter this reality. If anything, it amplifies it. The faster individual contributors become, the more visible systemic dysfunction becomes. You can’t paper over a broken approval process with faster code generation. You just end up with more finished work sitting in queues.

The Competitive Advantage Isn’t AI

Here’s something I genuinely believe will be borne out over the next five years: the next generation of competitive advantage is unlikely to come from simply adopting AI. Everyone will eventually have access to similar models. Everyone will have copilots, agents, and increasingly capable automation. The models will keep getting better and access will continue to become more democratic. Access is not a moat.

The differentiator will be organizational capability – specifically, whether the organization can actually move. Can it make decisions quickly? Can it remove friction from its own processes? Can it empower teams to act without running everything through three layers of approval? Can it identify waste and eliminate it rather than building process around it? Organizations that can answer yes to these questions will compound the productivity gains AI provides and experience something genuinely transformative. Organizations that answer no will continue wondering why expensive AI investments fail to produce the results they see in press releases and conference talks – and they’ll blame the technology.

The Answers Already Exist

What’s remarkable about all of this is that very few of these ideas are new. Deming wrote extensively about them. Lean practitioners have written extensively about them. Systems thinkers like Peter Senge have written extensively about them. Toyota demonstrated them repeatedly over decades. The playbook already exists. It just requires organizational will to actually use it.

Generative AI simply raises the stakes. For decades, organizations could survive despite bureaucratic inefficiencies because software creation itself was difficult enough that organizational dysfunction stayed hidden behind the sheer complexity of engineering work. That cover is disappearing fast. The engineering side of the equation is accelerating rapidly, and the remaining constraints are becoming impossible to ignore.

If I had to estimate — and I’m willing to commit to this number — I’d say the overwhelming majority of organizations, probably 90% or more, still carry enough management debt, process debt, and bureaucratic drag to prevent them from realizing even half of the value Generative AI could deliver. The technology is arriving right on schedule. The organizations are not.

Organizations that fail to introspect, simplify, and dismantle their accumulated management structures will realize only a fraction of what’s possible. Organizations that embrace systems thinking, Lean principles, continuous improvement, and genuine organizational simplification will unlock extraordinary advantages — not because AI magically transformed their business, but because they finally removed the barriers that had been slowing them down all along.

The gap is widening. But the bottleneck was never engineering. It was management. And management now has nowhere left to hide.

Further Reading

These are the thinkers and resources worth going deep on if you want to move beyond reading about these ideas and actually do something about them.

  • W. Edwards Deming — Start with his Fourteen Points for Management, then read *Out of the Crisis*. His work is the foundation for almost everything else on this list.
  • The Deming Institute — The most accessible ongoing resource for understanding and applying Deming’s system of profound knowledge in modern organizations.
  • *The Toyota Way* by Jeffrey Liker — The definitive English-language treatment of Toyota’s actual management philosophy, not just the production tools. The tools without the philosophy are just theater.
  • Toyota Production System — Understanding the origins and evolution of TPS is worthwhile context before diving into the derivative frameworks that have followed it.
  • Lean Enterprise Institute — Practical, applied Lean thinking for people who want to actually implement rather than just read about it.
  • *The Fifth Discipline* by Peter Senge — The foundational text on systems thinking in organizations. If you read one book from this list, make it this one.
  • *The Goal* by Eliyahu Goldratt — Theory of Constraints explained through a novel. Surprisingly readable and genuinely transformative for how you see bottlenecks. The production setting feels dated; the ideas do not.
  • Kaizen / Continuous Improvement — Understanding what kaizen actually means in practice versus how the word gets casually deployed in tech organizations is worth the time. The gap between those two things is significant.

“Loop Engineering” Is Mostly Just Broken SDLC Wearing a Costume

I’ve been watching the “Loop Engineering” conversation build up steam for a while now and I keep landing in the same spot: I’m not buying it. Not the way it’s being sold, anyway.

The pitch is seductive. Wrap the LLM in a loop — plan, act, observe, correct, repeat — and you get an autonomous-ish thing that grinds away at your problem until it’s done. Everybody nods. It sounds rigorous. It sounds like engineering. And that’s exactly the part that’s bugging me, because the more of these “loop” architectures I look at, the more they look like something I’ve seen before. They look like the same slow, cumbersome, ceremony-laden SDLC that companies have been stuck in for thirty years — except now we’ve bolted it onto a language model and called it innovation.

Let me be blunt about the thing I actually think is happening here. We are not designing new ways to work with LLMs. We are retrofitting the broken software development lifecycle we already had onto a brand new kind of tool, and then acting surprised when the result is convoluted and bloated. The loop isn’t a breakthrough. In a lot of shops it’s a reskin of the ticket-grooming, status-meeting, hand-off-and-wait machine that made software slow in the first place. Largely, it defeats the entire advantage of using an LLM in the first place.

The tell: it’s ceremony, not capability

Here’s what tips me off. Go look at a bunch of these “agentic loop” designs in some of the more advanced companies – the ones that are supposedly doing this right – and count the steps. Plan the plan. Break the plan into subtasks. Score the subtasks. Route the subtasks. Re-plan when a subtask fails. Reflect on the reflection. Summarize the reflection into a memory. Retrieve the memory to plan the next plan.

Squint! That’s not a novel machine intelligence workflow. That’s Scrum with a transformer in the standup. It’s the same “process as a substitute for thinking” instinct that gave us story points and RACI charts and forty-five minute refinement meetings. We took a slow, human, coordination-heavy process – one that mostly exists because humans forget things, go home at night, and don’t share memory – and we handed it, wholesale, to a system that doesn’t forget in the same way, doesn’t go home, and can share state instantly.

The loop, in a huge number of cases, is coordination overhead that the model doesn’t actually need. We’re paying for it in tokens, latency, and complexity, and calling the bill “engineering rigor.”

Loops exist to cover for things LLMs shouldn’t need covered

Programmer looks at the loop getting errors.
Programmer looks at the loop getting errors and the tears and pain of the absurdity of it is driven home! The horror, the nightmare!

A loop is fundamentally an error-correction and coordination structure. You loop when you can’t get it right in one pass and you have no better way to make forward progress than to try, check, and try again. That’s a completely reasonable thing to do sometimes. But notice why the classic SDLC is so loopy: it’s loopy because the humans and systems in it have terrible, lossy interfaces to each other. Requirements get garbled on the way to the dev. Context evaporates between the ticket and the code. Nobody can see the whole thing at once, so we iterate blindly and use process to catch the drops.

When you lift that structure and drop it onto an LLM, you inherit all of those assumptions – lossy hand-offs, missing context, blind iteration – even when they no longer apply. The model can hold enormous context. It can be given the whole picture at once. It can be handed clean, structured inputs instead of a garbled ticket. So a lot of the loop is there to solve a problem you’ve already got the tools to eliminate. You’re building a bucket brigade next to a working fire hose.

That’s the reframe I want people chewing on: don’t fit the LLM to the broken SDLC. Fix the SDLC around what the LLM is actually good at, and a lot of the loop – and the SDLC – just disappears. Not all of it — some feedback is real and necessary — but a lot of it goes away.

So what do you build instead? Workflows, not loops.

A workflow is a directed thing. It has a shape. It moves from a known input to a known output through steps that each do one clear job, and it only bends back on itself where a real signal says it must. Not on a fixed “reflect every turn” cadence because the architecture diagram had a box for it. Here are the approaches I’d actually reach for, and roughly in the order I’d reach for them.

1. Front-load context so the first pass is the good pass

The single biggest source of looping is a bad first attempt caused by starved input – a barren prompt that is missing context, scope, and specifics. So stop starving it. Instead of a thin prompt and a correction loop to claw the quality back, spend your engineering effort upfront assembling everything the model needs: the relevant code, the schemas, the conventions, the prior decisions, the actual constraints. Curate it. Structure it. Hand the model the whole board. Give it a specific thing to do with plenty of reference (MCP/RAG, etc) to get what it needs to get the job done right from inception.

This is a workflow move, not a loop move. You’re not iterating toward context — you’re delivering it before step one. The payoff is enormous, because every loop you avoid is latency and tokens and a chance to go off the rails that you never spent. A well-fed single pass beats a starved five-pass loop most of the time, and it’s cheaper and easier to reason about. Put the work where it compounds: the input.

2. Decompose along data flow, not along a status board

When a task genuinely is too big for one pass, the instinct from SDLC-brain is to break it into “tickets” and manage them in a loop. Don’t. Break it along the data instead – a pipeline where each stage has a typed input and a typed output, and stages connect because one’s output is literally the next one’s input.

Extract, then transform, then validate, then render. Parse, then plan, then generate, then check. Each stage is a small, boring, testable unit that does one thing to a known input. This is the old Unix-pipe wisdom, and it holds up beautifully with LLMs: small components with sharp interfaces that you can compose, test, and swap. The magic is that a clean pipeline removes the reason to loop – you’re not re-planning the whole job when stage three hiccups, you’re re-running stage three. The blast radius of a failure is one stage, not the entire task. That’s the difference between a workflow and a loop: the workflow contains failure; the loop lets it slosh around the whole system.

3. Make feedback event-driven, not clock-driven

Here’s where I think the loop crowd goes most wrong. In a canonical loop, you reflect and re-plan every iteration, on a cadence, whether or not anything happened worth reflecting on. That’s the transformer equivalent of a daily standup where nothing changed but everyone talks anyway. Pure ceremony.

Flip it. Don’t loop on a clock – react to events. Wire the model into a workflow where a specific, real signal triggers a specific corrective action. Tests failed? Route the failing output and the error back for a targeted fix – not a full re-plan, just “fix this.” Schema validation rejected the payload? Send it back with the exact violation. A confidence or a guard check tripped? Escalate that one thing. Everything else flows straight through.

The behavior looks loop-like from a distance, sure – things sometimes go back around. But the structure is completely different, and the difference is the whole point. Correction happens because something concrete demanded it, scoped to exactly what broke, instead of on a blind fixed schedule that burns tokens re-litigating work that was already fine. Nine times out of ten nothing needs to go back, and your workflow should sail straight through when that’s the case.

4. Push determinism to the edges and let the model do the fuzzy middle

A ton of what gets stuffed inside these loops is stuff the model has no business doing repeatedly – running code, hitting an API, checking a value against a rule, formatting an output. Every time you make the LLM babysit that inside a reasoning loop, you’ve added a slow, nondeterministic, expensive step to do a job that a plain function does perfectly, instantly, and the same way every time.

So carve it out. Let deterministic code own everything that can be deterministic: the tool calls, the validation, the I/O, the formatting, the branching on hard rules. Let the model own the genuinely fuzzy judgment in the middle – the part that actually needs a language model. When you draw that line cleanly, the “loop” collapses into a mostly-straight workflow with the LLM as one well-scoped component inside a larger deterministic system, instead of the LLM being the anxious general contractor re-checking every subcontractor’s work on every pass. Less looping, more determinism, and the model spends its cycles on the one thing only it can do.

Pulling it together

Put those four together and look at the shape you get. You front-load context so the first pass lands. You decompose along data flow so failures stay contained. You make correction event-driven so you only bend back when something real demands it. And you push everything deterministic out to the edges so the model isn’t looping over work a function should own. What’s left is a workflow – directed, inspectable, cheap, testable – with the LLM doing the fuzzy judgment it’s uniquely good at and nothing else.

Compare that to the canonical loop: an undirected grind, re-planning and re-reflecting on a cadence, coordination overhead standing in for capability, the whole thing shaped by the assumption that every hand-off is lossy and every input is garbled – assumptions that came straight out of the broken SDLC and mostly don’t apply here.

I want to be fair about it: loops aren’t always wrong. There are open-ended, genuinely exploratory problems where you can’t shape the path in advance and try-check-try is honestly the best you’ve got. Fine. But that’s the exception, and right now the industry is treating it as the default. We’re reaching for the loop reflexively because it feels like the rigorous, grown-up, “real engineering” thing to do – when a lot of the time it’s just the old ceremony in new clothes.

There’s more going on with LLMs than the loop. A lot more. The loop is one tool, and it’s become a bit of a security blanket for people who’d rather port their existing broken process than sit down and design a new one. My request is simple: before you wrap your model in yet another plan-act-reflect grinder, ask what the loop is actually for in your case. If the honest answer is “to cover for lossy hand-offs and missing context,” then you don’t have a loop problem. You have an SDLC you never cleaned up – and the fix is a workflow, not another lap.

Additive vs. Mutative vs. Destructive Code Changes (and Why AI Agents Love the Wrong One at 2:13AM)

There’s a particular kind of pain that only software developers know.

Not the “production is down” kind of pain.

Not the “we deployed on Friday” kind of pain.

No, I mean the slow creeping dread of pulling the latest changes, running the tests, and realizing the codebase has been “improved” in a way that feels like someone rearranged your entire kitchen… but left all the knives on the floor.

And lately, this pain has been supercharged by AI tooling. Cursor, Claude, Copilot, Gemini, ChatGPT-driven agents, whatever your poison is this week, all share a similar behavioral pattern:

They can produce a stunning amount of output at an impressive speed… while quietly reshaping your system into something that looks correct but behaves like an alien artifact from a parallel universe.

The reason is simple: AI agents don’t “change code” the way humans do. They don’t naturally respect boundaries unless you explicitly enforce them. They operate like an overly enthusiastic intern with root access and no fear of consequences. To understand why this happens, we need to talk about the different types of code changes, and how AI tooling tends to drift toward the most dangerous ones.

So let’s name the beasts.

The Four (Actually Six) Types of Code Changes

Continue reading “Additive vs. Mutative vs. Destructive Code Changes (and Why AI Agents Love the Wrong One at 2:13AM)” →