debug time

Professional software developer / hobbyist street photographer

Make invalid states unrepresentable (for your agents)

This blog post was written by me, a human (as evidenced by how long it takes to explain a simple idea). If you want to skip straight to the main idea, scroll to The Point.

There is an old saying: make invalid states unrepresentable.

Like all maxims in computer science, one could debate its usefulness. Maybe you read it and think: Yes! Exactly! Or maybe you read it and recall a time as a junior dev, when a senior dev ruined your Friday by commenting on your PR with these four words.

Regardless, as a guiding principle, it is certainly useful. Entire classes of bugs can be eliminated from programs, just by designing your solution in a way that certain failure states are not merely edge cases but logically impossible.

Lately I have noticed that this approach is very effective when applied to agents, massively increasing their reliability for many tasks. I will detail the pattern below.

First, some old-school code examples. Feel free to skip them if you already get the gist.

Some (contrived) examples

Ways you might “make invalid states unrepresentable”:

Maybe those examples are obvious, boring, even common sense. So let’s move on to agents.

“Make no mistakes”

I’d say there are two general approaches when designing LLM-powered solutions:

  1. The code drives the LLM.
  2. The LLM drives the code.

The former was common in prehistoric times (~2023). Think: langchain, semantic kernel, old-school OpenAI SDK. In this approach, you write “traditional” code, which calls into LLMs when it’s time to make a decision. You control the loop, you control the… well, the control flow. Your code drives the LLM.

The latter is becoming the norm, now that everything is “agentic”. Think: Claude Code, OpenClaw, the latest harness-du-jour, etc. In this approach, the LLM operates in a harness (as an “agent”) with lots of tools at its disposal. You may provide it with code it can run; whether it actually invokes the code is up to the agent. The agent has free rein, and you let it loose on a task. The agent owns the loop, decides what to do, decides when to stop. There isn’t strictly a “program” being executed; at each step, the next step is decided on the fly by the agent. The LLM drives your code, invokes your tools… if it chooses.

Apologies for the Agents 101. My audience surely is familiar with these concepts. But I wanted to outline it explicitly, to set up this thought experiment:

Say you wanted to auto-generate a technical wiki for any given codebase. Basically, implement DeepWiki. How would you do this?

Well, first let’s define what a “wiki” is. For our purposes, an example “wiki” created by our system might be a folder that looks like this:

wiki/
  OVERVIEW.md
  control-plane/
    OVERVIEW.md
    running-locally.md
    system-architecture.md
    plugin-system.md
  user-management-service/
    OVERVIEW.md
    account-creation-flow.md
    ...
  service-lifecycle/
  ...

On top of that, let’s say wiki generation is configurable. A config file for our wiki-generator system might look like this:

# wiki-config.yaml
max_sections: 12
max_pages_per_section: 3

# These sections are always generated, forcefully:
default_sections:
  - name: Application Lifecycle
    description: Explains the startup, shutdown, and deployment process

Basically, we may enforce some parameters (max sections, max pages per section) and allow certain sections to be user-defined such that the generated wiki will always include them (not leaving it up to the LLM to decide).

First, let’s consider the pre-agentic approach, where code drives the LLM. This is relatively straightforward. I can tell you from experience: eventually you will settle on a solution like this, using something like the OpenAI SDK or PydanticAI:

You have a Python program. It parses the config and keeps the whole wiki in memory as a data model – representing the tree you saw above – and pre-populating any default_sections before an LLM is ever involved. It then fills that model in three breadth-first stages:

  1. Discover wiki sections. Ask the LLM to explore the codebase and call an add_section tool as it goes. Loop until it finds no more, or we hit max_sections. (No pages or content added yet, just the empty sections.)
  2. Discover pages. Loop over each section, asking the LLM to explore the codebase and discover pages – title and description only – and add them via an add_page tool.
  3. Fill the pages. Loop over each page, asking the LLM to explore the codebase and fill in the page content via an append_to_page tool.

In each stage, the LLM gets read-only filesystem tools plus exactly one tool that mutates the state. Finally, when all stages are complete, the code renders the internal state to disk as real folders and Markdown files.

This approach works. It takes time to refine the prompts, to adequately express to the LLM what makes a good “section” and a good “page” and how to do things like link between pages, write Mermaid diagrams, etc. But it works. And we know that every time we run it, our program will perform the same behavior: first discovering sections, then pages, then fill in the pages. Because that’s not a decision made by the LLM; rather, it’s a decision we made as the programmers when we wrote the program. The code is driving the LLM.

This is what we’ve called “programming” for the last ~80 years; it just so happens that one of our program’s sub-routines calls out to a supercomputer in the cloud that knows how to, roughly speaking, “think”.

Flip the script

Now, let’s invert the control flow, and talk about the agentic approach, where the LLM drives the code.

In this approach–hold on. I see a hand raised. Yes?

Um, hi. If the above approach works, why even bother with the agentic approach?

Excellent question, glad you asked. The code-first approach certainly works, and will keep working. That said, the world is gradually converging on a new paradigm, where agents orchestrate themselves, unbound by programmatic control flow. It’s therefore worthwhile to consider how we may still keep the strict guarantees afforded to us by code (schema validations, reproducibility, determinism, etc) even when the agent is in charge (after all, they’re not perfect!).

That’s what I’ll talk about now.

Glad you’re finally getting to the point.

Please don’t talk without raising your hand, and also we’re too deep into this blog post to shift the tone or introduce new narrative devices, so I kindly ask that you not interrupt again.

Agent example (aka The Point)

Consider the exact same problem described above: create a system that generates a technical Markdown wiki for a given codebase. (Again, like DeepWiki.)

This time, imagine you’re using an agentic harness like Claude Code, or Codex, or OpenClaw. What would you do?

A completely valid first attempt would be: just tell the agent to write a wiki for the codebase. That’s it.

And I encourage you to try this. Maybe it even works, if the codebase is small enough! And whenever it makes a mistake (improper markdown link syntax, pages too short, etc), simply update the instructions to nudge the agent towards the desired behavior, and repeat.

But I’ve been down this path. You launch the agent. The resulting wiki is too short. You update the instructions, telling it to write more pages. You launch the agent. Now there are more pages, but each page is only two paragraphs. You update the instructions. You launch the agent. Now the wiki lacks Mermaid diagrams. You add to the instructions. You launch the agent. Now the Mermaid diagrams look good, but it’s missing a section for a crucial library in your codebase. You start to wonder how this same model solved an Erdős problem.

Okay, maybe it didn’t work. But we haven’t even tried skills yet. Or subagents. Or MCP servers! Or Ralph Wiggum!

Sure, we can try all those things. But first I want you to think about the types of failures we are seeing:

Notice something? These types of failures are entirely preventable by code. In the earlier code-first example, we’ll never exceed max_sections because the code breaks early when max_sections is reached. We’ll never have a page that’s too short, because the code loops until pages meet the minimum length. We can statically validate links to make sure they are not broken. We can map wiki sections to the areas of the codebase they cover, and guarantee that all parts of the codebase have wiki coverage. Etc, etc.

How can we get these same guarantees, but in the agent-first approach?

Well, in truth, we can’t – after all, the agent could always respond with “I’m tired, I don’t feel like doing it,” which is something I have never seen a Python program do. So we can never have the same level of certainty with agents as we can with code.

But we can get pretty damn close!

The pattern I have landed on is:

Do NOT allow the agent to interact with the system EXCEPT through a program that you control. This program strictly prohibits operations that would lead to invalid state, while providing timely tips that guide the agent toward correct behavior.

Um, what does that mean?

It means: our wiki agent does NOT write Markdown files. It doesn’t create folders for wiki sections. It doesn’t even have a file_write tool (we disable that tool in the agent frontmatter yaml, or when we launch the harness).

Instead, we give the agent a CLI program, which is the only way it may interact with the wiki’s state. If the agent wants to create a section, it invokes our program, in this example a Python program:

$ wiki.py add-section "User Management"

This updates (and creates, if necessary) a file state.yaml that represents the state of the wiki. Just like in the earlier code-first solution! In fact, the data model can even be exactly the same. Each invocation of wiki.py automatically loads ./state.yaml, resuming from the existing state and overwriting it with the new state.

What’s the output of this command?

$ wiki.py add-section "User Management"
Section added. There are now 8 sections. Max is 12.

And if the agent somehow ignores the above warning and still exceeds the max count:

$ wiki.py add-section "User Management"
Error: cannot add section; max of 12 sections already added. Move on to page creation.

Notice how the code is handling the deterministic stuff (how sections are added, maintaining the state, running validations), and the agent is only handling the reasoning stuff (what the name of the section should be). Also, the code emits helpful contextual tips exactly when they are most relevant – not in a 20k token system prompt where they may be forgotten.

By now, everyone knows agents love CLIs. So, of course the program has a nice help output:

$ wiki.py --help
Use this script while you are writing, or updating, a codebase wiki.
Add sections first, then pages, then page content.
Use render-to-filesystem when ready to persist to markdown files in the wiki directory layout.
All operations update (or create) a file state.yaml.
Check `wiki.py status` when unsure what to do next.

USAGE
    wiki.py add-section SECTION_NAME
    wiki.py add-page SECTION_NAME PAGE_NAME
    wiki.py append-page-content SECTION_NAME PAGE_NAME CONTENT
    wiki.py list-sections
    wiki.py list-pages [SECTION_NAME ...]
    wiki.py show-config # prints config values like max_sections etc
    wiki.py validate # runs all validations on the state
    wiki.py render-to-filesystem # transforms state.yaml to markdown files on disk
    wiki.py status # prints a status report tracking progress
    ...

A helpful command status will show overall progress and guide the agent along:

$ wiki.py status
Sections discovered: 12/12 (stage complete)
Current task: page discovery for section 'User Management'

- [x] Discover sections
- [-] Page discovery (in progress)
  - [x] Discover 3/3 pages for section 'Control Plane'
  - [x] Discover 3/3 pages for section 'Service Lifecycle'
  ...
  - [-] Discover pages for section 'User Management' (in progress 2/3)
- [ ] Page writing (pending)

When the agent has decided there are enough sections and pages, and the content is good, it may invoke wiki.py render-to-filesystem. This takes the state.yaml intermediate representation, and actually renders it to the filesystem as markdown files (following the filesystem layout shown earlier).

And it’s yet another chance to enforce validations:

$ wiki.py render-to-filesystem
Error: not ready to write to filesystem. The following validations failed:
- Page user-management-service/user-creation-flow.md is too short; currently 8789 chars, minimum is 10000.
- Page control-plane/storage-backends.md contains invalid markdown link on line 47.
- Codebase directory src/internal/data-models/ is not covered by any page.
- Section user-management-service is missing an OVERVIEW.md file.

Of course, handing the agent a CLI isn’t enough to explain what it’s supposed to do. We still need instructions (generally agent or skill definitions). I landed on a solution like this:

.agents/
  agents/
    wiki-writer.md # our "entrypoint"; instructions for invoking the below skills, and general CLI usage tips
  skills/
    section-discovery/SKILL.md # specific info related to section discovery
    page-discovery/SKILL.md # specific info related to page discovery
    page-writing/SKILL.md # specific info related to writing wiki content

In the above, we explain things such as:

What we don’t have to do is enumerate a massive list of rules and guidelines. The CLI tool wiki.py will handle that for us, and will surface that information to the agent when it is most relevant. We may simply write: “wiki.py will guide you as you go, so follow its warnings, suggestions, and tips.”

At this point, certain hyper-pedantic readers might take issue: “You’re not really making invalid states unrepresentable,” they might (fairly) argue. “I could still manually craft a state.yaml file that has more sections than max_sections. That’s an invalid state, and I represented it. A better description is, you made invalid states un-enterable.” To that I say: good point. But, 1) I think you could allow that the outcome is close enough to ‘unrepresentable’, especially considering where we started, and 2) that title is not nearly as catchy for a blog post.

Agents everywhere

This pattern is especially useful when you’re forced to use an agent-first approach but would really prefer a code-first one. Sadly, I can foresee a world where access to a direct /chat/completions endpoint (or newfangled /responses or whatever) becomes less and less common, and instead we have no choice but to interact with agent-first harnesses. (Partly why I’m a big fan of self-hosted AI).

If your only interface is a full-fledged agent, then your best bet is to strictly limit what actions it may take – ideally to a single CLI program that you control and that programmatically prohibits actions that would lead to invalid state.