A while back I asked a simple question in our team chat: how do I find a prompt engineer?
A friend wrote back, laughing: “Aren’t we all prompt engineers at this point?”
I like that friend. I disagreed anyway. Most people tell the AI to do something and accept whatever comes back. Or they install one of those little “skills” somebody posted and hope for the best. That is not engineering. Engineering is when you build the thing, break the thing, find out why it broke, and fix the cause instead of the symptom. It is a loop. Write it, run it, read what actually happened, change one thing, run it again.
Then he sent me a link to a post arguing there is no such thing as prompt engineering. I read it. It says prompt engineering is just writing and thinking and communicating. It says the tricks go stale in a few months. It says the same prompt gives you different answers on different days, so it can’t be engineering.
And you know what? Half of it is right. If prompt engineering means polishing a sentence until a chatbot answers slightly better, then call it writing and move on. The three-hundred-dollar prompt templates are noise. The certificates are noise. I would retire the term for that, too.
But that is not what I do all day.
I built prompts the way I built Beast
I wrote Beast, the HTTP and WebSocket library that became the second most used library in Boost. There was no flash of genius in it. I copied what worked, measured what didn’t, and iterated for years until the interface felt inevitable. Take in the problem, try something, measure, adjust, repeat. That is the whole method, and it has never changed. Networking, JSON, URLs, coroutines, standards papers. The domain rotates and the loop stays the same.
When AI showed up, I did what I always do. I treated it as a craft you earn through volume, and I put in a thousand hours before I let myself believe I was any good at it.
Here is what those hours taught me. A prompt is an API. Not a metaphor, an actual API, with a contract, with failure modes, with consumers that break in strange ways when you change it. Every instruction has to survive the removal test: delete it, and something must break, or it does not belong.
The day the foreman forgot the drawing
One of my tools is called the Staker. You point it at an institution and it works through sixteen steps. It surveys the place, finds the people who really benefit, runs a battery of diagnostic tests, tries to kill its own findings, and writes up whatever survives. It leans on helpers, lots of them, and one main model plays dispatcher, handing out the work.
While I was debugging it, I caught the dispatcher doing something I had not planned for. It wasn’t passing along my instructions. It was summarizing them.
Picture a foreman who, instead of handing the crew the drawing, tells them what he remembers of the drawing. Early in the day he remembers fine. Deep into a long job, with his head full of research and results, he gets tired and he paraphrases. A careful six-step challenge, every test numbered and ordered on purpose, turns into “challenge the findings.” Nothing crashes. Nothing errors. The helper cheerfully does something plausible and wrong. That is the worst kind of bug, the kind that looks like success.
You cannot fix that by asking nicely. “Please follow the instructions exactly” is a request made to the very thing that is failing. So I stopped letting the model write the instructions at all.
Every helper’s instructions now live inside a named XML tag in the tool file. The dispatch is a few lines. Roughly this:
Grep tools/staker.md for the tag survey_template.
Follow the block inside it exactly.
Section: 4
Organization: {organization}
Output: evidence-4.md
The helper opens the file, finds the tag, and reads what is inside, word for word. The dispatcher never has to say the instructions, so it cannot summarize them. And if the tags are missing, doubled, or decorated, the helper stops and answers “blocked” instead of improvising.
That is not a prompt trick. It is an interface: a failure, a mechanism, a countermeasure, and a check that fires when the countermeasure gets bypassed. It now sits in the Staker, Vibe Coder, Debt Collector, and Foreman. If you made me pick the crown jewel of everything I’ve built, that’s the one.
The blur
The second lesson was harder, because I did it to myself.
I watched three of my prompt files grow by 19, 44, and 146 percent with no new capability to show for it. Almost all of the growth came in a handful of commits where I had handed a model the whole file and said “improve this.” A model asked to improve a file does not edit it. It rewrites every word, and every word gets pulled a little toward what an average good document looks like: complete, justified, formal, and soft.
One of those files had a rule backed by real benchmark numbers about why a model should never review its own work in the same context. By the sixth revision the numbers were gone. A sentence that sounded wiser had taken their place. The model had sanded a fact into a vibe.
So I measured it from git history, wrote it up as a lesson, and changed how I work. I never edit the prompt anymore. I edit the plan the prompt came from, and I regenerate. The model can generate. Only the human can compress it back to sharp.
The handshake
My favorite part is when the pieces talk to each other.
Vibe Coder builds a plan as a series of tested commits and leaves a ledger of design facts in every commit message. Later, Debt Collector reads that ledger, works out which new work left debt behind, and writes a removal plan. The plan’s headings and tags are exactly the ones Vibe Coder’s integrity check demands. Two prompts, written separately, shaking hands through a schema, with a validator on the other side checking the handshake.
Try doing that with a nicer sentence.
“But it isn’t reproducible”
The article says real engineering gives you the same result every time, and prompts can’t. Nobody holds a network to that standard. The internet runs on a lossy wire, and we got TCP. You build something reliable out of something unreliable. It is the oldest move in engineering.
The model’s wording changes from run to run. The contract it has to meet does not. Tests are written first and shown to fail. A reviewer judges the code without ever seeing the author’s reasoning. A verifier answers in one line: pass, fail, or blocked. Retries have a cap. State lives in git, not in the chat, so a run that dies picks up where it stopped.
And the tricks go stale? Some of them do. The magic phrases, the personas, whatever worked last January. That is why I fence the flavor text off in a block the agents are told to ignore. What lasts is the dull stuff: tests before code, a reviewer who didn’t write it, evidence before claims, state in the repo. None of that cares which model is on the other end.
So what is it
Prompt engineering is the work of making a language model a trustworthy part of a system. The wording is the smallest piece. When I counted the rules in my own how-to, about a quarter of the 120 were about writing. The rest are about dispatch, state, partitioning, verification, and what to do when a run falls over at step eleven.
I am not mad at the article. It lands a fair punch on the hype, and the hype deserves it. But I will keep the name until someone hands me a better one. If “engineering” bothers you, call it LLM systems engineering. The work stays the same.
So, back to the question in the chat. How do you find a prompt engineer?
Don’t ask for their best prompt. Ask for their loop. What broke, how did they know, what did they change, and how did they check that it stayed fixed? If their eyes light up, hire them. If they show you a template, keep looking.
Yes, I know what this blog is called. The best slop is slop with a test suite.
Everything I mentioned here is public domain and free to take: tools-public. Load one. See what it does.
Measure twice. Vibe once.
2026-11-07 10:35 - Claude Sonnet 5.5