> ## Content Index > Fetch the complete content index at: https://operators-ai.ghost.io/llms.txt > Use this file to discover other available public pages before exploring further. # The operator audit: ten questions, twenty points, and my own score - URL: https://operators-ai.ghost.io/the-operator-audit/ - Published: 2026-09-06T19:00:58.000Z - Updated: 2026-09-06T19:00:58.000Z - Description: Eight weeks of issues, one rule each. I finally checked whether I follow them. The scorecard is free. The prompt that grades your setup, and my grade, are for Operators. - Author: Justin Zsimovan - Tags: lessons, playbook Since July I have sent you nineteen issues. Each one planted a rule. Least access. Draft first, send second. Make the automation prove it ran. Never let a sync reverse a decision a human made. Approve less, judge more. One job per agent. Fix the flow before you automate it. Do not change models because a chart moved. Nineteen rules is a lot to preach. So this week I did the thing I had been avoiding: I wrote all of them down as questions, scored each one 0, 1, or 2, and then had my agent go through my actual operation, file by file and job by job, and grade me against my own scorecard. Two scores came out of that. The one I gave myself from memory over coffee, and the one the evidence produced. They were not the same number. The gap between them is the whole issue. The scorecard is below, free, and I would like you to run it on yourself before you read anything else. The version that inspects your setup for you, plus my own graded results and what I am fixing first, is for Operators after the divider. ## The problem Every rule I have written is easy to agree with and easy to believe you already follow. That is what makes them dangerous. "Of course my agent only has the access it needs." "Of course I would notice if the nightly job stopped." You nod, you move on, and the nod becomes a memory of having checked. I know this because I did it. In August I wrote about [a review queue I found in my own operation with items seventeen days old](https://operators-ai.ghost.io/approved-is-not-reviewed/). I had built the control point, felt the control, and starved it. Nothing about that queue would have shown up in a memory audit. It only showed up when I opened the thing. A scorecard fixes the nod problem for one reason: it forces a number. You cannot score yourself a 2 without naming the structure that earns it. If you cannot name it, you are a 1, and if you cannot name anything, you are a 0\. The scoring rule is the audit. ## How to score Ten questions. Each is worth 0, 1, or 2. - 0 means no. Nothing in place, or "I would notice" is the whole plan. - 1 means partial. A rule exists as an instruction, a habit, a prompt line, or a setting someone could change. - 2 means yes, with structure. The rule is enforced by something that does not depend on anyone remembering it: a missing credential, a code path, a separate watcher, a file that has to exist before the action can happen. The difference between 1 and 2 is the line I keep coming back to from July: [a setting is a promise, a structure is a fact](https://operators-ai.ghost.io/this-week-thirteen-policies-one-enabled-and-the-difference-between-a-rope-and-a-wall/). Score honestly. A 1 that you know is a 1 is worth more than a 2 you talked yourself into. ## The ten questions **1\. Least access.** Does each agent, profile, or scheduled job have only the access its job requires? The tell that you are at 0: one set of credentials, one agent, everything enabled, because that was easier on day one. Planted in [Your AI doesn't need to be evil to burn your business down](https://operators-ai.ghost.io/obedient-not-evil/). **2\. The send and spend gate.** Can any automation message a customer or move money without a human in between, and is the "no" enforced by structure or by a sentence in a prompt? A 2 here means the sending path physically does not exist for the agent: no credential, draft-only mode, a window that closes. Planted in [rope or a wall](https://operators-ai.ghost.io/this-week-thirteen-policies-one-enabled-and-the-difference-between-a-rope-and-a-wall/). **3\. Proof of run.** For each scheduled automation, if it silently stopped tonight, would something other than the automation itself tell you within a day? Count your jobs, then count the ones something else is watching. The ratio is your score. Planted in [the heartbeat playbook](https://operators-ai.ghost.io/the-heartbeat-playbook-make-your-automations-prove-they-ran/) and sharpened in [the run was clean](https://operators-ai.ghost.io/the-run-was-clean/). **4\. Decision survival.** Is there any sync, import, refresh, or rebuild in your operation that could overwrite a decision a human made by hand? The tell: you have edited a record manually, and you are not sure the next refresh will respect it. Planted in [You made the call. Your automation quietly reversed it.](https://operators-ai.ghost.io/automation-reversed-your-decision/) **5\. Judged, not acknowledged.** At the moment someone clicks approve, can they see on screen what happens if they are wrong? And is there a queue that goes more than a week without a ruling? A parking lot with a gate that says oversight on it scores 0\. Planted in [approved is not reviewed](https://operators-ai.ghost.io/approved-is-not-reviewed/). **6\. One job, written down.** Does every agent or desk have a document stating what it owns and, above all, what it does not? Not a prompt. A file another person could read. Planted in [the desk system](https://operators-ai.ghost.io/the-desk-system-the-full-architecture-of-my-ai-back-office/). **7\. Flow before automation.** Does any automation in your business encode a process that was still a mess when you built it? The tell: the automation has a growing list of exceptions, workarounds, and "just this once" manual overrides. Planted in [You automated a mess. Now it's a fast mess.](https://operators-ai.ghost.io/you-automated-a-mess-now-its-a-fast-mess/) **8\. The finish line.** For your current AI project, can you state the original request in one sentence and name the existing work that disappears when it is done? If the plan has grown past the request and nobody decided that on purpose, you are at 0\. This is the newest rule and the one I broke most recently. **9\. Model routing by cost of being wrong.** Do you know which model runs which job, and is the choice based on what a mistake costs rather than what the task looks like? Did you switch anything this week because a new model shipped, without a test? Planted in [my agents pitched me](https://operators-ai.ghost.io/my-agents-pitched-me/) and [the model wars got a price list](https://operators-ai.ghost.io/model-wars-price-list/). **10\. One home for state.** For every kind of working state, files, sessions, memory, secrets, can you name the single place that is the authority? Two machines each holding a copy of the truth scores 0\. Planted in [Two machines, one mess](https://operators-ai.ghost.io/two-machines-one-mess/). ## What the bands mean Add it up. **0 to 7.** Your AI setup runs on attention. It works because you are watching, and it will fail the week you are not. Nothing here is a crisis, but do not add another automation until questions 2 and 3 are at least a 1\. Sending and silent failure are where the expensive surprises live. **8 to 14.** This is where most operators I talk to actually sit, and where I expected to land. The rules exist, mostly as habits and prompt lines. Your work is converting 1s into 2s, and the order matters: start with whichever question has the highest cost of being wrong in your business, not the one that is easiest to fix. **15 to 20.** Either you run a tighter shop than I do or you scored yourself generously. Run it again and, for every 2, write down the name of the structure that earns it. If you cannot write a name, it is a 1. ## Your move Five minutes, right now, before the divider. Score all ten from memory. Write the number down. Do not look anything up. Then, this week, open the actual thing for the three questions you scored highest. The scheduled jobs list. The permissions page. The queue. Rescore those three from what you see. The number you wrote from memory is your confidence. The number from the evidence is your position. Most of what I have learned since July lives in the space between those two, and the paid half of this issue is what I found in mine. ## The playbook: the audit that grades you Operators: three things below. The prompt that turns the scorecard into an inspection your own agent runs on your setup. My results, question by question. And the fix order, which is not the score order. ### The audit prompt Paste this into whatever runs your operation: Claude Code, Codex, Hermes, or the agent that manages your automations. It is written so the agent cannot grade you kindly. Every score has to point at a file, a setting, or a job it actually looked at, or the score is 0\. Read-only by design; it must not change anything. > You are auditing my AI and automation setup. Read-only: do not modify, create, delete, send, or run anything with side effects. If you cannot inspect something, say so and score it 0; do not assume. > > First, inventory. List every scheduled or recurring automation you can find (cron jobs, scheduled tasks, agent profiles, workflows, watchers), with its schedule, what it reads, what it can write to, what it can send, and which model runs it. List every credential or integration each one can reach. > > Then score these ten questions 0, 1, or 2\. 0 is no. 1 is partial: the rule exists as an instruction, a prompt line, a habit, or a changeable setting. 2 is yes with structure: enforced by something that does not depend on anyone remembering, such as a missing credential, a code path, a separate watcher, or a required file. For every score, cite the exact path, setting, or job you inspected. A score with no citation must be 0.Least access: does each agent, profile, and job have only the access its job needs? Name any broad grants, shared credentials, or profiles with everything enabled.Send and spend gate: can anything message a customer or move money without a human? Is the block a structure (no credential, draft-only path, closed window) or a setting or prompt instruction? Give examples of each.Proof of run: for each automation in the inventory, would a silent stop be detected within 24 hours by something other than the automation itself? Report watched versus unwatched as a count.Decision survival: list every sync, import, refresh, or rebuild that could overwrite a manual edit or a human decision. For each, say whether there is a ruling record it checks first.Judged, not acknowledged: list every approval prompt, review queue, or confirmation step. For each, say whether the consequence of a wrong click is visible at the moment of clicking, and how old the oldest unanswered item is.One job, written down: for each agent or desk, does a boundary document exist stating what it owns and does not own? Count with and without.Flow before automation: find automations with a high density of exceptions, workarounds, TODOs, or manual overrides. Name the worst one.The finish line: in the last 30 days of change history, find any fix or project whose scope grew well beyond the original request. Report the original request, the file and commit counts, and whether the growth was decided on purpose.Model routing: which model runs which job, when was it last changed, and was that change tested? Flag any routine job on the most expensive model and any switch made on release day.One home for state: how many machines and paths hold working state, sessions, memory, or secrets? Name any duplicated project files or session stores. > > Output: a summary table with the ten scores and the total out of 20, one section per question with evidence paths, and a final list of the five most uncomfortable findings ranked by how expensive they would be if left alone. Do not soften findings. Do not include secret values, only their locations. Two notes from running it. First, give it time; mine took about half an hour of reading before it wrote a word. Second, it will ask for permission to look at things. Grant read access generously and write access not at all. An audit that could change what it audits is not an audit. ### Where I stand ### Where I stand My score from memory: **\[JUSTIN: your from-memory number here\]**. My score from the evidence: **8 out of 20**. That is the middle of the middle band, in a business where I have been writing the rules. I want to walk through it question by question, because the specifics are where the lesson is, and because most of my 1s are the exact kind of 1 that feels like a 2 from the couch. For scale, this is what the audit found before it scored anything: about three dozen scheduled jobs (three quarters of them plain scripts with no model involved, which is the right default), nine agent profiles with one job each, a dozen repos with live automation, and seven always-on daemons. That is a real operation, and it was a real operation the whole time I was writing "give it one job first." **1\. Least access: 1.** Every one of the nine profiles reads the same model key from its own copy of the same file. Rotating one key means editing nine files. Eight of the nine profiles get the full toolset, browser, terminal, file writes, scheduling, image generation, everything, whether the job is writing a LinkedIn post or reviewing code. Exactly one profile is narrowed to the three tools it needs. The good news is that the written job descriptions are strict ("never sends," "never writes the brain," "never commits"). The bad news is that those are sentences. The one narrowed profile is the only place where the sentence became a wall. **2\. The send and spend gate: 1, and this is the one that stung.** On paper this is my best question. Customer messages on the field-service side cannot leave without a lane-scoped authorization file that expires every 24 hours; delete the file and sending stops. Ad spend goes through a policy class that refuses to write a budget above the stored cap, and every campaign is created paused. Three quarters of the scheduled jobs never touch a model at all. That is structure, and I was proud of it. Then the audit found the renewal job. That 24-hour human authorization is re-signed twice a day by a scheduled script, with my name written in as the actor. It has done this about a hundred times. I signed once, in July, and a cron job has been signing for me ever since. The gate is real. I am no longer standing in it. And on a different lane, the public LinkedIn posting, the only human gate is that I can delete the post in the scheduler before it goes out. That is a setting wearing a structure's clothes. **3\. Proof of run: 1.** Roughly one in five scheduled jobs has something separate watching it. The ones that do are the customer-facing ones, which is the right priority. But every watcher is itself a scheduled job inside the same scheduler it watches. The scheduler writes its own heartbeat file, and the audit could not find anything that reads it. If the scheduler dies, about thirty jobs and all seven watchers go quiet together, and nothing outside the building notices. This is the heartbeat playbook's first rule ("the watcher must be too simple to fail in interesting ways") violated by the person who wrote it: my watchers are exactly as alive as the thing they watch. Worse, live evidence. A daily digest job had failed seven days in a row under a status literally named "silent." The job whose only purpose is catching failed jobs had been disabled for over three weeks. I did not know either of those things until Saturday afternoon. **4\. Decision survival: 1.** The newest code is good: human decisions are append-only, the schema comments explicitly warn future me not to bypass the guard, and the renewal job re-reads before it writes. But older paths still use replace-on-write, including a history migration script that could overwrite human ad verdicts, and one buy-signal pipeline takes its input from a colored spreadsheet row that anyone could recolor. The new rule is enforced where I wrote it most recently and nowhere I wrote before I learned it. **5\. Judged, not acknowledged: 1.** Five approval surfaces. On the biggest one, the customer-draft cards, the numbers are: a bit under four hundred approvals, a little over a hundred rejections, and, on the ladder that is supposed to track whether I am reading, one hundred eighty-seven approved untouched against five approved with an edit. Ninety-seven percent of my approvals are a single tap with no change. The rejection rate (about one in five) says I am reading. The edit rate says I am not reviewing. And the card shows me the message but not what matters at decision time: how many sends remain under today's cap, how many touches this customer has already had. I wrote "consequence on screen" two weeks ago. My own card does not have it. **6\. One job, written down: 1.** Just over half the repos have a boundary document. All eight named agent profiles have a one-paragraph ownership statement with explicit "never" clauses, and that is the best hygiene in the whole system. The old agent framework, which is still running beside the new one, defines its agent as "infer domain from context," which is the opposite of the rule. And the repo that publishes this newsletter, the one you are reading, has no README and no boundary file at all. Fifty-one commits. Zero documents. **7\. Flow before automation: 1.** A digest job that describes itself in its own config as "a stopgap while the real product is built" is still running, next to the real product, which shipped. Backup copies of live scripts sit beside the live scripts in the production folder. The customer-contact fast loop has run roughly six thousand times and still documents itself as "read-only observation, no sends." It sends. The code moved on; the description did not, and the description is what a new person reads first. **8\. The finish line: 0.** The one I wrote about most recently, and the one I scored worst on. In the last thirty days, in the two repos closest to money, the pattern is: one feature commit, then a tail of thirteen to twenty "review correction" commits over two days, each adding hundreds of lines to the same two or three files. One feature that started as admitting one product category to a pipeline ended at about forty commits and several thousand added lines. The reviewer loop is doing its job. Scoping is not. Nothing ships without a dozen-commit tail, and three features in a row on the same repo repeated it. **9\. Model routing: 1.** Each profile has its own model, and the engineer and reviewer are deliberately on different model families so the review is not the model grading itself. That part is right. But the default model has changed four times in ten weeks, most recently the day of this audit, and a frontier model runs a board-orchestration loop every thirty minutes around the clock with no cheap tier and no fallback configured. I published the upgrade gate on Friday. I had not run it on my own default by Saturday. **10\. One home for state: 0.** Two agent frameworks live at once, the new one and the one it replaced, with the old one still holding a couple of terabytes and a fifth of the scheduled jobs. Nine separate state databases. Secrets in at least five locations. Scripts that reference the projects folder with two different capitalizations, which works only because this filesystem does not care and will break the day a Linux box joins. Four machines on the network and nothing written down about which one owns which desk. I wrote "two machines, one mess" last week as a lesson learned. It was a lesson observed. Three things I take from the exercise. The 1s are the story, not the 0s. Nearly every 1 is a rule that I did adopt, in the code I wrote after I learned it, and did not go back and apply to the code I wrote before. The audit is really a map of when I learned each rule. The most dangerous findings were the ones behind my best answers. The send gate was the question I would have scored a 2 with the most confidence. The renewal script was underneath it. And I did not find a single one of these by thinking about it. Every one came from the agent opening a file, reading a count, or diffing a date. The scorecard above gives you the questions. The prompt gives you the opening of the files. Only the second one produces a number you can act on. ### The fix order The temptation is to fix the lowest scores first. Wrong order. Fix in order of what a miss costs, which for almost every small business is: 1. Question 2, the send and spend gate. A wrong send reaches a customer. A wrong spend reaches a bank. Everything else is recoverable by comparison. 2. Question 3, proof of run. Silent failure is the most expensive way to fail because the bill arrives late and compounding. 3. Question 4, decision survival. A reversed ruling is a wrong send waiting for its trigger. 4. Question 5, the approval surface. Every click you retire behind a wall buys back attention for the clicks that matter. 5. Everything else, in whatever order your lowest scores fall. Questions 6 through 10 are hygiene. Important hygiene, and they compound, but nobody's business burned down over an undocumented boundary this month. Businesses did get burned by an agent that could send. ### Steal this Ninety minutes, one sitting. - Fifteen minutes: score from memory, write it down. - Thirty minutes: run the audit prompt, read-only, on your actual setup. - Fifteen minutes: put the two numbers side by side and write one sentence per question where they differ. That sentence is the finding. - Thirty minutes: take the top item from the fix order, not the top from the score list, and convert it from a 1 to a 2\. One wall. A credential removed, a watcher added, a queue given a drain time. Then send me both numbers. I will publish the spread, anonymized, in a few weeks. My guess is the memory score runs three to five points above the evidence score for almost everyone, and I would like to find out if I am wrong. Hit reply and tell me what's on your bench this week. I read every one.