The Agent Changed the Code Before Reproducing the Bug
A comparison of two diagnosing-bugs workflows: Matt Pocock's open-source original, and my adaptation with workflow routing, edit permissions, and evidence boundaries for everyday agent work.
First, the two versions being compared: Matt Pocock’s open-source diagnosing-bugs Skill, and the adaptation I made after using it in real agent projects and adding boundaries for everyday work.
Agent debugging feels most like magic when the agent reads an error and announces thirty seconds later: “Found the root cause.”
It changes three files, adds a test, and says the problem is fixed.
The only missing step is that the bug was never reproduced.
That is why I adapted diagnosing-bugs. The original is already technically strong. The problem is not that it has too many steps. In a real project, an agent also needs to decide when a full investigation is warranted, when a direct fix is appropriate, and whether the user asked it to make changes at all.
The strongest parts of the original stay intact
Matt Pocock’s original has a firm discipline: establish a feedback loop that reliably exposes the failure before discussing the root cause.
That is the right starting point.
Without reproduction, you cannot know whether the patch fixes the user’s problem or a neighboring issue that happens to produce an error. Without the same signal before and after, you cannot compare the result.
The feedback loop, minimal reproduction, falsifiable hypotheses, targeted probes, and regression checks all remain. They are the most valuable parts of the original.
The changes concern how that method fits into daily work.
First change: not every bug needs a full trial
The original is a disciplined process for difficult bugs. But not every bug an agent encounters is difficult.
A compiler may already point to the exact type error. A failed assertion may expose the faulty condition. Or the user may already know the fix and simply want it applied. Requiring five hypotheses and a full experiment in those cases can turn rigor into ceremony.
The adapted version classifies the problem first.
Obvious errors take a short path: state the evidence, then act within the user’s request. Intermittent failures, remote-environment problems, performance regressions, recurring bugs, and unknown causes enter the full diagnostic loop.
The original prevents difficult bugs from being treated casually. The adaptation adds routing so simple problems are not over-processed.
Second change: failed reproduction should not end the investigation
The ideal is a single local command that reproduces the problem reliably. Real failures often appear only in production, in one account, or under particular network conditions.
If “no local reproduction” means “cannot continue,” the agent ends up handing the whole problem back to the user.
The adaptation allows it to narrow the evidence: replay a real request, inspect log samples, HAR files, or traces, increase an intermittent failure’s frequency, or propose the smallest temporary observation needed. It should describe only the level of certainty the evidence supports.
It may say “this is the best-supported hypothesis so far.” It must not quietly turn that into “the root cause is confirmed.”
This matters for agents. They often seem less troubled by missing information than by appearing to have no answer. The most convenient explanation then gets written as a fact.
When evidence is incomplete, the next step is to narrow the gap to something testable, not invent a complete conclusion.
Third change: being able to fix it does not grant permission
This was not a major focus of the original, but it can cause problems in real agent work.
“Diagnose this” means find the cause. It does not automatically authorize code changes, let alone committing, pushing, creating an issue, changing a live system, or adding production instrumentation.
The adaptation therefore establishes whether the task is diagnosis only or diagnosis and repair.
In diagnosis-only mode, project files and external systems stay read-only. The agent may read code, run existing tests, inspect logs, and use an existing reproduction. It may not casually add tests, scripts, a temporary harness, or instrumentation.
If reproduction requires new files or observation code, the agent must explain what it would create, where it would go, and when it would be removed, then wait for approval. That gives “diagnosis only” a concrete meaning.
Logs, HAR files, traces, and request samples also cannot simply be pasted into a chat or issue. Cookies, Authorization headers, tokens, personal information, and unrelated customer data must be removed first. Keep only the smallest excerpt that supports the finding.
Diagnosis may require more evidence. That does not grant broader permissions or justify exposing more data.
Even when a fix is authorized, the agent should check for other people’s uncommitted work. Fixing its own bug by overwriting the user’s ongoing changes is not a useful success.
The differences are straightforward side by side
| Original emphasis | What the adaptation adds |
|---|---|
| A strict feedback loop for difficult bugs | Decide whether the full process is needed first |
| No premature conclusion without a reliable signal | Continue with layered evidence when complete reproduction is unavailable |
| Technical diagnosis and repair | Explicit separation of read-only diagnosis and permitted editing |
| Experiments that eliminate incorrect hypotheses | Protection for a dirty workspace and external write boundaries |
| Logs, HAR files, and traces as evidence | Local handling by default and redaction before quoting |
| Preservation of the investigation | A concise default handoff: symptom, cause, evidence, action, and uncertainty |
Calling this a “slimmer Matt Pocock Skill” misses the main change.
It takes a discipline for difficult debugging and makes it a diagnostic layer for daily agent work. The technical method stays rigorous, while the actions stay within the task’s boundaries.
diagnosing-bugs is directly adapted from Matt Pocock’s MIT-licensed open-source version. Its provenance label is OPEN-SOURCE ADAPTATION.
A minimal debugging example
This is a synthetic teaching scenario. Selecting “Chinese” empties an article list, but reloading brings articles back. Do not change the CSS first. On the same route, inspect the response data, article language fields, and filter condition. If a Chinese article lacks lang and the schema defaults to English, the fault is in content registration.
After correcting the field, repeat filtering, reload, and switching to English to make sure other articles have not been hidden. The useful part is the elimination process: an empty page is only a symptom; the reproduction conditions and data determine where to change the code. The public Skill’s diagnosis-only mode defines the write scope for that version. Actual execution must still follow the task scope authorized by the user.