Before asking AI to fix a bug, write one failing input

An explanation of a bug can sound complete long before the bug is understood. The words are familiar, the proposed fix is plausible, and the patch is small enough to look harmless. What I want beside that explanation is an input that makes the disagreement visible. A tiny example gives both the assistant and the reviewer something firmer to work on than confidence expressed in fluent prose.

The Brain Sandwich, published by Terrible Software on October 2, argues for deliberate human thought before and after using AI. I take it as a prompt to make those two moments concrete. The following example is an original, hypothetical bug review. It is not a story about a customer incident, a test I ran in a production system, or evidence that a particular assistant will solve the problem correctly.

Make the disagreement small

Suppose a CSV importer removes duplicate contact records. The input has three rows: contact A with identifier 0012, a second copy of contact A with identifier 0012, and contact B with identifier 12. The intended result contains two contacts. Leading zeroes are meaningful because the identifier is a string assigned by another system. A proposed implementation that converts both identifiers to integers would collapse the result to one contact. The important requirement fits in a sentence: exact string identity determines duplicates.

That sentence is a better starting prompt than a general request to improve the importer. It tells an assistant which choice belongs to the product and which part is still open to implementation. It also gives the reviewer an answer to compare with the patch. Without the requirement, a concise implementation might appear to be an elegant cleanup. With it, the same conversion is plainly a behavioural change. The model’s explanation cannot resolve the disagreement by calling the conversion a normalisation.

I would put the expected rows next to the input before looking at a generated fix. This is not about predicting the implementation. It is about deciding the observable result independently. The expected output should preserve the first 0012 and the distinct 12. Whether it preserves original order is another requirement that needs an explicit answer. If order matters to later processing, a test that sorts the result before comparison would hide a relevant failure. Even a tiny example contains choices worth owning.

Add one change that should not matter

A useful follow-up is to add another exact copy of the first row. Under the stated rule, the number of distinct contacts should remain two. This is a proposed metamorphic check: change the input in a defined way and state the relationship the output should preserve. It does not require us to enumerate every possible output. It does require us to be precise about the transformation. Adding a row with a different identifier is not the same test, because it should change the result.

There is a trap in expanding the example too quickly. Someone may suggest trimming whitespace, ignoring case or merging records that share an email address. Any of those might be appropriate for a different importer. They are new policy decisions here. I would keep them out of the first patch until the owner of the data contract decides what they mean. A bug fix that quietly invents identity rules can create a larger problem while making the original fixture pass.

After an assistant proposes a patch, my review would follow the data. Where is the identifier read? Where, if anywhere, is it transformed? What determines the duplicate key? Which original row is retained? The test is evidence for these questions, not a replacement for reading the change. A test can pass because its assertion is too weak or because the implementation special-cases the example. I want an explanation that connects the rule to the code path and identifies the boundary the patch deliberately leaves alone.

Keep the reason with the regression

The fixture’s name and a short comment can preserve the decision: string identifiers retain meaningful leading zeroes. That is more useful than a comment saying this fixes an AI mistake. Future maintainers need to know why the behaviour matters, regardless of who wrote the earlier code. The record should also distinguish a proposed test from an executed one. In a real change, the actual command and result belong in the review. This article describes the test design; it does not certify an importer implementation.

I do not expect a small input to make every bug easy. Concurrency, remote dependencies and incomplete requirements can resist a neat reproduction. In those cases, recording what is still unknown is part of the work. But when a disagreement can be reduced to three rows and one identity rule, I would do that before asking for a patch. It leaves me with a useful question after the assistant finishes: did this change preserve the decision I made before I saw the answer?