Keep a reusable program when another person, another dataset or a consequential decision depends on the method behind an AI answer.
At a family barbecue in May 2024, my twelve-year-old son Fitz and I had an afternoon math duel. He had already beaten me at ultimate tic-tac-toe. With paper and pencil, he solved the puzzle while I was still working through it. I wrote about our exchange at the time.
The puzzle was to use 6, 8, 6, 4 and 5, in that order, to make 37. Each number had to remain separate and appear only as often as it did in the list. Mathematical operations and parentheses were allowed. Fitz found:
(6 + 8 - 6) × 4 + 5 = 37
I proposed a different challenge. We both began designing prompts to get a language model to solve the problem. My prompt asked it to restate the rules, propose algorithms, compare them, choose an approach, generate Python and run the program.
The response I published, labeled Ragbot.AI with GPT-4 Turbo, reported:
6 × 8 ÷ 6 × 4 + 5 = 37
Both expressions work. The printed solver, however, only inserted operators between the numbers; it did not generate parentheses. Its reported expression is valid, but the listing does not cover every expression the rules permit. Having code to inspect makes that gap visible.
The lasting value of the exchange was the procedure it left us to examine. We could ask what the search covered, change the target, or extend the implementation. A result of 37 answered the puzzle. A program gave us a starting point for investigating related puzzles.
Using AI to turn a problem into inspectable code is central to what I call synthesis coding: human-directed software development with AI. I believe serious systems need that shared, durable representation of their behavior. Better models should expand what our teams can build while leaving us able to understand, question and improve the work.
Choose what you need to keep
An explanation of a stack trace, a calculation or a draft can be useful on its own. If you can check the result and have no further use for the procedure, saving the answer and its context may be enough.
A recurring calculation asks more of you. Next month's data will differ. A colleague may need to run it. Someone may question a result and ask which rows were included. At that point, retaining the method becomes part of completing the task.
This choice does not map neatly onto “chat” and “code.” OpenAI's current data-analysis documentation describes Python execution for some ChatGPT analysis tasks, with code and assumptions available for review. An answer in a chat may already be the result of a program. The practical question is whether you have inspected and retained the procedure you intend to rely on.
The conversation itself can also be an artifact. Prompts, explanations and outputs can be saved and compared. OpenAI documents chat-history export for eligible accounts, with account and workspace restrictions. A saved answer may contain a method, code or the reasoning you need later. Retaining executable code adds a way to apply a method to new inputs.
If I will need a procedure more than twice, or other people will depend on it, I usually want code I can inspect and maintain. Consequences matter too: a one-time financial calculation may deserve closer review than a routine daily report.
The output need not become a service with an owner on call. A documented script, a query or a notebook may be the appropriate unit. Choose something the people who depend on it can inspect, run and maintain.
Turn a median into a method
Worked example added September 2026.
Suppose you ask an assistant for the median value in each group of a CSV. A table of results tells you what it calculated for that input. To reuse the calculation, you also need to settle what “median” means and what happens to invalid data.
For this example, each record has equal weight within its group. Sort the values. For an odd number of records, take the middle value; for an even number, average the middle pair. This is the convention implemented by Python's statistics.median.
Here are the values in the accompanying constructed dataset:
| Group | Values in sorted order | Expected median |
|---|---|---|
| alpha | 1, 9, 100 | 9 |
| beta | 2, 10 | 6 |
| gamma | -3, -3, 7, 9 | 2 |
For beta, the calculation is (2 + 10) / 2 = 6. For gamma, it is (-3 + 7) / 2 = 2. These expected results are easy to check without trusting the implementation.
Download the program, synthetic CSV, tests and instructions. The example uses Python 3.12 and its standard library, and runs locally without an AI service.
The input contract is deliberately specific: exactly two CSV columns, group,value; a lowercase identifier of up to 32 characters for each group; and integer values from -1,000,000,000 through 1,000,000,000. The README gives the accepted identifier and number syntax. Blank records, missing or extra fields, nonnumeric values and values outside the range are rejected. The whole file fails instead of silently dropping a troublesome record. Files are limited to 1 MiB and 10,000 data records.
Those are choices for this example. Real measurement data may require decimals, missing-value rules or weights. Changing those requirements should change the implementation and its tests.
After extracting the archive, run these commands from its reusable-median folder:
python3 group_medians.py measurements.csv
python3 -m unittest -v test_group_medians.py
The first command produces:
group,count,median
alpha,3,9
beta,2,6
gamma,4,2
The program validates the complete input before producing result rows. Its calculation step then applies the chosen median convention separately to each group. This excerpt is from render_medians; the download contains the complete parser and command:
for group in sorted(groups):
values = groups[group]
result = median(values)
if isinstance(result, float) and result.is_integer():
result = int(result)
writer.writerow([group, len(values), result])
The output includes the count because a median alone can conceal a change in the number of records used. A count does not prove that the right records were selected, but it gives the reviewer another concrete value to compare.
The tests check the worked medians, odd and even groups, duplicates, negative numbers, grouping and rejected input. They also change beta's values from 2 and 10 to 2 and 14 and expect its median to change from 6 to 8. That check exercises reuse on different data.
The example passed all 19 tests with Python 3.12.3, both from the source files and from a fresh extraction of the packaged ZIP. A deliberately modified copy substituted median_low, which chooses the lower middle value for an even-sized group. The unchanged tests rejected it. In beta's original group, that substitution returns 2 instead of the required 6.
A lower median is a valid statistical convention for other purposes. Here it violates the rule we chose. That makes it a useful test fault: a plausible answer can still implement the wrong method.
Keep the input CSV, program, tests and instructions together. Record the Python version and the invocation alongside a result you need to retain. Someone else can then inspect the method, rerun these checks and decide whether the example's assumptions fit their data.
Make later questions answerable
Natural language is useful for discovering requirements, explaining decisions and discussing exceptions. A receiving API or database still needs particular fields and behavior. “Compute a median for each group” leaves open the treatment of missing values, grouping keys and even-sized groups. A written contract, examples and executable checks let people resolve those questions before a result reaches another system. For software behavior that other systems depend on, I regard code as our best format for expressing those contracts.
Code alone does not fix every condition of a run. A program that reads a live API, the current configuration or a random value can behave differently without a source change. Retain the relevant inputs and state as well as the logic. For programs with dependencies, pinned versions, hashes and retained packages help constrain installation; the pip documentation also explains platform and availability limitations. A source file is one part of a repeatable procedure.
During an incident, users may be waiting while you try to determine what the system actually did. A stack-trace explanation can suggest a cause. A test harness that reproduces the error and records the relevant SQL queries can help establish it. Use controlled data and the project's logging rules, especially where queries or identifiers could reveal sensitive information.
When you can reproduce the failure, you can add a test, instrument the affected path and compare versions or bisect commits to locate a regression. Logs and metrics connect the program to what happened when it ran. Conversation records can help explain why a change was proposed.
The same reasoning applies when someone asks, “Why did this customer receive that price on that date?” The relevant evidence may include the input, code version, configuration, policy, model version if one was used, approvals and actual output. Retaining source code does not recreate missing external data or prove what a past execution did.
I want review tied to a particular change: who reviewed this rule, which version they reviewed and what evidence they considered. Those records help a team answer questions about its decisions.
Give the procedure a place in the team's work
Framework update, April–May 2026: I added transferable knowledge as the fifth principle in April, and broadened the fourth from iterative context building to a self-improving system in May.
The principles I developed through the synthesis engineering series give the retained procedure a home: humans own architectural decisions; systematic standards govern quality; people actively understand the system; reviewed lessons improve later work; and knowledge is transferable to other people and tools.
The framework article develops the principles beyond their application here.
For me, the larger gain is institutional capability. A team needs shared mental models of the system: where its boundaries are, which patterns it uses and why, and what should happen when a dependency fails. Code review can teach those models as well as catch defects. Code, explanations and decisions in shared repositories give the next engineer something to learn from and challenge when people change roles. LLMs can accelerate that collective learning, provided the useful knowledge from individual conversations becomes part of the team's shared work.
For a reusable program, architectural decisions include where state lives, who may access it, which interfaces it uses and which external calls it may make. Shared conventions should also specify errors, timeouts, retries and performance expectations. AI can research and propose designs. The responsible people decide which design meets their requirements and which changes they authorize.
- Implement the repository pattern already used by this microservice, including its error handling and transaction boundaries.
- Generate a migration from schema v3 to v4 using the project's migration process, with checks for the data being transformed.
- Extend the logging middleware with correlation IDs and permitted user identifiers, following the project's privacy and logging policy.
Each request points to an existing system and a behavior that can be inspected. A generated migration still needs review against the actual schema and data. A logging change still needs a check that the intended identifiers reach the intended logs without exposing fields the policy excludes.
The Claude Code workflow article discusses reviewing AI-assisted changes with tests and security checks. A reusable script should enter the team's review process too, including its applicable tests, linters and security analysis. AI can suggest tests and edge cases, but the expected outcomes need scrutiny. Our median example would have been poorly tested if the implementation and every assertion had quietly agreed to use the lower middle value.
Ask for an explanation or diagram of the data flow, call graph or error paths, and check it against the code, tests and execution traces. A person responsible for a critical subsystem should be able to explain its dependencies and failure behavior well enough to review a change or investigate an incident.
In polyrepo synthesis, I apply this reasoning across repositories. Shared records can explain authentication, logging, error handling and feature-flag conventions. Instruction files such as CLAUDE.md can make those conventions available where the tool loads them. The team must still keep the records current and check that the relevant context reached the task.
Over time, those reviewed artifacts compound: a design decision informs the next implementation, a failure becomes a regression test, and an explanation helps the next engineer ask a better question. The model can navigate and extend that accumulated body of work. That is the purpose of the context meshes I described in polyrepo synthesis.
Carry the constraints into larger systems
The family puzzle made its constraints unusually visible: an ordered list, permitted operations and a target. Software often requires us to discover the constraints as well as implement them.
A pricing calculation may have policy limits. Risk scoring may apply business rules to several data sources. A scheduler may need to respect precedence among jobs. A recommender may have requirements about diversity or fairness. These examples all benefit from stating the objective and the rules before choosing an algorithm.
They also demand work the puzzle never tested: validating source data, defining access controls, handling unavailable dependencies, observing behavior and reviewing the domain-specific consequences of an error. An AI-generated program can be part of that work. The fact that it runs does not settle those requirements.
Update (December 7, 2025): The ownwords case study provides a further example through WordPress-to-Markdown conversion, including design choices and implementation checks.
Three prompts for reusable work
Choose the algorithm before implementing it
Restate the problem, inputs and constraints. Identify any ambiguity that could change the result. Compare suitable approaches where there is a meaningful choice, including correctness, complexity and implementation cost. Recommend an approach and explain why. Then generate the code and show how to run it.
For the median example, a standard-library implementation is a sensible starting point. Asking for three or five alternatives would add little unless scale, memory or a different statistical definition created a real choice. The earlier math prompt asked for five algorithms; the reusable lesson is to examine the choice, not to fill an option quota.
Ask for tests that challenge the chosen rules
Generate the implementation and tests with independently worked expected results. State the invariants in plain language. Include invalid inputs and a changed dataset. Name a plausible implementation error that the tests should detect, and demonstrate that an unchanged test fails when that error is introduced into a separate copy.
An invariant is a condition the program is meant to preserve. In this example, each output group uses only that group's records. For even-sized groups, its median follows the average-of-the-middle-pair rule. Those statements give the reviewer something more specific to check than a plausible table.
Do not accept the test author's assertion that a test would fail. Inspect what it checks. The deliberately wrong lower-median copy shows that these checks catch that particular error; it does not establish that the suite detects every defect.
Refactor against the existing behavior
Given this function and the target behavior, propose a focused change. Explain why each part is needed. Keep existing behavior that the task does not change, use the project's conventions, and do not add dependencies without authorization. Show the relevant tests before and after the change.
A diff gives a reviewer a bounded change to inspect. Git can compare text files, including saved prompts and explanations, as well as source code. Comparing versions tells you what changed; tests and review address what the change means.
What happens when LLMs get much better?
I expect models to become orders of magnitude more capable: to make fewer local reasoning mistakes, work with richer context, run tests, inspect logs and adjust code with greater autonomy. That prospect strengthens my case for explicit artifacts. The more a tool can change, the more consequential it becomes to inspect its method, retain the relevant inputs and review a particular version of the work.
A managed system may retain versions and execution records for you. Evaluate its ability to expose the method, preserve the necessary inputs and let a responsible person review a change.
Compilers let people work at a higher level of abstraction instead of writing machine code by hand. Source code remains a representation they can inspect and maintain.
The representation may include code, configuration, a query, policies and a record of the decisions behind them. People still need to discuss what the system should optimize, whose interests it serves and which trade-offs are acceptable. A program implements choices; it does not make those choices appropriate.
I expect organizations, customers and regulators to adapt more slowly than the models themselves, often over years. They need a stable representation of the systems they depend on through that transition. We may spend less time typing boilerplate and more time orchestrating work, while continuing to express architecture and standards in code and supporting records that people can inspect.
For your next recurring AI-assisted calculation, save one representative input and work out its expected result before asking for the implementation. Then ask a colleague to run the saved procedure on a different input. The questions they need to ask will tell you what your reusable program still needs.
Let LLMs expand what your teams can build, and keep code as the durable, shared source of truth. We will still be writing code as the models improve, with far more capable collaborators at our side.