A content pipeline of mine had an export module that used custom code to parse front matter, the metadata block at the top of a Markdown file. Nested arrays were coming out mangled: tags that should have read ["tag1", "tag2"] arrived as malformed strings. My first instinct was to add logging inside the export module and trace what it was doing.
Running the same input through gray-matter gave the expected arrays; my custom code did not. That comparison helped resolve the parser bug. A comparison still needs an expected result whether the implementations agree or disagree.
Originally drafted in December 2025 as part of a longer best-practices guide; revised in August 2026: split out as a standalone article, with its examples generalized beyond their original publishing-workflow setting.
September 2026 update: This revision adds the runnable examples and the discussion of the linked OwnWords implementation.
Use the trace to locate the change
When an AI assistant writes an export module that produces the wrong output, asking whether its code is correct may bring a fluent explanation. Add logging and you can watch the wrong answer assemble itself in greater detail. That trace can be valuable: it may show the exact point where an array became a string. An assertion inside the module can also catch a value that violates the intended contract.
The question is where that contract came from. If the code and its check both assume that a nested array should be flattened, the check can approve the mistake. A passing regression test is evidence that the tested behavior now meets its expectation. The expectation still needs review.
I think of orthogonal verification as checking along another axis: choose an additional check that does not depend on the assumption suspected of causing the failure. A second parser can avoid a defect in my custom parsing code while sharing the same runtime, input and machine.
In synthesis coding, the human-AI collaboration discipline this series describes, an assistant can produce plausible code faster than a person can review it deeply. Trustworthy checking is the scarce resource. That makes the design of your checks an architectural concern. Asking the assistant to explain its work may help locate a problem; its assurance alone does not establish that the output meets the requirement.
Decide what the parsers should return
The downloadable exercise uses this constructed front matter:
---
tags:
- [tag1, tag2]
- [tag3]
---
Body.
The dash introduces an item in the outer sequence, and each bracketed sequence remains a nested item. The YAML collection examples explain these two forms. The expected JSON was written before implementing or running the exercise's parser comparison:
{"tags": [["tag1", "tag2"], ["tag3"]]}
gray-matter separates front matter from the document body and returns the parsed metadata in .data. The package includes gray-matter 4.0.3 and its runtime dependencies, including js-yaml 3.15.2, so this example runs without downloading packages. A deliberately faulty custom producer turns each nested group into a comma-separated string.
The compare-parsers.cjs file checks the library and faulty producer against that same expected value:
const fs = require('node:fs');
const { isDeepStrictEqual: equal } = require('node:util');
const matter = require('./vendor/node_modules/gray-matter');
const { flatteningProducer } = require('./exercise.cjs');
const input = fs.readFileSync('fixtures/nested.md', 'utf8');
const expected = JSON.parse(fs.readFileSync('fixtures/expected.json', 'utf8')).nested;
const outputs = {
library: matter(input).data,
custom: flatteningProducer(input).data
};
for (const [name, data] of Object.entries(outputs)) {
console.log(name, JSON.stringify(data), equal(data, expected) ? 'PASS' : 'FAIL');
}
process.exitCode = Object.values(outputs).every(data => equal(data, expected)) ? 0 : 1;
Run it from the extracted example directory with Node.js:
node compare-parsers.cjs
It exits 1, with the library matching the expected nested structure and the custom producer failing. Disagreement by itself would not decide which parser was right: the intended syntax and result supply that decision.
The shared-wrong control makes the comparator flatten the library's correctly parsed arrays too:
node exercise.cjs parser shared-wrong
Now both outputs contain {"tags":["tag1, tag2","tag3"]}. They agree and both fail the unchanged expectation. The command exits 1. Its trace shows the first group's members before the faulty transformation and the string stored afterward:
{"line":3,"members_before":["tag1","tag2"],"stored_after":"tag1, tag2"}
The trace locates the transformation. The separately written expectation tells us that the transformation is wrong. Keep both when investigating a failure; do not regenerate the expectation from the output merely to make the check pass.
Check the extraction boundary too
In ownwords, my open-source content-syncing tool, I built a separate verification path to check the converter's output. The ownwords case study includes a nested-HTML fixture that exposes a weakness in the earlier published examples: converter and verifier excerpts used the same expression to find the article's end. Both could stop at an inner closing div and discard the same tail paragraph.
The linked converter and verifier use different extraction patterns. Each still needs an expected whole-article boundary; different patterns alone do not establish complete selection.
Extraction, conversion and content comparison answer different questions. Did we select the whole article? Did we transform that selection correctly? Did its words, links and code survive? A word-count check can help flag missing text while overlooking a substitution: “Keep the tail” and “Drop the tail” both have three words. The exercise rejects the changed text despite the equal count.
The same discipline applies across interfaces. Inspect an output file, then query the system that received it. Look at the UI, then read the corresponding database row. Before treating those views as corroboration, check whether they read the same stale cache, use the same identifier or omit the same field.
Verify at each stage
A typical content flow fetches from an API, normalizes, stores, transforms and publishes. An end-to-end check can reveal that the destination is wrong without showing where the value changed. The failures have familiar forms: a date sheds its time component, an encoding degrades into garbled characters, or an identifier quietly stops matching its record. I have chased the date failure myself; the search was the expensive part, because nothing along the flow recorded where the value went bad.
Probe the value at each boundary. These schematic, read-only shell probes use $API and $DEST for your workflow's endpoints and content/record-123.md for the saved record.
# Source of truth: what does the upstream API hold?
curl -s "$API/records/123" | jq '.updated_at'
# After the fetch stage: what landed in the local file?
grep '^updated_at:' content/record-123.md
# After the publish stage: what does the destination hold?
curl -s "$DEST/records/123" | jq '.updated_at'
Check the command's success and the response before interpreting its value. The grep probe only shows a matching line; it does not validate the whole document. A failed request or an absent field is not a successful check returning an empty timestamp.
Compare every observation with the declared contract as well as its neighbor. If a correct timestamp leaves the source and a date-only value enters storage, those observations narrow the investigation to the intervening work. If the wrong identifier enters at the source and survives every stage, neighboring agreement conceals the error.
The local exercise's fictional record must retain exactly these values across JSON, Markdown front matter and the parsed destination:
| Field | Required value |
|---|---|
id | fictional-17 |
updated_at | 2026-02-03T09:10:11.123Z |
text | Café naïve — Keep the tail |
The timestamp is a string with milliseconds and an explicit UTC suffix. This fixture requires that exact representation; it does not silently treat a date-only value as the same information. Timestamp truncation, damaged encoding, a word substitution and a wrong ID each fail. The upstream-id control changes the source ID as well: every neighboring pair agrees, but all three stages fail the fixed expected ID.
Written interface contracts make those expectations reviewable. Data format contracts for AI pipelines develops the date and representation questions further. Stage checks show where an observed value stops meeting the contract.
The ten-second check
Before an operation that creates, modifies or deletes, inspect the state that already exists. About to generate files into a content directory? List what it holds first. The existing layout gives you evidence about the conventions new files should follow; if the conventions conflict, that is something to resolve before adding more files.
My working rule: if checking costs less than ten seconds, always check. A few seconds spent inspecting the directory can save hours cleaning up a wrong assumption spread across a batch operation.
The habit matters with an AI in the loop because a confident answer can conceal a guess about the workspace. In the project's instruction file (CLAUDE.md, for Claude Code) I state it as standing policy: any file operation starts by inspecting existing structure, and when the structure is unclear, ask before proceeding. Look for the directory listing or other evidence that the requested inspection happened.
Read the dry run like a reviewer
A dry run before a batch edit or a migration plan before a migration gives the reviewer something concrete to inspect. The existence of that report does not establish that its proposed actions are the ones you authorized.
Decide what the output should say before looking at it. For an update-only operation, write down the target IDs, expected number of updates and expected number of creates. Updates also write data; the expected number of creates is zero. A plan that names the wrong existing record can be wrong even when it contains no creates.
The exercise's requested updates target two fictional records. Its expected plan is:
[
{"operation": "UPDATE", "id": "fictional-17"},
{"operation": "UPDATE", "id": "fictional-23"}
]
The checker compares the complete plan with that expectation. Changing the first operation to CREATE fails. Keeping UPDATE but changing its target to fictional-99 fails too. The exercise contains no executor: a passing preview does not perform an action.
In synthesis coding the working loop is: the AI proposes, the human reviews, the AI executes. For destructive operations, the dry run makes the review step concrete. Read it against the authorized change, then check that the relevant state has not changed before execution. If it has, the old preview no longer describes the proposed operation on today's inputs.
Run the complete exercise
The ZIP contains the fixtures, their literal expectations, code, tests and dependency licenses. From its extracted directory:
node exercise.cjs demo
node test.cjs
The demo shows the expected parser result, all three record stages and the two-update preview. The test command runs the deliberate failures and checks that they exit with the expected codes.
For a closer look at a stage or preview failure, these commands each exit 1:
node exercise.cjs stages timestamp
node exercise.cjs stages upstream-id
node exercise.cjs plan create
node exercise.cjs plan target
Give each layer a specific job
The strongest check is one the design makes unnecessary. A naming convention makes identity easier to inspect; a uniqueness constraint or explicit collision check can reject a duplicate. A tool that finds an existing resource can propose an update rather than a create, provided it has matched the intended resource. The plan's target still needs checking.
The pre-operation inspection catches a mistaken picture of existing state. The dry-run review catches an unexpected action or target. Stage probes help locate a changed value. A separately reviewed expected result catches the shared flattening assumption that let two parser outputs agree. Each layer gives a mistake missed by an earlier check another chance to be caught. A shared bad requirement or incomplete fixture can still defeat several checks, so review their expectations as deliberately as their code.
That is how I put systematic quality standards into practice: name the failure a check should expose, give it an expected result worth reviewing, and keep a case that fails for the right reason.