I was reviewing timing instrumentation an AI coding agent had added to three parallel API services in a single session. Each service was supposed to capture a timestamp before an external API call, compute the elapsed time afterward, pass it through the response and write it to the database. All 1,297 tests passed.
Something about the third service bothered me. I opened the database model for its output table. The timing column wasn't there. The service calculated the value and the route assigned it to the model object, but the model mapping and the migration to add the database column were missing.
In that implementation, the tests used fresh databases generated from model definitions, and no test exercised the new value through the complete write path with a real commit. The green suite hadn't answered the question I needed answered: would this service retain and return the timing value?
A missing mapping and a missing database column can produce different failures. In the SQLAlchemy example below, assigning an ordinary, unmapped attribute leaves a value on the Python object but doesn't persist it. If the attribute is mapped to a column that the actual table lacks, the attempted insert raises an error.
Follow the value through each boundary
For a schema change, I want a concrete value to follow. “The timing field is wired up” leaves too much room for assumptions about units, names and persistence. A request carrying 1,250 milliseconds should store 1.25 seconds and return 1,250 milliseconds after a fresh read. An upgrade must also keep the rows already in the database.
The downloadable workbook uses three fictional services, alpha, beta and gamma. It calls a request handler directly, writes to local SQLite files and reads a response through a new SQLAlchemy session. It does not run an HTTP server or reproduce the historical application. The recorded checks used SQLAlchemy 2.0.30, Python 3.12.3 and SQLite 3.45.1.
The first request has ID 8 and this value contract:
| Boundary | Name | Expected value |
|---|---|---|
| Input JSON | elapsedMs | 1250 milliseconds |
| Python model attribute | duration_seconds | 1.25 seconds |
| Database column | elapsed_s | 1.25 seconds |
| Response JSON | elapsed_ms | 1250 milliseconds |
The names differ intentionally. SQLAlchemy permits a Python attribute to map to a differently named SQL column. Here, the object-to-database mapping, or ORM mapping, includes:
duration_seconds: Mapped[float | None] = mapped_column(
"elapsed_s", Float, nullable=True
)
The request handler divides milliseconds by 1,000 before assigning the mapped attribute. It commits the row. The response code opens a new session, retrieves that row and converts seconds back to milliseconds.
Checking the value on the original Python object would miss several possible defects. It could be an ordinary attribute the ORM never writes. The transaction might have been flushed to the database connection but never committed. A response assembled from the request could look correct even if storage failed.
The workbook therefore checks the actual stored rows with Python's separate sqlite3 reader, checks a freshly loaded ORM object, and compares the returned responses with literal expectations written before the implementation. A second request carries 2,750 milliseconds, and a third carries zero. Those inputs expose a constant response and distinguish a valid zero duration from a missing value.
Searching for a field name is useful for finding code. It cannot establish this contract: the legitimate names here are different, and a matching name could still carry the wrong units.
Test a new database and an existing one
A fresh database asks whether the current schema and code work together. An upgrade asks whether the change can move the previous schema forward while retaining its data.
The workbook runs both paths for each fictional service. On the fresh path, SQLAlchemy creates the current table before the new rows are written. On the upgrade path, the starting table has only id and label, including row 7 with the label kept before upgrade.
Calling create_all with the new model leaves that existing table unchanged. SQLAlchemy's table-creation documentation explains that it checks for existing tables before creating them; it is not a general schema-upgrade operation. The exercise then applies its explicit migration:
ALTER TABLE measurements ADD COLUMN elapsed_s REAL;
Afterward, row 7 retains its ID and label. Its new column is NULL, because this exercise specifies no backfill. Row 8 must store 1.25 and return 1250, just as it does on the fresh path. The other two requests must also survive the roundtrip. This is a deliberately small migration for one known previous schema; a real change needs explicit decisions about old rows, defaults, constraints and deployment order.
The download also separates the two missing-field mechanisms:
- Unmapped attribute: The old model has only
idandlabel. Assigningduration_seconds = 1.25afterward adds an ordinary instance attribute. The commit succeeds for the mapped fields, but a freshly loaded old-model object has noduration_secondsattribute. - Mapped column absent from the table: The new model expects
elapsed_s, but the database still has the old table. The insert raisesOperationalErrorfor the missing column. After rollback, row 7 remains.
To run the workbook, extract the ZIP, create its runs directory and follow the README. It includes the pinned Python dependency wheels, so the checks require no package installation or network service. The fault controls change only the implementation while keeping the tests and expected values fixed:
| Deliberate defect | What the checks detect |
|---|---|
| Omit the migration | The fresh path works; the upgrade cannot insert the mapped column. |
| Assign an unmapped attribute | The original object has the value; the SQL column and fresh response are NULL. |
| Store milliseconds as seconds | The stored value is 1250.0, and the response becomes 1250000. |
| Return a constant without reading storage | The second duration and zero fail the response contract. |
| Skip gamma | The inventory detects the missing third service. |
| Flush without committing | The same-object check passes, but SQL contains no new row and a fresh read fails. |
All six wrong implementations were rejected. An equivalent implementation that multiplies by 0.001 instead of dividing by 1000.0 passed the same checks. These are selected faults with inspectable consequences, not exhaustive coverage of every way a schema change can go wrong.
Find what the suite actually exercises
My historical suite missed a particular write path. That does not make migrations, integrations or deployment conditions inherently untestable. For example, Django's documented default test runner in version 4.2 creates a test database and runs migrations. Even then, creating a schema from an empty database differs from upgrading a populated database from the previous release.
Four gaps from my implementation reviews are worth checking explicitly:
Missing migrations. A test that creates the latest schema directly can pass while an existing installation lacks the new column. Exercise the actual upgrade path from the supported previous schema, with old rows whose preservation and new-field treatment you assert.
Mocks that stop before the risky boundary. A route test with a mocked service can check how the route handles that return value. It leaves the real route-to-service interaction untested. Keep that focused test, then add a check through the actual service and persistence path when that is the behavior being changed.
Cached static assets. A component can compile and pass frontend tests while users receive older JavaScript from a cache. Inspect the asset URLs and bytes the built page and browser actually consume. A source-file assertion does not establish which bundle was served.
Different environments. A SQLite test does not establish PostgreSQL behavior. SQLite's flexible typing, for instance, can accept values that expose problems when moving to a more rigidly typed database. Test the behavior on the engine you deploy. Likewise, a variable present on a developer's machine may be absent from CI or deployment configuration; inspect and exercise the configuration the runtime will actually receive.
Each gap points to a missing check or a different consumer. Tests can cover those consumers when the setup and assertions include them.
Check every service's required layers
In my review, the first two services had the required changes across five layers. The third stopped after the route handler:
| Service | Service logic | Return value | Route handler | Model column | Migration |
|---|---|---|---|---|---|
| First | Present | Present | Present | Present | Present |
| Second | Present | Present | Present | Present | Present |
| Third | Present | Present | Present | Missing | Missing |
Two complete instances can make the third look complete by association. The first implementation forces you to think through the pattern. By the third, repetition can feel like verification. The third implementation inherits your trust from the first two without inheriting your diligence. The velocity that makes AI coding agents valuable is the same velocity that makes systematic verification essential. A human might implement the same pattern across three components in an afternoon. An AI agent might do it across a dozen in twenty minutes, producing code faster than you can review it.
When an AI agent applies a pattern across many components, I want the list of affected components before I start spot-checking. For each one, I record the layers the change requires and the evidence for them. The last service gets the same examination as the first. If a layer is legitimately unnecessary for one instance, I want the reason recorded instead of a copied checkmark.
Three complementary review scopes
Self-review is a distinct discipline: before handing over a change, the implementer should account for everything it requires. Peer review can challenge that account, and a broader audit can find patterns across changes. Unit, integration, end-to-end, property-based, mutation and fuzz tests, coverage tools, CI pipelines and pre-merge gates can all contribute evidence. Inspection helps determine whether those checks reach the changed behavior and which ones are missing.
I described this approach in an earlier article on installing methodology into an AI workflow, and codified the three scopes as Agent Skills:
| Scope | Skill | Core question |
|---|---|---|
| Single change | synthesis-implementation-integrity | Have I completed and checked the required parts of this change? |
| Proposed merge | synthesis-pr-review | Should this change enter the codebase? |
| Entire system | synthesis-codebase-review | Is this system healthy? |
The implementation review belongs after the work is built and before the pull request. It should produce evidence a peer can inspect, including gaps it found in the test suite. A reviewer can then assess both the implementation and whether those checks answer the right questions.
Seven passes through the change
-
Chain completeness. Trace the value from origin through transport, validation, transformation, storage, retrieval and presentation. Record the name, unit and expected value at each boundary. Follow aliases and serializers; then check stored and returned values.
-
Placeholder detection. Inspect changed files for unfinished branches,
TODOandFIXMEmarkers, temporary values and stubs. A marker is a clue to inspect, not proof of a defect. A constant duration in this request path would fail the second-input check; a fixed value in an explicitly synthetic test fixture may be exactly what the test needs. -
Test honesty. Read the assertions and identify the real code path they reach. Temporarily omit the migration or break the unit conversion in an isolated copy and keep the tests unchanged. The test should fail for the expected consequence. An unchanged test file can already cover new behavior; the useful question is whether it detects this defect.
-
Environment parity. Compare fresh and persistent databases, the actual database engines, local and cloud storage, single- and multiple-instance execution, and source assets versus cached delivered assets. Run the checks required by this change against the relevant consumer.
-
Each-instance verification. Enumerate all affected services and inspect their required layers. Confirm the third service's row through its own write and read path. Do not use the first service's result as its evidence.
-
Companion changes. Check the artifacts the implementation depends on: a migration for the schema change, deployment configuration for a new variable, and a frontend or other consumer update when its contract changes. For this field, the model declaration and the migration solve different parts of the problem.
-
Boundary behavior. Exercise invalid input and the relevant external-service failures: slow responses, malformed results and unavailability. The workbook rejects missing, negative, fractional and boolean duration inputs under its chosen integer-millisecond contract. Your service needs its own explicit behavior for each boundary it exposes.
One useful challenge is to remove or break the new behavior while keeping the tests. If nothing fails, inspect the coverage. The workbook's uncommitted-row variant demonstrates why an assertion on the original object can survive even after persistence is broken.
Five habits for the review itself
The hardest part of self-review is the posture. After building a change, I have a ready-made explanation for why it should work. I need checks that can contradict that explanation. Would I deploy it at 5 PM on a Friday, confident nothing will page me at 2 AM? If I haven't checked the migration, that confidence is hope.
- Open the actual file. Remembering that I added a column is not evidence that the declaration or migration exists. Read both.
- Read what passing tests assert. A green result is useful evidence for the behavior exercised. Identify the mocks, setup and omitted paths before extending that conclusion.
- Verify each implementation in a repeated change. Give the last instance the same required checks, and account for legitimate differences between instances.
- Use the compiler's guarantees precisely. Compilation and static analysis can catch much more than syntax. TypeScript's type checker can detect invalid operations and property access before execution. Those checks still do not establish that an existing database has received its migration or that the deployed process has the required configuration.
- Make confidence answerable to a result. Name a plausible failure and the assertion that would reveal it. If the evidence is missing, record the unanswered question rather than substituting a confident completion message.
Use the skills with your agent's actual workflow
For AI-written code, this verification gap is the biggest practical risk. I want the completion report to identify the inspected files, executed checks and unresolved conditions. After the agent finishes building, I ask it to run an implementation-integrity review: a structured challenge to its work before it reports success. The report still needs examination; invoking a skill does not establish that every required path was checked.
The three skills cross-reference the change review, proposed-merge review and periodic system audit. Their Agent Skills format makes instructions and supporting resources reusable. It does not establish identical discovery, permissions or execution across Claude Code, Cursor, Codex CLI, Copilot and other tools. The format specification leaves some execution details to the host; for example, supported script languages depend on the implementation.
Updated: November 3, 2026: The next paragraph now gives the current install command. The skills install as the synthesis plugin for Claude Code, Codex and Muse; other tools that read Agent Skills can use the skill files, as the install guide explains.
Install the synthesis plugin in Claude Code, Codex or Muse with one command:
curl -fsSL https://raw.githubusercontent.com/synthesisengineering/synthesis-skills/stable/onboard.sh | sh
All three skill files declare a CC0 license in the public repository.
The repair in my original review was three lines: a column definition, an import and a migration. The hard part was noticing the incomplete third service amid passing tests and two complete implementations. Testing is necessary. It isn't sufficient to establish what the tests never checked. For the next schema change, I want the review to show the new value read back from storage on both database paths, with the old rows accounted for. That gives the passing suite a specific claim to support.
Use the skills, adapt them, build on them. If they save you one Friday night page, they've earned their keep.