An installation check can become stale during the first use of the software it approved. A plugin's hook might run successfully, write a cache file inside the installed package, and leave that package in a state the client refuses to use again. If you maintain plugins for AI coding clients, the acceptance test needs to inspect the installed copy after its code executes.
I want that test to answer two questions separately: did execution leave the package usable, and did the hook accomplish its intended job? A hook is a command the client runs at an event, such as starting or stopping a session. Returning from that command answers neither question by itself.
The installed copy changed
The September 18, 2026 records for the Muse integration of synthesis-skills provide a concrete case. Synthesis-skills is the open-source collection of methods and tools behind synthesis engineering. Its 4.102.0 release had added Muse as a third client; the following day's repair addressed Python bytecode written by its lifecycle hooks. The public release change records that history.
The retained CLI output identifies Muse Code 1.3.0. An early inspection reported synthesis-skills 4.102.0 as enabled, active and valid, with the two hooks still requiring review. A later inspection reported the same version as inactive and invalid, with a diagnostic that the plugin cache was invalid and capabilities were blocked.
A subsequent Start hook test returned a context excerpt. The Stop hook test that followed said its hook was not installed and enabled. Another inspection still reported an invalid package. A search for recently changed files then listed Python bytecode beneath the installed package's __pycache__ directories.
The public repair identifies the cause: Python imports had written bytecode into a package whose contents Muse checked against its lock record. The observed new files and the later repaired behavior support that diagnosis. The retained commands do not give a complete trace of the first transition from valid to invalid.
Importing a Python module can create a compiled .pyc file beside the source, under __pycache__, while leaving the source itself unchanged. A check limited to whether the Python source changed would miss the added file.
The repair and its installed test
The 4.102.1 patch added this line to both Muse hook wrappers, before they launched Python:
export PYTHONDONTWRITEBYTECODE=1
Python's command-line documentation explains the setting: a nonempty value prevents Python from trying to write .pyc files when importing source modules. It controls that bytecode behavior; it does not prevent arbitrary code from writing other files.
The installed test after the repair exercised Start twice. The first invocation reported completion. The second returned another context excerpt. Inspection then showed 4.102.1 active and valid, with both hooks marked trusted_enabled.
Running the hook twice matters because the first execution can change the conditions for the second. A release test that stops after the first returned result can overlook that change. I would retain a package manifest before execution and compare it after each invocation, including added files as well as changed and removed ones.
The later Stop test returned a hook response, and the following inspection counted two trusted-enabled hooks. A search of Muse's plugin-cache directory displayed no __pycache__ entries modified in the preceding ten minutes.
Its checkpoint result said checkpoint_accepted was false, and the output was cut off inside the accompanying message. The hook ran, but this record does not establish an accepted project checkpoint or explain the complete reason for the rejection. A later recorded Start receipt reports binding to the native session transcript. That receipt does not turn the rejected Stop checkpoint into a successful one.
Make the mutation visible
The mechanism can be demonstrated without installing an agent client. The accompanying synthetic example uses a Python hook that imports a helper module. It runs twice in each of two fresh package directories, once with ordinary bytecode behavior and once with the setting above. A manifest records each relative filename and its SHA-256 hash.
This example is newly constructed. It is not a reproduction of the historical Muse installation, and it does not implement Muse's cache validator. It measures whether these fixture files change.
The measured result under CPython 3.12.3 was:
| Fixture | After first invocation | After second invocation |
|---|---|---|
| Ordinary helper import | Added __pycache__/helper.cpython-312.pyc; original source files unchanged | Package still differs from its initial manifest |
| Bytecode writing disabled | Package matches its initial manifest | Package still matches its initial manifest |
All four processes returned zero. In all four, the fake hook deliberately emitted a checkpoint result of false. That chosen output makes the distinction visible: process completion, package integrity and an application-level result can disagree.
The runnable example retains its generated files and report in a new directory. It refuses an existing output directory, launches the child interpreter directly, and records each return code separately. It invokes no agent client and changes no real plugin configuration.
What the acceptance record should retain
For a real plugin, I would start with the exact installed version and the client's own validity and trust report. Record the package contents before execution. Then invoke the relevant hook through the client, retain its complete result and individual process status, and inspect the installed package again. Repeat the invocation to test the state left by the first run.
The application result needs its own assertion. For a checkpoint hook, that might mean an explicitly accepted checkpoint whose recorded state matches the intended project. A context-producing hook needs evidence that the intended session received the context. The assertion should name the outcome the hook is supposed to deliver.
Several commands in the historical record shortened their output with head or filtered it through a pipeline. Their aggregate zero exit status cannot certify every preceding command. When building the acceptance test, capture full outputs and separate return codes first; generate a concise display from that retained evidence.
This case supplies a specific test for the installed and live behavior described in the five-plane conformance model. The extra observation is the package's state after execution. A result that says the hook completed should sit beside that observation and the hook's application result, so a reviewer can see exactly which conditions passed.
Case sources: preserved CLI outputs from September 18, 2026, and the linked public 4.102.1 repair. The operational transcript is not published here. The synthetic example was constructed for this article and is separate from that historical evidence.