Model collapse used to be a research term. Now it shows up in AI-risk conversations, usually with the same framing: models trained on the output of other models can degrade, the open web is filling with synthetic text, and the companies training frontier models have a data-quality problem on their hands. That framing invites a conclusion I keep hearing from technology leaders: this one is not our problem. We do not train foundation models. We buy them. If a vendor's training pipeline fills with synthetic sludge, we can switch vendors.
You do not control what the labs train on. You do control your knowledge base, your retrieval pipelines, and the decision to promote a draft into a source of record. When a summary becomes easier to find than the decision it summarizes, an editing error can become an operating instruction.
The practice I have in mind is synthesis engineering, the discipline I have been writing about for building production systems with AI while keeping architectural control and human understanding intact. Those principles apply to the knowledge a system uses as well as the code it runs.
How a summary acquires authority
Documents summarizing other documents. A model drafts a design document from a meeting transcript. Another summarizes that document into a wiki page. A third answers questions from the wiki. If a condition disappears from the first draft, each later document can carry the omission forward.
Suppose a team records this decision about a new data importer: enable it for internal accounts after the retry tests pass; customer rollout remains unapproved. A summary reduces that to “the team approved the new importer.” A later launch note, based on the summary, says customer rollout is approved.
The summary drops both the test condition and the limit on who can use the feature. The launch note then asserts an approval the original record does not provide.
| Record | What it says | What a reader can establish |
|---|---|---|
| Original decision note | Internal accounts only, after retry tests pass; customer rollout unapproved | A conditional decision with a defined limit |
| Derived summary | The team approved the new importer | The summary has lost the condition and scope |
| Launch note citing the summary | Customer rollout is approved | The cited chain does not support this claim |
The original note still exists. Returning to it can repair the claim, and a careful summary could have preserved the decision correctly.
Retrieval corpora that ingest whatever exists. Retrieval-augmented generation, or RAG, gives a model documents to draw on when answering. That helps only if what the documents say is itself grounded. Index the launch note and its unsupported approval can arrive in the next answer with a citation attached. The citation shows where the assistant found the statement; it does not supply the missing approval.
Knowledge bases that cite themselves. An FAQ built from the launch note adds another document without adding independent support. Meanwhile, the decision record it should depend on may become harder to find or fall out of date. Repetition can make a claim look established even when every version leads back to the same mistake.
Inside an organization, this can look like documentation that reads plausibly but says little specific to you. Pick a page from your knowledge base and ask whether it could have been written about any company in your industry. If it could, investigate what decisions, constraints, and local experience are missing.
What the research establishes
Model collapse concerns the training of successive models on generated data. A 2024 Nature paper studied recursive training under specified conditions and found that information about the original distribution, including its less common events, could be lost. A separate study of data accumulation found that retaining the initial dataset alongside successive generated datasets kept validation loss stable or lower in its language-model experiments. Its image-model experiments still showed some deterioration.
Those results depend on the training setup. Rewriting a project note does not, by itself, retrain a model's weights. The importer example is a failure of document provenance: the chain of records no longer supports the claim someone wants to act on. Good recordkeeping addresses that failure; these studies do not establish it as a cure for model collapse.
What you control
You control what your systems treat as ground truth. You decide which documents the retrieval index trusts, and what a page has to pass before it becomes the version people and machines rely on. You decide whether an AI draft can become a source of record without a human ever having owned it. These are architecture decisions, and they are yours.
The original four pillars of the synthesis engineering framework apply directly here: human architectural authority, systematic quality standards, active system understanding, and iterative context building. I framed them for software teams. They also give leaders a way to govern organizational knowledge.
Human architectural authority determines who can make a decision and which record establishes it. Quality standards determine what review a summary needs before others rely on it. Active understanding means someone can trace and repair a claim. Iterative context building keeps those decisions and corrections available for the next piece of work.
Context that compounds
A model can produce a plausible description of the average company while missing what makes yours different. Give it durable records of your actual decisions and constraints: why an architecture was chosen, which customer commitments apply, what an experiment found, and what the team ruled out.
This is iterative context building at the level of the organization. Project records, architecture notes, and a knowledge layer designed for AI to query make your specifics available in the work itself. A corrected record can inform the next engineer, the next summary, and the next review.
Teams that maintain this notice a second effect: the context compounds. Each correction becomes a resource for later work. That depends on retrieving the relevant record and using it faithfully; accumulating documents alone does not produce the benefit. The strategic investment is in a body of knowledge the organization understands and can keep improving.
Keep the decision and its evidence together
For the importer example, I would keep the original decision record and link the summary's approval claim directly to it. The link should identify the relevant passage and the version or date of the record. A link to a changing wiki homepage leaves the next reader with another search.
The corrected summary could say:
Internal use is approved once the retry tests pass. Customer rollout remains unapproved. The decision note is the source for these limits; the test result is still needed before enabling internal use.
The note establishes what the team decided, but it does not establish that the tests subsequently passed. Those are separate claims with separate evidence.
A primary record is the underlying record a claim depends on: for example, the decision note, the original transcript or the actual test result. Its role depends on the claim. A transcript can establish what someone said; the team's decision process determines whether that statement constituted an approval. A test result can establish what a particular run checked; it cannot establish that every deployment has the same behavior.
A useful project summary often includes more than facts. It may recommend a next step, infer a risk or identify a disagreement. Keep those contributions, but identify them as interpretation where the distinction affects a decision.
In the example, “customer rollout needs a separate approval” follows from the original limit. “The team probably intended to launch next week” is an inference that needs support. If the source does not provide it, leave the timing unresolved. Adding that date would create a claim the decision record does not support.
Make review part of the knowledge system
AI-generated text should meet the same quality bar as human-written text before it ships. Treat consequential AI output as a draft until a person has read it, corrected it, and taken ownership of it. Human authorship does not make a record accurate, and an AI-written summary can preserve a source faithfully. Review the claim and its support in either case.
Keep the source material available under appropriate access controls, label summaries as derived records, and make their review status visible wherever they are reused. If a draft summary is allowed into a search index, its draft status needs to travel with the retrieved passage. Otherwise, a cautious label on the original page may disappear from the answer.
Record which claims were checked, against which source versions, and what remains unresolved. Recheck the relevant claims when those sources change.
For the importer, the reviewer can confirm the internal-use condition and the customer-rollout limit from the original note. Someone still needs to locate the retry-test result before enabling internal use. If a later decision authorizes customer rollout, link that decision and mark the earlier limit as superseded rather than silently rewriting its history.
When a consequential claim cannot be traced, record that limit. Here the actionable state is “customer approval not established by these records,” followed by a request to the decision owner for the missing record or a new decision. Do not reconstruct an approval from what would make the launch plan convenient.
Active system understanding supplies a practical test. For code, it is whether an engineer could debug the system at 2 a.m. For knowledge: when this page turns out to be wrong, can someone trace the claim to its source and fix it there? A source link, a responsible owner, and a way to correct the record make the answer useful beyond the current conversation.
Judgment does not delegate
Humans decide what the organization will treat as established knowledge and take responsibility for the decisions it enables. Models can draft, summarize, translate, and refactor at a speed that makes this work far more accessible. Their output acquires authority through the organization's decision and review process.
Someone owns every consequential claim your organization relies on, or nobody does. The part of the work that was always judgment is the part that does not delegate.
Make new steel
In March 2025 I wrote about pre-AI content as the new low-background steel. There is a real industrial history behind that analogy: prewar steel has been used in radiation-sensitive measurement systems. But the useful material is not confined to a finite stock of old metal. Researchers have also manufactured and tested low-background stainless steel. Material quality depends on how it is produced and checked.
Preserve valuable archives, and keep producing good new records. Pre-AI archives may become especially valuable for training and research when their provenance is known. People in your organization write, decide, correct, and verify every working day. Older text is not automatically true, and a review mark is not a purity seal. The value comes from maintaining the connection between what a record says and what supports it.
Start with a summary that already guides work. Pick a statement that authorizes an action, names a constraint or assigns responsibility. Follow it to its source and check the qualifiers. Keep the summary if it helps people find and understand the record; correct any claim whose wording asks the source to support more than it says.
The constraint on how much AI can do for your organization is shifting from the capability of the models to the quality of the material you ground them in. You can grow that material through work your organization already needs to do. Keep judgment human, keep the gates honest, and keep making new steel.