Slop detection, not AI detection

Editors have been asking the wrong question: was this written by AI? The question that matters is whether the piece is worth publishing.

Some of the prose I read every week is unedited LLM output: copied, lightly tweaked, sent on. Some of it is human-written, carefully styled, and carries no claim a reader can disagree with because no claim is being made. Both fail the editorial test. A piece can be AI-generated and excellent. A piece can be human-written and unreadable for the same reason a sales pitch is unreadable: it has nothing to say.

Slopcheck is the tool I built around that distinction. It examines style patterns separately from substance and depth. An editor needs to know whether a passage repeats an idea, hides a missing fact behind polished language, or makes a claim its evidence cannot carry. Knowing that a phrase resembles a model's usual output does not settle any of those questions.

Two readings, followed by editorial judgment

The analysis is organized around two readings. One matches wording and structure against a catalog that includes eight model families: Claude, GPT, Gemini, Llama, Grok, DeepSeek, Mistral and Qwen. The other examines substance regardless of authorship: what would be lost if a paragraph were deleted, whether a business claim could apply unchanged to almost any company, and whether an impressive sentence actually says anything.

The AI-provenance reading identifies patterns worth examining. Its family labels do not prove who wrote a draft, and a confidence rating does not establish an accurate probability. An editor can judge the cited passage and the reason for changing it.

Separating those readings still helps. A draft can look model-like and contain a well-supported argument. Another can have few recognizable model patterns and be empty. A third can have both familiar patterns and substantive weaknesses. A fourth can trigger few findings on either reading and still contain a factual error, a missing attribution or an argument that serves no reader. None of those combinations supplies an automatic publication decision.

An op-ed whose author used an LLM to organize their thoughts may be worth running with appropriate disclosure. Carefully styled human writing with nothing to say should not get a free pass. Editors can make that distinction without accusing a writer of undisclosed AI use on the strength of a pattern match.

A 2023 study by Weixin Liang and colleagues found that the GPT detectors they tested misclassified writing by non-native English writers and could be bypassed with prompting. The arms race between content production and content detection is structural; it does not end.

Checking a finding against the draft

Consider this example:

Our new review process has dramatically improved the quality of our work and made collaboration more effective.

An editor can identify a specific problem here. The sentence asserts an improvement without saying what changed, how quality was assessed or what became easier for collaborators. Replacing “dramatically improved” with a quieter phrase would leave that gap intact.

A useful review asks the writer for the missing evidence. Was a second reviewer added? Did the process catch an error before publication? Is there a comparison with the previous process? If the writer has none of that information, the proposed improvement cannot be presented as a demonstrated result. The writer can describe the intended goal or remove the claim. The tool must not invent a metric, an incident or a success story to make the sentence more convincing.

A finding should give the writer enough information to make a decision:

  1. Quote the exact passage. A label such as “generic” is difficult to act on without a location.
  2. Explain the reader's problem. Identify the missing evidence, repeated idea, unclear reference or unsupported conclusion.
  3. Test the criticism against the surrounding text. The evidence may appear in the next paragraph; a repetition may be needed after a long explanation.
  4. Suggest a change that preserves the writer's meaning. If a fact is missing, ask for it. Before cutting a sentence, check that the remaining passage carries the information the reader needs.

These checks apply to the tool's recommendations as well. A polished rewrite can introduce an error, erase a useful qualification or make a claim stronger than its source supports. It can also drain an argument of conviction by treating an opinion as a defective measurement. “I want this service to remain free” expresses a commitment; it does not need to be rewritten as uncertainty about future prices.

There should also be room for a finding of no problem. In an explanation of a color puzzle, “Each fusion reduces the total number of eggs by one” may be exactly the sentence the reader needs. Its plainness is useful. An unusual verb, a personal anecdote or a more elaborate rhythm would not make the arithmetic clearer.

A catalog that retains its history

The methodology uses a compounding archive: keep old patterns, mark their era, and add new observations. The early ChatGPT "As an AI language model" preamble, Bard's "as a large language model trained by Google" stem, early Claude disclaimers and older refusal templates remain useful examples when examining writing from those periods. Keeping them does not mean they occur only in those periods, or that later models have universally stopped producing them.

This gives an editor examining a 2023 draft access to the same accumulated catalog as an editor checking a new one. Historical labels help the editor interpret an observation in context.

The original catalog covered eight model families, roughly one hundred family-specific patterns, seventeen substance-and-depth sub-patterns and seventy-six cross-family criteria.

Its substance tests draw on work including Harry Frankfurt's On Bullshit (2005), Hicks, Humphries and Slater's 2024 paper, and Pennycook and colleagues' 2015 study of pseudo-profound statements. These works help articulate problems such as indifference to truth and impressive language without adequate meaning.

The methodology article explains the catalog and its contribution template. A report that identifies a false alarm, a missed problem or an exception worth preserving helps improve the method. Adding another forbidden word does not necessarily do so.

Update, October 4, 2026: The workflow and privacy sections below reflect the current installation options, service limits and data handling.

Choose the workflow that fits the work

The repository explains how to use Slopcheck from your browser, a terminal, or the tools where you already read and write.

WorkflowHow to use it
Web appPaste a draft and compare the style and substance readings. The hosted tier has a 200,000-character default submission limit and a daily allowance. With a personal key, provider limits apply.
CLIInstall from the installation hub, which lists current choices including Homebrew, npm, Bun and direct downloads.
BrowserSend selected text or a page to Slopcheck with the bookmarklet or Chrome/Edge extension. The extension captures up to 8,000 characters; the repository explains how to load it unpacked.
Chat toolsUse the Slopcheck Custom GPT or follow the Claude Project setup recipe for the chat tool you already use.
IntegrationsFollow the setup guides for an Apple Shortcut, self-hosted Slack and Discord bots, or email submissions.

Editors who read submissions by email can forward the message body to [email protected] and receive the analysis in reply. Per-sender limits apply. The repository also includes instructions for running this email workflow on your own domain.

Updated: November 3, 2026: The two paragraphs below now give SlopCheck's published installers (its npm package is @synthesiswork/slopcheck, and the AUR carries no SlopCheck package) and install the skills through the synthesis plugin, which also replaces the direct-copy command mentioned in the July 30 update.

If you ship software and want the analysis in your pipeline, the CLI installs via npm install -g @synthesiswork/slopcheck, brew install synthesisengineering/tap/slopcheck, or curl -fsSL https://github.com/synthesisengineering/synthesis-slopcheck/releases/latest/download/install.sh | sh. Source is at github.com/synthesisengineering/synthesis-slopcheck.

If you use Claude Code, Codex or Muse, install the synthesis plugin: curl -fsSL https://raw.githubusercontent.com/synthesisengineering/synthesis-skills/stable/onboard.sh | sh. The slop-detection capability becomes part of the agent's editorial repertoire. Other agents that read Agent Skills can use the skill files; the install guide covers both.

Free to use, with a data path worth understanding

I built Slopcheck as a public service. There is no paywall and no plan to add one. GitHub Sponsors provides a voluntary way to support it, one time or on a recurring basis.

Free software does not make model usage free. The hosted tier works within a community budget and rate limits. With a personal API key, the browser sends the analysis request directly to the selected provider, whose charges and limits apply. The hosted mode sends it through the service's proxy to a provider. These are different data paths, and a writer deciding what to submit should know which one is in use.

The web app needs no signup or account. I do not harvest email addresses or use submitted drafts for analytics. Slopcheck's hosted application keeps expiring usage and budget counters; it does not store submitted drafts or analyses. I do not use submissions for leaderboards, a public archive, examples galleries, or screenshots in blog posts or social media.

Your chosen AI provider has its own data-handling rules. Use a provider arrangement suitable for any confidential material you submit.

The app repository identifies its license as MIT and the skill catalog as CC0. The architecture article explains the multi-pass analysis, hosted-tier controls and tools-router design in more detail.

The writer keeps the choice

Synthesis writing is one form of the craft, available to anyone who wants it. That has been the position since introducing synthesis writing. I do not see a calligrapher's craft invalidated by a word processor, or a writer's choice to work without AI invalidated by another writer's use of it.

Writers and editors can apply these tests without AI. If an editor uses an automated service on someone else's unpublished work, they should agree on what the review involves and what may be submitted. The point is to improve the piece, not to make adoption of a particular tool a condition of taking writing seriously.

Try Slopcheck on a draft you can appropriately submit, then compare each criticism with the passage and its sources. Keep the useful findings. Report what fires that should not, or what slips through that you would have caught. Before accepting a rewrite, ask what it helps the reader understand and what of the writer's meaning it might remove.