Essay / criticism / software culture

When To Call A Project "AI Slop," And When Not To

There is a real problem with low-effort AI-generated projects. There is also a real problem with people learning one bad pattern, then applying it carelessly to everything in sight. Those two realities now collide in public software culture, where some projects deserve criticism, some deserve scrutiny, and many get dismissed by people who mistake familiarity for analysis.

Posted | Back to blog archive | Home

1. The real problem is not just AI, but failed evaluation

It is easy to frame the current debate as a simple war between "real work" and "AI slop." That framing is emotionally satisfying, which is usually a terrible sign. Human beings love easy categories. They reduce effort, preserve ego, and let people perform intelligence without doing the expensive part, which is actual thinking.

The deeper issue is not merely that AI-generated work exists. It is that the presence of AI has degraded the way many people evaluate work. Instead of asking careful questions such as What is this project trying to do?, How coherent is the architecture?, Does the implementation reveal genuine understanding?, and Can the author explain tradeoffs?, many observers now jump straight to a social verdict.

"This feels fake" has quietly replaced "I investigated this and found concrete reasons to doubt it."

That shift matters. Once suspicion becomes aesthetic rather than technical, evaluation decays. A person no longer needs evidence. They only need a familiar vibe. If the project has a polished readme, a confident tone, ambitious claims, or code patterns that resemble common AI output, that can be enough for some people to declare it fake. Not questionable. Not worth examining. Fake.

That is not criticism. That is pattern-triggered dismissal. Sometimes it lands on real slop. Sometimes it lands on legitimate work. Either way, the method is broken.

2. How pattern recognition goes wrong

Pattern recognition is one of the most useful and dangerous tools in human judgment. Experts depend on it. Amateurs also depend on it. The difference is that experts usually know when a pattern is only a clue, while amateurs treat the clue as a verdict.

In software culture, people rapidly learn a few recurring characteristics of bad AI-generated projects. For example:

  • inflated promises with no grounded implementation details
  • generic architecture diagrams that say almost nothing
  • code that looks syntactically valid but structurally uncommitted
  • documentation that sounds polished yet strangely non-specific
  • feature lists that are broad, impressive, and suspiciously disconnected
  • repositories where everything appears assembled but little appears tested

Those are real patterns. The problem begins when people stop using them as reasons to investigate and start using them as reasons to stop investigating.

A pattern is not proof. It is a prompt. It tells you where to look next. That distinction is the difference between criticism and superstition.

Why this failure happens

Once people see enough obviously AI-generated projects, their threshold for suspicion collapses. They begin overfitting. In machine learning terms, they train on noisy public examples, then generalize badly. In ordinary human terms, they see one kind of mess often enough that they start hallucinating it everywhere. Humans do this constantly. Religion, politics, fandom, workplace gossip, software communities. Same animal, different costume.

If ten bad projects all share one trait, people begin to believe that trait itself is disqualifying. But real projects can share surface traits with fake ones for completely ordinary reasons. Ambitious projects often sound ambitious. New developers often write awkward documentation. Highly technical work can be uneven, overexplained, underexplained, or inconsistent because real humans are inconsistent. A repository can be messy because it is alive, not because it is fake.

3. Why dismissal culture spreads so easily

Dismissal culture spreads because it is cheap. Real evaluation costs time, attention, humility, and enough competence to know what you are looking at. Dismissal costs a sentence.

There are several social incentives that make this worse.

Status through skepticism

Calling something fake can make a person appear sharp, discerning, and hard to fool. This image is often maintained whether they are correct or not.

Fear of praising bad work

Many people would rather wrongly dismiss a legitimate project than risk looking naive by praising something low-quality or fraudulent.

Algorithmic incentives

Harsh verdicts spread faster than careful analysis. "This is AI slop" travels. "I checked the commit history, code consistency, implementation depth, and test behavior" does not.

Collapsed trust

Once communities are flooded by low-effort output, general trust falls. People stop evaluating individuals and start evaluating by category.

This is how legitimate projects end up caught in the blast radius. Not because they were carefully analyzed and found lacking, but because they appeared inside a culture where suspicion has become recreational.

A community can become so tired of being fooled that it starts rewarding people for accusing first and thinking second.

4. Common mistakes people make when judging projects

Mistake 1: Treating polish as evidence of fakery

Some people assume that if a project has a clean presentation, dramatic language, or a confident tone, it is probably AI-generated or deceptive. That is lazy. Presentation quality tells you almost nothing by itself. Some fake projects are polished. Some real projects are polished. Some fake projects are chaotic. Some real projects are chaotic. Polish is style, not proof.

Mistake 2: Treating ambition as evidence of dishonesty

A project aiming high is not automatically fake. Unrealistic claims can be suspicious, but ambitious scope by itself is not evidence. The real question is whether the implementation shows corresponding depth, constraints, and technical tradeoffs. Real builders often overreach. Fake builders often overstate. Those are not the same thing.

Mistake 3: Confusing inconsistency with inauthenticity

Real projects are inconsistent all the time. They contain old code, new code, abandoned ideas, sudden quality drops, odd naming conventions, ugly fixes, and unfinished experiments. Humans build unevenly. AI also produces inconsistency, but inconsistency alone proves nothing. You have to ask what kind of inconsistency it is.

Mistake 4: Using single-signal judgment

One suspicious trait is not enough. Not generic comments. Not a certain style of error handling. Not a README that sounds a bit corporate. Not a codebase that has verbose abstractions. None of these alone justify a hard verdict. Strong judgments require converging evidence.

Mistake 5: Evaluating authorship instead of evaluating understanding

The obsession with whether a person used AI often replaces a more useful question: does the project embody real understanding? A person may use AI as a helper, autocomplete engine, draft tool, or search substitute. That does not automatically invalidate the work. The issue is not mystical purity. The issue is whether the project's substance holds up under technical examination.

5. Technical rules for evaluating whether a project is likely AI-generated

If you want to evaluate a project seriously, use technical rules, not emotional fog. None of these rules are perfect individually. The point is to build a disciplined picture from multiple observations.

Rule 1: Check for implementation depth, not just implementation presence

Many weak AI-generated projects contain a lot of code, but very little depth. Depth means that the code reveals constraints, local reasoning, ugly edge-case handling, and problem-specific decisions. Presence only means that files exist and compile-looking text fills them.

  • Do components have meaningful relationships, or are they merely adjacent?
  • Are awkward edge cases handled in ways that imply real contact with failure?
  • Do abstractions solve specific recurring problems, or do they exist because abstractions sound advanced?
  • Does the code reveal iteration, not just generation?

AI often produces plausible structure. It is much worse at producing grounded depth across an entire project unless a human has heavily shaped it.

Rule 2: Look for consistency of mental model

Real projects usually exhibit a governing mental model, even when they are messy. There is often a recognizable way the author thinks about boundaries, error handling, naming, state management, layering, performance, or interfaces.

AI-heavy projects often drift between styles and assumptions because generated parts are locally plausible but globally weak. One module may suggest a careful low-level mindset, another may use high-level shortcuts that contradict earlier design decisions, and another may include patterns that solve a different problem entirely.

Ask: does this project feel like one evolving mind, several collaborating minds with coherent agreements, or an assembly of locally convincing fragments?

Rule 3: Inspect the failure surfaces

A project becomes most revealing where things go wrong. Mature work usually contains evidence that the author has fought specific failures. Error messages, fallback logic, debug scaffolding, cautious checks, comments explaining weird behavior, and defensive handling around hard boundaries are all useful signals.

AI can write generic error handling. What it rarely does well without guidance is encode the weirdness of lived debugging. Real failure surfaces are oddly specific.

Rule 4: Check whether documentation matches the code at a deep level

One of the strongest tests is cross-consistency. Does the documentation describe the same system the code actually implements? Not just the same buzzwords. The same real behavior.

  • Do docs mention tradeoffs that are visible in the code?
  • Do claimed features appear in implementation, tests, examples, or issue history?
  • Do architecture descriptions reflect actual control flow and module boundaries?
  • Do limitations appear honestly, or does the project sound complete where the code is obviously partial?

Fake or shallow projects often have "description surplus." The explanation is broader and cleaner than the code reality beneath it.

Rule 5: Examine history, not just snapshot

A single repository snapshot can mislead you. Real evaluation should inspect evolution where possible. Commit patterns, issue discussions, refactors, regressions, renamed modules, broken ideas, recovery attempts, and deleted experiments all reveal whether a project grew through real contact with problems.

This does not mean commit history is infallible. People can squash, rewrite, or curate history. But a living project often leaves behind traces of struggle. AI-only generation tends to produce a cleaner snapshot than the real process required to reach it.

Rule 6: Test for explainability under pressure

One of the strongest ways to evaluate a project is to ask whether its author can explain design choices at multiple levels:

  • high-level goals and architecture
  • mid-level component interactions
  • low-level implementation details and tradeoffs
  • known weaknesses, ugly compromises, and unfinished areas

Real builders can usually explain why something is the way it is, even if the answer is "because the other approach kept exploding." That kind of answer is worth more than sleek prose. Human understanding sounds grounded, constrained, and occasionally irritated by reality. Because reality is annoying and software is a long argument with it.

Rule 7: Distinguish AI-assisted work from AI-substituted work

This is one of the most important distinctions and one of the most ignored. There is a major difference between:

  • a person using AI to accelerate pieces of a project they understand, and
  • a person using AI to impersonate understanding they do not possess.

The first case may still yield solid work. The second usually collapses under scrutiny. Your evaluation should target whether the project depends on real understanding, not whether the author maintained mythical purity from all machine assistance. Software has always involved tools, templates, references, examples, stack traces, autocomplete, copied patterns, and outside help. The ethical and technical question is where understanding actually lives.

6. Strong signals, weak signals, and useless signals

People often fail because they do not rank evidence properly. They mix strong indicators with weak aesthetic impressions and then act equally confident about both. That is how nonsense gets mistaken for discernment.

Stronger signals

  • major architectural contradictions across core subsystems
  • documentation that claims behavior the code clearly does not support
  • tests, examples, or interfaces that fail in ways suggesting no one actually ran them
  • large volumes of code with shallow local correctness but no global coherence
  • repeated use of generic patterns where domain-specific handling is necessary
  • inability of the author to explain implementation details beyond surface summaries
  • code that appears broad but shows almost no traces of debugging, iteration, or failure-driven refinement

Weaker signals

  • overly polished README language
  • verbose comments
  • generic naming in some modules
  • ambitious project scope
  • similarity to common tutorial or boilerplate structure
  • aesthetic resemblance to other suspicious repositories

These may justify closer examination. They do not justify a conclusion on their own.

Mostly useless signals

  • "the vibe feels AI" with no technical explanation
  • "it sounds too confident"
  • "it looks too clean"
  • "it looks too messy"
  • "I saw a similar fake project once"
  • "the person writes in a way I personally distrust"

These are not analysis. They are moods dressed as conclusions.

7. The proper mindset for identifying AI in any project

The right mindset is not gullibility and not paranoia. It is disciplined skepticism. That means being willing to suspect, willing to investigate, and unwilling to convict on vibes.

First principle: suspicion is a starting point, not a conclusion

If something feels off, that is fine. Feeling that a project may be artificial, deceptive, inflated, or shallow can be useful. But feelings are triage tools, not verdict engines. Their job is to tell you where to look, not what to believe.

Second principle: look for interaction between signals

A serious judgment should come from multiple signals reinforcing one another. Weak documentation plus weak code-to-doc alignment plus shallow implementation depth plus contradictory design assumptions plus inability to explain tradeoffs form a meaningful pattern. One of those alone does not.

Third principle: reward concrete evidence over cultural familiarity

People often confuse "this resembles content I distrust" with "this has properties I can demonstrate." The first is social memory. The second is analysis. When in doubt, pick the second. It takes longer. Tragically, that is how thought works.

Fourth principle: separate authorship concerns from quality concerns

A project can be legitimate and bad. It can be AI-assisted and still substantive. It can be fully human and still incoherent. It can be deceptive and technically impressive in places. Do not mash all categories into one emotional blob. Ask separate questions:

  • Is the project technically sound?
  • Is the project honestly represented?
  • How much real understanding appears to be behind it?
  • Is AI being used as a tool, or as a substitute for comprehension?

Fifth principle: preserve the right to say "unclear"

One of the healthiest habits in evaluation is tolerating uncertainty. Not every project can be cleanly classified from the outside. Sometimes the right answer is that the evidence is mixed. That is not weakness. That is intellectual hygiene.

8. A practical investigation process that actually works

If you want a real method rather than public-performance skepticism, use a structured process.

  1. Start with the project's claims.

    Write down what the project says it does. Not what people around it say. Not what the aesthetic suggests. The actual stated claims.

  2. Map claims to evidence.

    For each significant claim, look for code, tests, examples, screenshots, benchmarks, issue discussions, or docs that support it. Claims with no implementation trail deserve caution.

  3. Inspect one level deeper than presentation.

    Do not stop at the README or the top-level file tree. Follow real control flow. Read internal modules. Find where the difficult parts should live and see whether they actually do.

  4. Look for coherence across files.

    Does the project feel architected, evolved, patched, or pasted? Real projects can be messy, but they usually preserve some logic about why things are where they are.

  5. Check for failure knowledge.

    Mature work often reveals scars. Comments about weird hardware behavior. Temporary workarounds. Known limitations. Debug hooks. Performance compromises. Strange boundary checks. These are often more revealing than polished overviews.

  6. Evaluate whether the complexity is earned.

    Is the complexity responding to real constraints, or is it decorative complexity? AI often generates structures that look "advanced" without the pressure that would justify them.

  7. Ask whether the project could survive targeted questions.

    If the author had to explain why module A interacts with module B this way, why this fallback exists, why this abstraction boundary was chosen, and what the ugliest known bug is, would there likely be real answers?

  8. End with a graded conclusion, not a dramatic one.

    For example: likely genuine, likely heavily AI-assisted but substantive, likely shallow and overstated, likely deceptive, or inconclusive. Binary thinking is neat and stupid. Reality prefers categories with edges.

A compact evaluation checklist

  • Are the project's claims specific enough to test?
  • Does the code reflect real contact with constraints and edge cases?
  • Do docs and implementation match at the behavioral level?
  • Is there a coherent mental model across the codebase?
  • Do the rough parts look like lived engineering or generated filler?
  • Can the work be explained beyond buzzwords?
  • Am I judging from evidence, or from resemblance to things I dislike?

9. Final principle: critique the work, not the vibe around it

The rise of low-effort AI output has created a real need for stronger criticism, sharper standards, and more technical skepticism. But that need does not justify sloppy accusation. In fact, it makes disciplined evaluation even more important.

Once "AI slop" becomes a reflex rather than a conclusion, criticism decays into branding. People stop asking whether a project is coherent, tested, grounded, explainable, and honestly represented. They start asking whether it resembles a category they have already decided to despise.

That is a failure of judgment. It is also a failure of respect toward the craft itself. Software should be evaluated through implementation, constraints, tradeoffs, internal consistency, failure handling, historical evolution, and demonstrated understanding. Not through vague contempt sharpened into a label.

The right question is not "does this remind me of AI slop?" The right question is "what, specifically, does this work reveal about the understanding behind it?"

That question is slower. Less glamorous. Harder to perform in public. It is also the only one that deserves authority.

If a project is fake, shallow, inflated, or mechanically assembled, technical scrutiny will usually expose it. If a project is legitimate, unusual, rough, ambitious, or badly marketed, technical scrutiny gives it its fairest chance. That is the standard worth defending.

Not blind trust. Not reflex suspicion. Just the old and irritating discipline of looking at the thing in front of you and judging it by what is actually there.