the image linked to by the post is the Pangram LLM-detection report of paragraphs in the first screenshot below, reading “100% AI”. in lieu of alt text for the screenshots below, I suggest scrolling through @eicker@lemmy.world’s profile and https://lemmy.world/comment/26323310, which the screenshots are of. Look at the vote counts.
FYI: Pangram sucks. It’s about as accurate as throwing darts these days.
A long, long time ago (i.e. 2024) there was only one LLM that was widely used by the populace: ChatGPT. It only had a few versions and they didn’t differ much in how they worded output.
Now it’s 2026 and we have hundreds of thousands of AI models. That includes over 1000 commercial vendor models!
Another thing that’s really important that you need to understand is that even if two providers run the same model, their configurations will be different and thus, their output won’t be the same.
To summarize: There’s no fucking way Pangram can work properly anymore. It’s useless garbage.
Yeah, I don’t think that there’s going to be a viable future in distinguishing between LLM and human text either, certainly not for people who have an interest in defeating it. That being said:
-
Some people are just going to use the most-common models.
-
Some models are derived from other models and will share statistical similarities.
I still think that trying to detect LLMs is largely going to be a dead-end road, though.
There are also other wrinkles. I’d bet that an increasing amount of not-trying-to-maliciously-impersonate-a-human text out there is going to come from LLMs. Like, okay, Google Translate uses an LLM. I remember listening to an interview with a senior British EU translator (this was back before Brexit). This is going to be a quality human translator. He was saying that machine translation had gotten so good that these days, that now he always did his first pass with Google Translate and just cleaned it up by hand. Okay, so you figure that this is probably widespread, probably even more so now, right? So if most of the EU’s documents are coming from an LLM, even if you could hypothetically detect all LLMs, you probably wouldn’t want to trigger on some of them.
Not to mention that people adopt mannerisms from text that they read or hear spoken. If someone is using, say, automated browser translations, they’re seeing a lot of the world through an LLM, and you’d expect them to pick up characteristics of that.
-
The third screenshot is me asking if it was written by AI and not receiving a reply. The user pretty much does not reply to people, just obviously-LLM summaries to his own posts. Almost all such comments are the same amount of lines. I used the first result for an LLM detector just to give a number; Eicker’s language always ticks the “this is LLM” neurons to me.
and for that matter, looking into it, Pangram seems to have an extremely low false-positive rate these days: https://www.theatlantic.com/technology/2026/05/pangram-ai-detection-accuracy/687381/ . the only complaints seem to be about its false negative rate. the FP rate of 1 in 10,000 is maybe higher than you want but whatever of Eicker’s text you throw at it it always comes up positive.
Now it’s 2026 and we have hundreds of thousands of AI models.
And yet they still all write the same. And the vast majority are only using of a handful of models anyway.
- They do not all write the same. There’s benchmarks for this and if you just do a basic search or actually test them yourself you’ll see that their output differs wildly. Not only between vendors, but been versions of the same model. For example, Claude stopped using em dashes for most output for a while but now they’ve come back.
- There’s vastly more than “a handful of models”! Are you living under a rock‽ Go to HuggingFace today and then again tomorrow. There’s going to be a dozen new ones! Probably more!
- Changing the back end configuration even a tiny bit can vastly change the output of a model.
They do not all write the same
They all have the same hallmarks.
There’s vastly more than “a handful of models”! Are you living under a rock‽
You’re intentionally misrepresenting my statement. It wasn’t about the models that exist, it was about the models people are using.
I’m not misrepresenting anything. If you take any model and just change the temperature a little bit (it’s a setting that controls the influence of the RNG) you can have wildly different output.
Different AI providers using the exact same model will have outputs so different using the same exact prompts you’d think they were in different universes.
They absolutely do not “have the same hallmarks.”
This stuff is benchmarked to absurd degrees every time a new model comes out. Just look it up already! You’re so obviously arguing from an ignorant position it’s silly.
I’m not misrepresenting anything
You absolutely did. So much so that you didn’t even bother to explain yourself.
I don’t need to look up anything. It’s been shoved in my face for the last 5 years. You are so obviously arguing about nuances that don’t matter in reality and it’s ignorant and silly! Things like em-dashes, bullet points, and gargling the user’s balls are all signs that everyone who’s paying attention can quickly and easily recognize at this point.
Piefed has some LLM detection tools for moderators as well as bot detection for admins.
It’s also worth pointing out that some users on here use translation tools which may trigger some of these “AI Detectors”.
Yep, that account was banned from piefed.social 3 months ago.
Probably because they generated something you disagree with and felt the need to delete the account.
You’re on one of the only actively pro-AI instances…
And you came here to complain an account seems to be using a chat bot?
lemmy.world is pro AI? Huh?
OP is on dbzer0, but they posted to a Lemmy.world community.
Oh, I misunderstood. I wasn’t aware dbzer0 was pro AI either, but I don’t really know what happens on there.
Holy GPT batman, yeah that’s clearly generated.
The therapist is a database, and the waiting room is a data center.
That’s the unsettling part.
Democracy becomes a lot less inconvenient when you can X, Y, and Z.
Researchers needed one query.
X, Y, and Z on the same day. Somewhere, a sysadmin is discovering that Y is actually Z. Meanwhile, [flowery adjective soup storytime].
Next up:
You can tell there’s no real thought in those word choices, too. You wouldn’t prepare your origin story, you would experience it. The data center would less aptly be described as a therapist’s waiting room and more as their office. It also reads like it’s trying to write long-form prose entirely out of headlines-- every single word is designed to suck as much attention out of me as possible. It makes me feel disgusted, like seeing an ad.
alexa, quote that post where it says frankenstein was 90% ai written and the a user saying they hope the writer addresses this grave accusation
eicker links several websites in his bio, such as this one:
it scares me that 90% of Fediverse users do not recognize his obviously LLM writing.
Yeah I blocked that guy since one of his first posts. Everyone here going “oh no, you can’t trust pangram”… Irrelevant. If you can’t even spot the sloppiest slop that is his website collection and his comments (aside from some short replies to others), you’re the problem.
I couldn’t tell it’s AI because it’s such a basic take I stopped reading after the first sentence.
also, can we talk about how Eicker’s website literally advertizes LLM social media marketing services? see third screenshot in body
the “LLM detectors” are just modern era coffee-grains-prediction
the field’s state-of-the-art-s have gotten a lot better at eliminating false positives over the past year: https://www.theatlantic.com/technology/2026/05/pangram-ai-detection-accuracy/687381/
i just block botspammers as soon as I see em, usually reporting but also usually without posting my suspicion of botspam. if my behavior is common, then many people who notice it just block and move on, too. others will refuse to see it when pointed out, insistently pointing out an individual phrase or line that seems unlikely for an ai to produce, which… well.
at least for my part, the vigilance is not lacking, its just that my vigilance doesn’t federate. i suppose i’ll be more careful about making sure to leave suspicion comments instead of just reports.
i think this might be some human using llm translate though
He uses far too many emojis, AI loves using non standard characters you can’t easily find on a keyboard, nor would they survive a translation tool.
Other indicators:
- only talks about AI related news, doesn’t have any other hobbies.
- all comments are single paragraphs, no line breaks, no complicated thoughts that require building up with a thesis or supporting evidence.
- unusually polite and supportive, it’s not full on glazing but the threadiverse is culturally pretty cold and unwelcoming and he does not fit.
- posts multiple times within the span of minutes.
only talks about AI related news, doesn’t have any other hobbies.
False. They talk about Xbox pricing in your own screenshots.
your
there’s a pretty big blue indicator to tell you which comments’ authors are OP and which ones are not
unusually polite and supportive, it’s not full on glazing but the threadiverse is culturally pretty cold and unwelcoming and he does not fit.
lmfao true
Anytime I see a gigantic paragraph of bullshit I know it’s AI.
I know “gigantic” is relative, but the user in question always had a paragraph of roughly 3-4 sentences, each of which only had up to 1 dependent/independent clause. So the paragraphs were actually short, we’re not talking walls o’ text. But that also means the user was never communicating complex ideas or nuance. If for whatever reason I didn’t want to outright label them an AI bot, I would still feel comfortable labeling them a low-effort quipper or engagement baiter.
New Villain Unlocked: Impersonators
