Is Substack's New AI-Detection Tool Good or Bad for Writers?
How to use Pangram, or not, and my advice for writers on this whole AI madness
Fellow writers: There’s some scary shit happening out there in the world of writing, which warrants this special extra edition of the Writer’s Guide (regularly publishing on Tuesdays).
In case you haven’t heard, Substack announced July 21 the launch of an AI-detection tool called Pangram to scan posts, notes and comments with 100+ words from July 21 onward for signs of AI-generation. For now, readers (and writers) decide whether to use it (Substack’s how-to).
Wherever you publish your words, you can expect similar attempts to deal with artificial intelligence’s threat to human-generated writing, so I see this development as a harbinger we all need to pay attention to, much as we might wish to bury our heads in the proverbial sand.
We can’t yet know how Pangram affects the platform, and how it affects human writers. But given the challenges of AI-detection, and the expected ongoing improvement of AI to replicate human writing, we can expect a lot of controversy and headache ahead. Let me get put my stake in the ground right here. As I’ve written before:
Among writers, you got your AI writers, and those of us who love writing.
Below I’ll spell out what you can do (including a new Substack feature that lets you tell readers your stance on AI).
But first…
Substack’s stance
Much as I wish AI did not exist, I’m not surprised by Substack’s reasoning for launching an AI-detection tool, nor their position on AI-generated writing.
“We’re not against people using AI to assist their work, and we think people should be free to choose which tools they use to express themselves,” Chris Best, Co-founder & CEO of Substack, wrote in a blog post. “But people should know what they’re getting. For example, you might be fine reading someone’s sharp analysis in which they leaned heavily on AI to present their findings but only want to spend time engaging with replies written by a human hand. Either way, you should have a choice in the matter.”
Best continues: “The core problem is not people using AI, or the quality of its output. Not everything made with AI is slop, and not all slop is made with AI. The problem is when there is a mismatch between a reader’s expectation and reality, especially when they unwittingly invest their attention in something with no human thought on the other end. That’s Claudefishing.”
Now there’s a term destined for the dustbin of AI-generated language. Good riddance.
So, does Pangram work?
A small early test of the tool by CNET writer Alex Valdes found it to be pretty accurate at detecting AI and acknowledging his human-generated writing.
I ran my own post—the one you’re reading now—through Pangram and got an accurate result:
It’ll take time to see if the accuracy holds, of course.
As AI gets better at generating human-like writing, human writing will look more like AI-generated writing. Where that ends up is anyone’s guess, but it’s no stretch to imagine AI-detection tools becoming thoroughly confused eventually (and at AI-speed, that could mean months or weeks, not years).
(NOTE: See the update on this at the bottom)
What you can do
For the record, I do not use AI to assist the writing of the Writer’s Guide or any of my work, so if Pangram says I do, it’s flat out wrong. I put this logo on the newsletter’s About page two years ago, and in other places where I write, just to make my position clear. (You are free to use the logo—I created it and released it into the public domain).
On that note, Substack also introduced a “how I make this” setting, so you can share your stance with readers. Go to Settings > Details > “How I make this” statement and click Edit.
Here’s what my “How I make this” statement now reads: This is a NO-AI publication: All posts are written by a real human (me).
You can also choose whether AI-detection is employable in each article or note you post. On the Publish page, scroll to the bottom, where you can enable or disable Pangram.
If you disable it, readers will see a message that reads “AI detection unavailable.” It’s too soon to offer advice on whether this will be a good or a bad thing. I’m leaving it available for now, in part because I want to check how Pangram treats me.
My overarching advice: Keep writing in your own voice and style, AI be damned. We human writers must not cave to either the use of AI in our writing nor the pressure to change our writing based on the false positives of AI detection.
Stand firm, folks, and write on.
Rob
PS: An aside: Also in Substack Settings, under privacy, you can choose to “Tell AI tools not to train their models on your content.” Or you can allow it, if that’s your preference.
UPDATE 7/27/26: Tony Stubblebine, CEO of Medium, posted a highly critical analysis of Substack’s use of Pangram. While Medium and Substack are direct competitors, Stubblebine’s criticism stems from his own experience with Pangram, so it’s worth noting. He wrote:
The obvious problem is that AI detectors don’t work. There are already tons of examples demonstrating false positives and false negatives.
These failures in their AI detector affect only the good actors. The bad actors have already run their AI-generated text through “humanizers” and will always be able to defeat AI detectors. So the obvious problem is that the AI detector is an unnecessary annoyance that will smear real writers while doing nothing to slow down slop lords.
END UPDATE
It said mine was 60% AI when it was actually 100% written by me. Guess that's not a compliment from them?
There's another word that should be destined for the bin for low-effort: slop. I mean c'mon people, *any* of these would be a better choice: crap, junk, trash, garbage.