A recent analysis of Substack newsletters shows most leading writers still primarily produce human-authored content, despite increasing visibility of AI-generated posts, highlighting cautious adoption in information-heavy niches amid ongoing debates over synthetic writing.
An analysis of Substack newsletters suggests that most of the platform’s biggest writers are still publishing human-made work, even as AI-generated posts become more visible in some of its most analytical categories. The finding comes after a viral March 2025 post on Substack purporting to show a debate between Elon Musk and Keanu Reeves drew tens of thousands of likes and reposts despite never taking place at all; the piece was generated entirely by AI. The episode has become emblematic of a wider unease about synthetic writing spreading through online publishing.
To gauge the scale of the problem, the writer behind the analysis used Pangram, an AI-detection tool that says it can identify machine-written text even after attempts to disguise it, and examined the 10 most recent posts from the top 25 Substack Bestsellers in each category. Pangram says its detector has been independently verified and is available through a browser extension as well as an API. The resulting picture was more mixed than alarmist: roughly two-thirds of the 575 newsletters reviewed showed no detectable AI content in their recent archives, according to the analysis.
Where AI writing did appear, it was concentrated in categories built around explanation, analysis and argument. Technology showed the highest proportion, with 28 per cent of writing in top newsletters judged to be fully or partially AI-generated. Philosophy and health followed, at 23 per cent and 22 per cent respectively, while culture stood at 13 per cent. By contrast, sports, food and drink, and music registered far lower levels. The pattern suggests readers may be more accepting of machine assistance in information-heavy niches than in areas that depend more heavily on voice, taste and personal judgement.
The analysis also found that AI use tends to be all or nothing. Rather than lightly editing a few drafts, some newsletters appear to be running almost entirely on automation, with several publishing posts that were assessed as 100 per cent AI-generated. A small number of big operators distorted the overall figures: just 29 publications accounted for half of all majority-AI posts, while one AI-heavy News newsletter made up 35 per cent of the category’s majority-AI content. Similar outliers in culture and world politics had an outsized effect on the averages.
There are limits to the data. Some newsletters place key material behind paywalls, so only free preview text could be scanned, and Pangram requires at least 50 words before it will make a call, with a bias towards “Human” when uncertain. Even so, the findings echo a separate Wired analysis that estimated about 10 per cent of Substack’s top newsletters publish AI-generated or AI-assisted material. Taken together, the evidence points to a platform that is still broadly human-led, but increasingly shaped at the margins by high-volume AI publishing.
Source Reference Map
Inspired by headline at: [1]
Sources by paragraph:
Source: Noah Wire Services
Noah Fact Check Pro
The draft above was created using the information available at the time the story first
emerged. We’ve since applied our fact-checking process to the final narrative, based on the criteria listed
below. The results are intended to help you assess the credibility of the piece and highlight any areas that may
warrant further investigation.
Freshness check
Score:
7
Notes:
The article references a viral AI-generated post from March 2025, indicating the content is over a year old. The analysis uses Pangram, an AI detection tool, which is a recent development. However, the article does not provide a specific publication date, making it challenging to assess the freshness of the content. The lack of a clear publication date raises concerns about the timeliness of the information.
Quotes check
Score:
6
Notes:
The article mentions a viral post from March 2025 but does not provide direct quotes or specific excerpts from that post. The analysis references Pangram’s AI detection tool but does not include direct quotes from Pangram’s findings. The absence of direct quotes makes it difficult to verify the accuracy and context of the information presented.
Source reliability
Score:
5
Notes:
The article cites Pangram, an AI detection tool, and a Wired analysis. Pangram is a relatively new tool, and its claims about accuracy are not independently verified. The Wired analysis is not provided in full, making it hard to assess its credibility. The lack of access to the full Wired article and independent verification of Pangram’s claims raises concerns about the reliability of the sources.
Plausibility check
Score:
7
Notes:
The article discusses the prevalence of AI-generated content in Substack newsletters, which aligns with known trends in digital publishing. However, the lack of specific data and direct quotes makes it difficult to fully assess the plausibility of the claims. The absence of concrete evidence and specific examples raises questions about the accuracy of the information presented.
Overall assessment
Verdict (FAIL, OPEN, PASS): FAIL
Confidence (LOW, MEDIUM, HIGH): MEDIUM
Summary:
The article discusses the prevalence of AI-generated content in Substack newsletters, referencing a viral post from March 2025 and an analysis using Pangram’s AI detection tool. However, the lack of a clear publication date, direct quotes, specific data, and access to the full Wired article raises significant concerns about the freshness, originality, and reliability of the information presented. The absence of independent verification and concrete evidence further diminishes the credibility of the content.
