The CEO of the largest digital and print publisher in the U.S. has accused Google of being a bad actor for crawling its websites to support the search giant’s AI products.
Neil Vogel, CEO of People, Inc. (formerly Dotdash Meredith), a publisher that operates over 40 brands, including People, Food & Wine, Travel + Leisure, Better Homes & Gardens, Real Simple, Southern Living, Allrecipes, and others, said that Google is not playing fair because it uses the same bot to crawl websites to index them for the Google search engine as it does to support its AI features.
“Google has one crawler, which means they use the same crawler for their search, where they still send us traffic, as they do for their AI products, where they steal our content,” said Vogel, speaking at the Fortune Brainstorm Tech conference this week.
He noted that three years ago, Google Search accounted for about 65% of the company’s traffic and that has since dropped to the “high 20s.” (Vogel shared an even more startling statistic with AdExchanger last month, saying that as of several years ago, Google’s traffic accounted for as much as 90% of People Inc.’s traffic from the open web.)
“I’m not complaining. We’ve grown our audience. We’ve grown our revenue,” Vogel told conference attendees. “We’re doing great. What is not right about this is: You cannot take our content to compete with us.”
Vogel believes publishers need more leverage in the AI era, which is why he feels it’s necessary to block AI crawlers — automated programs that scan websites to train AI systems — as that can force them into content deals. His company, for example, has a deal with OpenAI, which Vogel described as a “good actor.”
People Inc. has been leveraging web infrastructure company Cloudflare’s latest solution to block AI crawlers that don’t pay, prompting AI players to approach the publisher with potential content deals. While Vogel wouldn’t directly name the companies involved, he said they were “large LLM providers.” No deals have been signed yet, but Vogel said the company is “much further along” than before it adopted the crawler-blocking solution.
Techcrunch event
San Francisco
|
October 27-29, 2025
REGISTER NOW
However, Vogel pointed out, Google’s crawler can’t be blocked because doing so would also prevent the publisher’s websites from being indexed in Google Search, cutting off that “20%-ish” of traffic that Google still delivers.
“They know this, and they’re not splitting their crawler. So they are an intentional bad actor here,” Vogel declared.
Janice Min, the editor-in-chief and CEO at newsletter provider Ankler Media, agreed, calling Big Tech companies like Google and Meta longtime “content kleptomaniacs.”
“I don’t see the benefit to us in partnering with any AI company right now,” she said, adding that her company blocks AI crawlers.
Meanwhile, Cloudflare CEO Matthew Prince, whose company makes the AI-blocking solution (and who was also on the panel), said he believed that things would still change in the future when it comes to how the AI companies behave. He suspected those changes could be prompted by new regulations.
The Cloudflare exec also questioned whether fighting the AI companies using legal solutions around things like copyright law, created for the pre-AI era, was the right answer.
“I think that it’s a fool’s errand to go down that path, because, in copyright law, typically, the more derivative something is, the more it’s protected under fair use … What these AI companies are doing is they’re actually creating derivatives,” Prince said. “And so if you look at the best case law that’s come out so far, it’s actually said that the use by Anthropic and others — the reason Anthropic settled recently with all the book publishers for $1.5 billion — was for them to be able to preserve the positive copyright ruling that they got.”
Prince also proclaimed that “everything that’s wrong with the world today is, at some level, Google’s fault,” because the search giant had taught publishers to value traffic over original content creation, triggering publishers like BuzzFeed to write for clicks. Still, he admitted that Google was in a tough spot right now from a competitive standpoint.
“Internally, they’re having massive fights about what they do, and my prediction is that, by this time next year, Google will be paying content creators for crawling their content and taking it and putting it in AI models,” he said.
Could you share the article you’d like me to rewrite?