Leading web‑scraping companies say their bots only collect publicly available information, despite lawsuits from major platforms. Executives from Bright Data, ScrapingBee and Oxylabs stress compliance with open‑web principles and note legitimate uses such as cybersecurity and investigative journalism. The growing demand for AI‑trained data has spurred a new market, with over 40 firms offering bots for AI training and a nascent marketing approach called generative engine optimization. Industry leaders predict this trend will intensify through 2026, creating both opportunities and challenges for publishers and regulators. Leer más →

New data shows AI bots are rapidly increasing their share of internet traffic, often bypassing standard safeguards like robots.txt. Publishers and website owners are confronting a sophisticated arms race as bots disguise themselves as human browsers and employ advanced scraping techniques. Companies such as TollBit, Cloudflare and others are offering tools to detect, block, or monetize bot access, while a growing market promotes services that help content appear in AI-driven search results. The shift is reshaping how the web functions and creating new revenue streams for digital publishers. Leer más →

Google has filed a lawsuit against SerpApi, alleging that the web‑scraping service bypassed the company's SearchGuard technology to harvest search results and copyrighted partner content. Google says SearchGuard, introduced in January 2025, was designed to block automated access, but SerpApi employed fake browsers and a network of IP addresses to make its queries appear human. The lawsuit claims SerpApi’s actions violate federal law and constitute misappropriation of protected content, highlighting the growing tension between tech firms protecting their data and third‑party services that scrape it. Leer más →

Japanese media giants Nikkei and Asahi Shimbun have filed a joint lawsuit against AI search company Perplexity, alleging that the firm copied and stored articles from their servers and presented inaccurate information as if it were factual. The suit seeks damages and the removal of stored content. The case arrives as Perplexity recently launched a publisher revenue‑sharing subscription, while earlier reports have accused the company of web‑scraping practices that bypass robots.txt and firewall protections. The lawsuit highlights ongoing tensions between AI developers and traditional news outlets over content use. Leer más →

AI startup Perplexity is accused of scraping content from websites that have explicitly blocked AI scraping, according to Cloudflare. Perplexity allegedly ignores blocks and hides its activities, prompting Cloudflare to delist its bots and add new blocking techniques. Leer más →