They're blocking Internet Archive?
0 up · 0 down · 0 ratings
Description
23 major news outlets are blocking the Internet Archive's incredibly useful Wayback Machine from crawling their sites. We cannot, in fact, have nice things anymore. USA Today just recently used the Wayback Machine to call out ICE for altering detention stats, and then immediately blocked the archival tool from crawling their own site. Just rude. The news outlets say this is necessary to stop AI crawlers from treating their sites like an all-you-can-scrape buffet. The Guardian's director of business affairs said that the Wayback Machine's API is a real AI risk, so they are only going to be blocking that. Maybe don't attack them on social media. Wayback's director, Mark Graham, responded calling those AI fears unfounded and restating how much effort they put into preventing possible abuses of the crawler. Groups like Fight for the Future and Electronic Frontier Foundation are rallying journalists to publicly back the archive, and over 100 of them have signed a letter thanking the Internet Archive and the Wayback Machine for being an essential and critical tool.
The short summarizes a wave of actions by major news outlets to block Internet Archive's Wayback Machine from crawling their sites, highlighting a tension between open archival access and the rise of AI scrapers. It notes USA Today recently used the Wayback Machine to cite ICE detention statistics and was blocked from crawling its own site, framing the move as a necessary measure to deter automated scraping. The piece references arguments from The Guardian's director of business affairs, who characterizes the Wayback Machine API as a real AI risk and explains that certain access will be blocked, while the Wayback’s director Mark Graham pushes back, describing AI fears as unfounded and detailing safeguards against abuse. The video also mentions advocacy from groups like Fight for the Future and the Electronic Frontier Foundation rallying journalists to support the archive, with over 100 signatories thanking Internet Archive for providing a critical tool. In summary, the clip presents a debate over archival access, corporate pushback against scraping, and broad support from digital rights groups for preserving online history against AI-driven misuse, while underscoring the Wayback Machine’s ongoing efforts to mitigate abuse of their crawler.
Topics · tech_news · digital_archives · media_ethics · internet_policy · ai_risk
Questions answered
- Waarom blokkeren grote nieuwsorganisaties de Wayback Machine en wat staat er op het spel met AI-crawlers?
- Zij noemen AI-gedreven scraping als een risico en willen voorkomen dat crawlers ongecontroleerde toegang krijgen tot hun artikelen, terwijl er tegengeluid is van archieven zoals Internet Archive die benadrukken hoe belangrijk en beveiligbaar hun crawlerbeheer is.
- Welke partijen steunen de strijd voor open archieven en wat is hun inzet voor transparantie?
- Advocacygroepen zoals Fight for the Future en de Electronic Frontier Foundation roepen journalisten op om het archief te steunen en benadrukken het belang van een onafhankelijk historisch naslagwerk tegen manipulatie van online informatie.