Meta Quietly Drops Clause That Opened Door to News Site Scraping

A Swift Reversal Under Pressure

Meta has removed language from its commercial terms page that legal experts interpreted as granting the company permission to scrape content from external websites for artificial intelligence training. The clause, which disappeared following industry criticism, had suggested Meta could harvest material from any domain to "provide, develop, improve, research, train, and maintain the integrity of" its suite of products and services.

The deletion marks a rare instance of Meta backtracking on terms-of-service language before it could be tested in practice. At Opentechwire, we have tracked a rising pattern across the AI industry of platforms attempting to expand training data rights through updated user agreements, but few have provoked such immediate pushback from the publishing sector.

What the Deleted Language Permitted

The removed clause stated that Meta could "use any content from such domains, web pages, or URLs" covered under its commercial terms. Legal scholars, including James Grimmelmann, a professor specialising in digital and information law at Cornell University, flagged the provision as unusually broad. The wording did not distinguish between content users voluntarily shared on Meta platforms and material Meta might access by crawling third-party sites.

For news organisations already navigating tensions over AI companies using their archives without compensation, the clause raised immediate alarm. Unlike agreements that govern user-uploaded posts on Facebook or Instagram, this language appeared to extend Meta's data collection reach beyond its own ecosystem. The Nieman Journalism Lab was among the first to highlight the implications for publishers.

The Regional Context for Publishers

Across Asia, where digital advertising revenue has increasingly concentrated in the hands of Meta and Alphabet, news organisations have limited leverage to negotiate data-use terms. Singapore's media regulator has explored transparency requirements for generative AI training sets, while publishers in South Korea and Japan have begun forming coalitions to demand licensing fees from large language model developers.

Meta's initial language would have complicated those efforts. If commercial terms alone could authorise scraping, publishers would face the choice of blocking Meta's crawlers entirely, potentially losing referral traffic and search visibility, or accepting that their content might fuel competing AI summaries. The economics are particularly stark for small and mid-sized newsrooms that depend on platform distribution but lack the resources to litigate terms-of-service disputes.

Why Meta May Have Acted Quickly

The removal suggests Meta's legal team recognised the clause could not withstand scrutiny under emerging copyright frameworks. The European Union's AI Act and proposed revisions to Singapore's Copyright Act both emphasise that scraping for commercial AI purposes requires explicit consent or licensing. In the United States, several publishers have filed suit against AI developers, arguing that wholesale ingestion of articles exceeds fair use.

Meta has invested heavily in its Llama family of open-weights models, which compete with OpenAI and Anthropic in the enterprise market. A protracted legal battle over training data sourcing could hinder adoption, especially among corporate clients sensitive to intellectual property risk. Removing ambiguous language before it generates litigation is cheaper than defending it in multiple jurisdictions.

What Remains Unclear

Meta has not issued a formal statement explaining the change or clarifying what data practices it considers permissible. The company still operates one of the world's most active web crawlers, and its privacy policy reserves broad rights to collect information "from partners and third parties." Whether those partners include content delivery networks or analytics providers that handle news site data is not publicly documented.

Publishers also lack visibility into whether Meta has already used scraped news content in past training runs. Llama 3, released earlier this year, was trained on datasets Meta described only as "publicly available online sources." That description could encompass anything from open government databases to paywalled journalism accessed through institutional subscriptions. Without model transparency requirements, proving unauthorised use remains difficult.

The Broader Industry Pattern

Meta's reversal arrives as the AI sector faces mounting pressure to disclose training data provenance. Stability AI recently settled a class-action lawsuit with photographers after admitting its image models were trained on copyrighted works without permission. OpenAI has signed licensing deals with the Associated Press, Axel Springer, and the Financial Times, setting a precedent that training on news requires compensation.

At the same time, several AI labs have argued that existing copyright law does not prohibit learning from publicly accessible content, even if that content is protected by copyright. The legal question turns on whether ingesting millions of articles to derive statistical patterns constitutes "copying" in the sense the law recognises. Courts in the United States and the European Union are expected to issue rulings on this question within the next eighteen months, and those decisions will shape whether clauses like Meta's deleted language can ever be enforceable.

What Publishers Should Monitor

News organisations should audit their own commercial relationships with Meta, particularly any agreements related to Facebook News partnerships, Instant Articles, or advertising integrations. Some of those contracts may contain data-sharing clauses that were drafted before generative AI became a commercial priority and may now carry unintended implications.

Technical measures also matter. Publishers can use robots.txt files to block Meta's crawlers, though doing so may reduce content distribution. More sophisticated approaches include differential serving, which delivers different content to bots than to human readers, or embedding invisible watermarks that can later prove unauthorised reproduction. These tactics require engineering resources, which not every newsroom possesses.

Industry coalitions, such as the News Media Alliance in the United States and the Asia News Network, are exploring collective licensing models that would allow AI companies to access publisher archives in exchange for revenue shares. Whether Meta will participate in such arrangements remains to be seen, but the removal of the scraping clause suggests the company is at least willing to negotiate rather than assert unilateral rights.

A Temporary Retreat or a Lasting Shift

The deleted language may reappear in a different form. Meta could, for instance, introduce opt-in programmes that offer publishers traffic or monetisation benefits in exchange for training data access. It could also pursue partnerships with aggregators or data brokers who claim to have secured the necessary rights. The fundamental tension, between AI developers' appetite for large, diverse datasets and publishers' need to control and monetise their work, has not been resolved.

For now, the reversal signals that public scrutiny and the threat of legal action can still shape platform behaviour. As jurisdictions across Asia and Europe tighten rules around AI training data, companies that attempt to write themselves broad permissions through terms of service are likely to face similar resistance. The question is whether that resistance will translate into enforceable rights and sustainable revenue, or remain a series of tactical retreats with no durable framework behind them.