2 min read
Add as a preferred source on Google

Google Targets 2027 Support as Microsoft Sets Feb. 1 Start for AI Signal

The proposed robots.txt token separates AI training preferences from crawler access; Microsoft’s page-level controls for grounding will not change Bing search eligibility.

A blank page pauses at the threshold of a narrow metal gate / TokenPost.ai
A blank page pauses at the threshold of a narrow metal gate / TokenPost.ai

Google and Microsoft outlined plans Friday, Oct. 30, to recognize a shared robots.txt signal for AI training, expanding a proposal that gives publishers a common way to express how participating developers may use their content.

Google is aiming to support the signal in 2027. Microsoft will begin applying its preferences as it crawls pages on Feb. 1, 2027, adding a dated start to the proposal’s rollout plans.

The proposed __usage_ai-training token records a publisher’s preference about using website content to train generative AI models. It addresses training use, not whether a crawler can request a page. Website operators would continue to use separate crawler rules to limit access. A sitewide training opt-out example in the draft pairs the token with Disallow: /.

Google plans to treat the shared preference as a Google-Extended disallow when a site opts out and does not separately specify Google-Extended. If a site does not include a policy-token group, the draft treats that as no expressed preference.

Microsoft is also adding page-level controls for AI training and “grounding,” which uses content to inform AI-generated responses. Microsoft’s preferences will not affect a page’s eligibility for traditional Bing search results. Its stated default when no preference is set is “Allow.”

OpenAI has also made a commitment to honor the shared signal, with implementation planned in the coming months. Proposal author Kevin Kelley, who is affiliated with Anthropic, wrote, “Anthropic has already started implementing support for this draft.” Kelley said extending support across Anthropic’s services will take time.

The proposal remains an individual Internet-Draft with no formal standing in the Internet Engineering Task Force’s standards process. Microsoft’s Feb. 1, 2027, start is the rollout’s next specified date; Google has given a 2027 target without a more specific day.

Simon Yoon

Reporter

Simon Yoon reports on blockchain technology for TokenPost. Send corrections or tips to info@tokenpost.com.

Loading…