How to Set Bing AI Training and AI Grounding Controls

Microsoft Bing now treats AI training and AI grounding as two separate choices, and each one is set in a different place. AI training can be refused for a whole site with two lines in robots.txt. AI grounding, which decides whether a page can be used and cited in Copilot and Bing's AI answers, can only be set page by page through a meta tag or an HTTP response header.

Microsoft announced the change on the Bing Search Blog on October 30, 2026, and Bing starts honoring the new preferences as it crawls from February 1, 2027. Both preferences default to Allow, so a site that publishes nothing is opted in to both uses. Neither setting changes how a page is crawled, indexed or ranked in ordinary Bing search results.

The split matters most for brands that want to appear in AI answers while keeping their content out of model training, because that combination was awkward to express with the old NoArchive and NoCache tags. It also creates a deadline. Sites that relied on those old tags lose their protection after March 1, 2027 unless they publish the new preferences first.

What AI Training and AI Grounding mean in Bing

AI Training covers the use of your content to change the learned parameters of a generative AI model. In plain terms, your pages become part of what the model knows permanently, with no link back to you when it later writes an answer. Microsoft follows the working definition from the IETF AI Preferences group here, which leaves out models used only for non-generative tasks such as spam detection or ranking.

AI Grounding covers the use of your content as input at the moment an answer is written. When someone asks Copilot a question, Bing retrieves relevant pages, the model reads them and the response usually carries citations to those sources. Grounding is the mechanism behind brand visibility in AI search, which is why the two preferences deserve different decisions.

Microsoft is clear that these preferences govern use and have nothing to do with crawl access. Indexing and ranking for traditional results is not counted as grounding, so a page with grounding disallowed can still rank in Bing as before. The reverse also holds. An Allow preference does not override a robots.txt rule that blocks Bingbot, so a blocked page cannot be grounded on regardless of what its tags say.

The three AI Grounding options and what each one covers

Training has two states, allow and disallow. Grounding has three, and the middle one is new. DisallowExtended keeps a page available to the AI products Microsoft operates itself while removing it from the grounding services Microsoft sells to other companies.

The third-party column is larger than it looks. Many AI assistants and enterprise chat tools do not run their own web index and buy web retrieval from providers such as Microsoft. Microsoft does not publish a list of the products built on its grounding services, so a site that chooses DisallowExtended cannot see exactly which assistants it has left.

Microsoft attaches a warning to both restrictive values. Its announcement says that choosing DisallowExtended or Disallow will significantly reduce a site's visibility in AI-generated experiences, while eligibility for Bing search results stays untouched. It also states that no choice, including Allow, guarantees inclusion, traffic or payment.

Which combination fits which business goal

For most brands the sensible setup is to disallow training and leave grounding at its default. Training gives nothing back to the publisher, while grounding is the route by which a brand gets named and linked inside an answer. A company selling products, bookings or services loses reach every time it removes itself from the retrieval step, and its competitors fill the space.

DisallowExtended suits a narrower case. A publisher with a licensing business may not want its articles resold through an API to companies it has no agreement with, yet still wants readers arriving from Copilot. The cost is the unknown set of third-party assistants described above, so it is a choice to make per content type and not as a sitewide habit.

Full Disallow belongs on content whose value depends on not being summarised, such as paid research, subscriber-only articles, course material and gated reports. Applying it to marketing pages, product pages or a public blog works against the reason those pages exist. The practical pattern for a mixed site is training disallowed everywhere, grounding allowed on public sections and grounding disallowed on the premium paths.

How to block AI training in robots.txt and on single pages

The training preference uses a shared token defined in an IETF Internet-Draft, written as a user-agent group in robots.txt. Because the token is shared, the same lines are meant to work for other AI providers as they adopt it, and industry coverage reports that Google aims to support it during 2027. The group below refuses training for an entire site.

User-agent: __usage_ai-training
Disallow: /

Path rules work the same way as ordinary robots.txt rules, so training can be refused for selected sections only.

User-agent: __usage_ai-training
Disallow: /archive/
Disallow: /premium/

The token is a usage preference and not a crawl rule. Adding it does not stop Bingbot from fetching the page, and it should never be replaced with a Bingbot crawl block as a way of controlling AI use, because that removes the page from search as well.

Bing also reads a training preference on individual pages, and the page-level value overrides robots.txt for that URL. It can be published as a meta tag in the HTML head or as a response header, which is the only option for PDFs and other non-HTML files.

<meta name="Bingbot" content="ai-training:disallow">

X-Robots-Tag: Bingbot: ai-training:disallow

How to set AI Grounding on a page with a meta tag

Grounding has no robots.txt equivalent. Microsoft's documentation states that there is no sitewide grounding preference, so every page that needs a restriction has to carry it in its own HTML or its own HTTP response. On a CMS this usually means adding the tag to a template or exposing it as a field per post.

<!-- Remove the page from all AI grounding -->
<meta name="Bingbot" content="ai-grounding:disallow">

<!-- Keep Microsoft's own AI products, exclude third-party grounding services -->
<meta name="Bingbot" content="ai-grounding:disallowextended">

A page that needs both a training and a grounding preference should carry two tags. Every example in Microsoft's help documentation shows one preference per tag or header line, and it does not describe a combined comma-separated form. Until Microsoft documents one, separate lines are the safe way to publish.

<meta name="Bingbot" content="ai-training:disallow">
<meta name="Bingbot" content="ai-grounding:disallowextended">

Template edits are fine for a handful of page types. For a whole directory, or for a site where the CMS is hard to change, the response header is easier because it can be added once at the server or CDN and applied by URL pattern.

How to set AI Grounding across a section with Cloudflare

Sites behind Cloudflare can add the header at the edge without touching the origin server. A response header Transform Rule does this from the dashboard with no code, and Cloudflare's documentation covers the rule builder in detail. The steps are short.

1. Open the zone in the Cloudflare dashboard, go to Rules, then Transform Rules, and create a rule under Modify Response Header.
2. Set the match condition. For one section, use URI Path starts with /premium/. The generated expression reads (starts_with(http.request.uri.path, "/premium/")).
3. Choose the Add operation so the new header is sent alongside any X-Robots-Tag the origin already returns. The Set operation replaces existing values and can remove a noindex that the CMS relies on.
4. Enter X-Robots-Tag as the header name and Bingbot: ai-grounding:disallow or Bingbot: ai-grounding:disallowextended as the value.
5. Deploy the rule and confirm that it shows as active in the rules list.

A Worker is the better tool when the decision depends on logic that a path match cannot express, such as reading a cookie, a content type or a response from the origin. The script below adds the header to every response on its route and leaves existing headers in place.

export default {
  async fetch(request) {
    const response = await fetch(request);
    const headers = new Headers(response.headers);

    headers.append("X-Robots-Tag", "Bingbot: ai-grounding:disallowextended");

    return new Response(response.body, {
      status: response.status,
      statusText: response.statusText,
      headers,
    });
  },
};

Bind the Worker to a route such as example.com/premium/* and keep it off static assets. A Worker runs on every matching request and counts against the plan's request allowance, which is the main reason to prefer a Transform Rule when a simple path match is enough.

How to set the header in Nginx and Apache

Sites without a CDN in front can send the same header from the web server. Both major servers handle it in one line, and both have a behaviour that causes the header to disappear or to overwrite something else if it is missed.

Nginx

Nginx uses the add_header directive. The always parameter makes Nginx send the header on every response code, including redirects and error pages.

server {
    server_name example.com www.example.com;

    add_header X-Robots-Tag "Bingbot: ai-grounding:disallowextended" always;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }
}

For one directory, the directive goes inside a matching location block.

location ^~ /premium/ {
    add_header X-Robots-Tag "Bingbot: ai-grounding:disallow" always;
    try_files $uri $uri/ /index.php?$args;
}

The Nginx headers module only inherits add_header lines from the level above when the current level defines none of its own. A location block that already sets a cache or security header will silently drop the server-level X-Robots-Tag, so the line has to be repeated in every such block, including the PHP location that many WordPress configurations use. Run sudo nginx -t before sudo systemctl reload nginx so a syntax error never reaches production.

Apache

Apache sets response headers through mod_headers. On Debian and Ubuntu it is enabled with sudo a2enmod headers followed by a restart. The line below works in a virtual host or in the site's root .htaccess file.

<IfModule mod_headers.c>
    Header add X-Robots-Tag "Bingbot: ai-grounding:disallowextended"
</IfModule>

The choice of action matters. According to the mod_headers documentation, Header set replaces any existing header of the same name, which can erase a noindex sent by the application. Header add sends a separate header line and keeps the existing one, which matches the one-preference-per-line form Microsoft documents.

Path scoping belongs in the server configuration, because Directory and LocationMatch containers are not allowed inside .htaccess.

<LocationMatch "^/(archive|premium)/">
    Header add X-Robots-Tag "Bingbot: ai-grounding:disallow"
</LocationMatch>

On shared hosting without access to the virtual host, the alternative is a small .htaccess file placed inside the directory itself, containing only the Header add line. Test with sudo apachectl configtest and reload once it returns Syntax OK.

How to check the header and get Bing to pick it up

The header can be confirmed from any terminal before Bing ever sees it. A HEAD request with curl prints the response headers for a URL.

curl -I https://www.example.com/premium/sample-page/

A correct setup returns the preference on its own line, next to any other robots header the page already had.

HTTP/2 200
x-robots-tag: noindex
x-robots-tag: Bingbot: ai-grounding:disallow

Three checks cover most failures. Test one URL inside the restricted path and one outside it to confirm the scope is right. Test a URL that returns a redirect or a 404 to confirm the header behaves as expected on non-200 responses. Test a page served from cache, because a CDN or page cache can keep serving the old headers until it is purged.

Publishing a preference does not activate it. Bing applies the value only after Bingbot recrawls the page and finds it, and Microsoft gives no guaranteed timing for that. Submitting the changed URLs through IndexNow or Bing Webmaster Tools signals the change and improves the odds of a prompt recrawl, though Microsoft says it does not guarantee a date.

What sites using NoArchive or NoCache must do before March 1, 2027

Since 2023, Bing has read the noarchive and nocache robots values as AI opt-out signals. That arrangement ends on March 1, 2027. After that date Bing no longer reads either tag as a training or grounding preference, and pages that carry only the old tags fall back to Allow for both uses. Microsoft's announcement tells existing users not to assume their old restrictions carry over.

The transition runs on four dates.

• On December 15, 2026, temporary grounding preferences open in Bing Webmaster Tools, only for sites that use NoArchive or NoCache on that date.
• On February 1, 2027, Bing starts honoring the new AI Training and AI Grounding preferences as it crawls.
• March 1, 2027 is the deadline to publish the new preferences and record the temporary grounding preference.
• On September 30, 2027, temporary grounding preferences expire, or earlier for any page where Bing has already recrawled and found the new tag.

Training and grounding need different actions. For training, publishing the robots.txt group or the page-level tag by March 1 is enough, and there is no temporary training setting in Bing Webmaster Tools. For grounding, Microsoft asks for two steps, which are publishing the page-level preference and recording a temporary preference in Bing Webmaster Tools. The temporary preference covers pages that Bing has not yet recrawled, so the restriction holds during the gap.

The work starts with an inventory. A crawl of the site with any SEO crawler will list every URL that returns noarchive or nocache in a meta tag or an X-Robots-Tag header, and that list often includes pages nobody remembers tagging. Each group of URLs then needs a decision using the three combinations above, since a tag added in 2023 to keep content out of Bing Chat may no longer reflect what the business wants from AI search in 2027. Search Agency runs this audit and the header rollout for enterprise sites as part of its technical SEO and AI search work, and the team can be reached through search.agency.