An XML sitemap still does the job it always did: it gives a crawler a list of the URLs you want found, and when each one last changed. What changed is the number of crawlers you might hope reads it. The short version is that the two search engines whose indexes feed their own AI answers, Google and Bing, document how they use sitemaps. The AI vendors that run their own crawlers do not.
That split decides what is worth doing. A clean sitemap with honest lastmod dates is plain search hygiene that also reaches AI Overviews, AI Mode, and Copilot through the indexes behind them. Treating the sitemap as a direct line to ChatGPT, Claude, or Perplexity is a claim no vendor has made. This post covers what each system documents, the three fields and limits that matter, how to audit a sitemap against your crawl, and how it relates to llms.txt. It sits under AI crawler access: the technical layer of AI search.
Which AI search systems document using sitemaps?
Google and Bing document sitemap use. OpenAI, Anthropic, and Perplexity do not. Here is what each one publishes:
| System | AI surfaces fed | What its documentation says about sitemaps |
|---|---|---|
| AI Overviews, AI Mode (through Googlebot and the Search index) | Full sitemap documentation; uses lastmod if “consistently and verifiably” accurate | |
| Bing | Copilot and AI summaries in Bing | Sitemaps “remain a foundational signal”; lastmod “remains a key signal” for recrawling |
| OpenAI | ChatGPT search (OAI-SearchBot) | Crawler page covers robots.txt and IP ranges; no mention of sitemaps |
| Anthropic | Claude (Claude-SearchBot, ClaudeBot) | Crawler help article covers robots.txt; no mention of sitemaps |
| Perplexity | Perplexity search (PerplexityBot) | Crawler page covers robots.txt and IP ranges; no mention of sitemaps |
Sources: Google, Bing, OpenAI, Anthropic, Perplexity.
For Google, the route to AI features runs through ordinary indexing. Google’s page on AI features says that to be eligible as a supporting link in AI Overviews or AI Mode, “a page must be indexed and eligible to be shown in Google Search with a snippet,” and that “there are no additional technical requirements” (Google Search Central). Anything that helps Googlebot find and refresh a page helps it there, and a sitemap is one of those things.
Bing is more direct about AI. Its July 2025 post opens by naming “AI powered search engines like Bing Copilot” and says that “for AI powered search engines like Bing, freshness signals directly influence how quickly updates are reflected in search results and AI generated answers” (Bing Webmaster Blog).
For the other three, the honest position is that sitemap use is undocumented. A crawler may well read Sitemap: lines in your robots.txt; nothing published says whether it does or what it does with them. Plan as if their discovery runs on links, and keep your internal linking sound. GPTBot vs OAI-SearchBot has the full agent list for those vendors.
Does lastmod still matter?
Yes, and it matters more than any other field in the file, on one condition: it has to be true. Google says it “uses the <lastmod> value if it’s consistently and verifiably (for example by comparing to the last modification of the page) accurate,” and that the value “should reflect the date and time of the last significant update to the page.” It gives the line between significant and not: an update to “the main content, the structured data, or links on the page is generally considered significant, however an update to the copyright date is not” (Google Search Central).
Bing says the field helps it “prioritize URLs for recrawling and reindexing, or skip them entirely if the content hasn’t changed since the last crawl.” It asks for ISO 8601 values “including both the date and time,” and warns: “Avoid setting lastmod to the time your sitemap was generated unless the content on that URL was actually updated” (Bing Webmaster Blog).
Read those together and the failure is clear. A sitemap generator that stamps every URL with the build time tells both engines that everything changed today. Google’s wording implies what happens next: a value that is not consistently accurate stops being used. One bad deploy can cost you the field for the pages where it was true.
The rule matches the one for on-page dates. The sitemap’s lastmod, the page’s dateModified, and the visible “Updated” line should all move together, and only when the content does. Freshness signals and honest dateModified covers what counts as a modification.
What about changefreq and priority?
Leave them out or leave them alone. They do nothing at either engine that documents sitemaps. Google: “Google ignores <priority> and <changefreq> values.” Bing: “Optional sitemap tags like changefreq and priority are ignored by Bing and do not influence how your content is crawled or ranked.” Time spent tuning them is wasted, and a sitemap that is only <loc> and an honest <lastmod> is complete.
How large can a sitemap be, and when do you need an index?
A single sitemap can hold 50,000 URLs or 50MB uncompressed, whichever comes first, and anything larger must be split. Google states the limit for all formats and says you “must break your sitemap into multiple sitemaps,” optionally listed in a sitemap index file you submit once (Google Search Central). Bing gives the index side: up to 50,000 child sitemaps per index file (Bing Webmaster Blog).
Split by template even when you are under the limit. Google notes that separate sitemaps “may be useful if you want to track the search performance of each individual sitemap in Search Console.” A sitemap-products.xml, sitemap-docs.xml, and sitemap-blog.xml behind one index lets you see indexing coverage per template. That is the view you want when one template has a problem.
Two placement rules from Google’s page: sitemaps must be UTF-8, and unless submitted through Search Console, “a sitemap affects only descendants of the parent directory,” so the root is the right place. Reference the file from robots.txt with a Sitemap: line. Both Google and Bing list that as a submission method, and it is the one route any other crawler that looks for sitemaps would find. robots.txt patterns for AI covers the rest of that file.
Which URLs belong in the sitemap?
Only URLs you want indexed: canonical, returning 200, and not marked noindex. Google’s instruction is to “include the URLs in your sitemap that you want to see in Google’s search results,” using “fully-qualified, absolute URLs” (Google Search Central). Every other entry is a contradiction. A redirected URL says “index this” while the server says “go elsewhere.” A non-canonical variant says “index this” while the page’s own canonical says “index that.” Status codes, redirects, and canonical drift on AI surfaces explains why those contradictions are worse than either signal alone.
The inverse matters too. An indexable page missing from the sitemap and linked from nowhere is an orphan. A sitemap entry may be the only way a crawler finds it, and for crawlers that don’t document sitemap use, it may not be found at all. The fix for orphans is a link, not just a sitemap entry.
How is a sitemap different from llms.txt?
A sitemap lists every URL you want indexed, for search engines that document reading it. llms.txt is a short, curated, Markdown reading list for language-model agents, and no major AI vendor documents reading it. They answer different questions and neither replaces the other. llms.txt: what it is, how it works, and whether to bother covers the proposal and its adoption. The practical order is the sitemap first, because its readers are documented, then llms.txt only if you have a reason.
How do you audit a sitemap against your site?
Without a terminal, Google Sheets can read a sitemap directly. In cell A1:
=IMPORTXML("https://example.com/sitemap.xml", "//*[local-name()='loc']")and in B1, for the dates:
=IMPORTXML("https://example.com/sitemap.xml", "//*[local-name()='lastmod']")local-name() sidesteps the sitemap namespace, which a plain //loc would miss. For an index file, the same formula returns the child sitemap URLs. Import each child the same way. Then, with your crawler’s export of URL, status, indexability, and canonical in a second tab, three checks:
- Bad entries.
=VLOOKUP(A2, Crawl!A:D, 2, FALSE)next to each sitemap URL. Any status other than 200, any non-indexable row, or any canonical that differs from the URL is an entry to remove or fix. - Missing pages. In the crawl tab,
=COUNTIF(Sitemap!A:A, A2)=0flags indexable pages the sitemap omits. - Stamped dates.
=COUNTIF(B:B, B2)on thelastmodcolumn. When hundreds of URLs share one timestamp, it is the build-time signature Bing warns about.
Search Console’s Sitemaps report and Bing Webmaster Tools’ Sitemaps section show whether each engine fetched the file, when it last did, and any errors. Bing says it revisits a submitted sitemap “typically at least once per day.” Server logs show who else requests it. Reading server logs for AI bot traffic covers filtering by user agent, and a log line of an AI crawler fetching /sitemap.xml is the only evidence you will get that it does.
In a terminal, the URL and date lists for one sitemap:
curl -s https://example.com/sitemap.xml | grep -oE '<loc>[^<]+' | sed 's/<loc>//'
curl -s https://example.com/sitemap.xml | grep -oE '<lastmod>[^<]+' | sort | uniq -c | sort -rn | headThe second command prints the most-repeated lastmod values first. One value with a very large count is the stamped-date fault.
GEOBee discovers and parses your sitemap during a crawl, so you can filter the results down to the URLs actually listed in it (crawl settings), and it raises orphan pages as a site-wide finding. That covers the first two checks above in one pass. The lastmod honesty check stays a comparison you make against your own edit history, because no crawler can see when you last changed a page.
To see which URLs in your sitemap redirect, fail, or aren’t indexable, and which indexable pages it leaves out, download GEOBee and crawl your site with the sitemap filter on.