Updated
A sitemap checker fetches your XML sitemap the way a search engine does and tests it against the sitemaps protocol and Google’s rules: the HTTP status, the content type, whether the XML is valid, the 50,000-URL and 50MB limits, and whether robots.txt points to it. This one goes a step further. It also fetches an even spread of up to 50 of the URLs your sitemap lists and checks whether each one is really indexable, because a perfectly valid sitemap full of redirects, noindex pages and non-canonical URLs is the more common problem.
What the sitemap checker tests
| Check | What it looks for |
|---|---|
| HTTP status | The sitemap answers 200. A 404, 403 or 5xx, or a redirect, is reported, with the URL it redirects to. |
| Content type | Served as application/xml or text/xml. A text/html page at the sitemap URL usually means the site returned its home page instead. |
| XML validity | One <urlset> or <sitemapindex> root with the sitemaps namespace, balanced tags, no unescaped & and nothing before the XML declaration. Errors come with a line number. |
| Sitemap index and gzip | Follows an index to its child sitemaps (up to 20) and unzips .gz files. Flags a nested index and a .gz file that isn’t really gzipped. |
| Size limits | At most 50,000 URLs and 50MB uncompressed per file, and 50,000 sitemaps per index. |
| robots.txt | Whether robots.txt lists this sitemap, lists a different one, or blocks Googlebot from fetching it. |
| lastmod | The share of URLs with a <lastmod> date, dates that aren’t in W3C format, and dates in the future. |
| Duplicates | URLs listed more than once, within a file or across child sitemaps. |
| URLs on another host | URLs whose host or protocol differs from the sitemap’s, such as http:// URLs in an https:// sitemap, or a missing www. |
| Status of each sampled URL | Up to 50 URLs spread evenly across the whole sitemap: 4xx and 5xx errors, and every redirect with where it lands. |
| noindex | A noindex in the robots meta tag or the X-Robots-Tag HTTP header of each sampled page. |
| Canonical | A canonical tag that points to a different URL than the one listed. |
| Blocked URLs | Sampled URLs that robots.txt blocks for Googlebot. |
Why it tests the URLs, not just the XML
Most sitemap validators stop at the file: is the XML well-formed, is it under the limits. Those problems are real but rare, because sitemaps are written by plugins that get the XML right. What plugins get wrong is which URLs go in. Google’s sitemap guidelines say to list only the canonical URLs you want in search results, and Search Console flags the rest in the Page indexing report:
- URLs that redirect show up as Page with redirect. Common after a move to https or a change of permalinks, when the sitemap still lists the old addresses.
- URLs with a noindex are reported as “Submitted URL marked ‘noindex’”. Typical for expired listings, empty categories and thank-you pages that a plugin adds anyway. See excluded by noindex tag.
- URLs whose canonical points elsewhere become alternate pages. Google treats sitemap inclusion as a weak canonical signal, so a sitemap that disagrees with your canonical tags sends mixed messages.
- URLs that return 404 or 410 waste crawling on pages that are gone, and URLs blocked by robots.txt are reported as blocked by robots.txt.
The sample is spread evenly through the whole list, not taken from the top, so on a directory with separate sitemaps for listings, cities and categories it covers every page type. When one template is wrong, say every expired listing has a noindex, it shows up in the sample even if that template is only a fifth of the sitemap.
How to read the results
| Label | Meaning |
|---|---|
| ✕ Problem | Something Google reports as an error or that stops the sitemap, or the URLs in it, being used: an HTTP error, invalid XML, a broken limit, noindex or blocked URLs. |
| ! Warning | Works, but costs you something: redirects, canonicals pointing elsewhere, duplicates, missing lastmod dates, a missing robots.txt line. |
| i Note | Worth knowing, not necessarily wrong, such as robots.txt listing a different sitemap. |
| ✓ Passed | Checked and fine. |
The URL sample counts are out of the URLs tested, not the whole sitemap. If 6 of 50 sampled URLs redirect, expect roughly 12% of the sitemap to redirect, and look at which kind of URL they are to find the template or plugin setting behind them.
This checker vs Search Console
Use both. This checker shows the state of the file and its URLs right now, in one run, without access to the site’s Search Console. Search Console’s Sitemaps report is the authority on what Google itself managed to read, and when; see sitemap could not be read for its messages.
One limit to keep in mind: this checker fetches as DirectoryLensBot, not as Googlebot. If a firewall, CDN or security plugin blocks only Googlebot, the sitemap will pass here and still show “Couldn’t fetch” in Search Console. The live test in the URL Inspection tool fetches with Google’s own crawler and is the check for that.
Limits of the free checker
- It reads the sitemap, and for an index up to 20 child sitemaps, and tests up to 50 URLs. A run stops after about 45 seconds and reports what it reached.
- Each visitor can run 5 checks every 10 minutes.
- It fetches as
DirectoryLensBotand obeys robots.txt for that name when testing pages; a page that robots.txt closes to DirectoryLensBot but not to Googlebot is reported as not fetched. - Only public sites: addresses on private networks are refused.
- Results aren’t stored. Nothing about the sitemap or its URLs is kept after the page shows them.
Fixing what it finds
- Error status or HTML instead of XML: the path is wrong or the sitemap feature is off. On WordPress the index is usually
/sitemap_index.xml(Yoast, Rank Math) or/wp-sitemap.xml(core). The sitemap read checklist goes through every cause. - Text before the XML: a PHP notice, a stray space in a theme file or a plugin echoing output. Turn off error display on the live site and find the plugin that prints it.
- Unescaped &: URLs with parameters must write
&as&. Better still, keep parameter URLs out of the sitemap. - Redirects: list the final URL. Usually a site-wide setting (https, www, trailing slash) the sitemap plugin doesn’t know about.
- noindex or canonical elsewhere: leave those URLs out of the sitemap, or remove the noindex if the page should be indexed. Yoast and Rank Math leave a post type or taxonomy out of the sitemap when you set it to noindex in the plugin’s own settings, so set it there rather than in the theme.
- Not listed in robots.txt: add a
Sitemap:line; see sitemap in robots.txt for the syntax. - Over the limits: split the file and list the parts in a sitemap index.
Sitemaps on directory websites
A directory’s sitemap is mostly listings, and listings change: they expire, merge, move city or lose their owner. Every one of those leaves a URL that may now redirect, return 404 or carry a noindex, and the sitemap plugin often keeps listing it. Run the checker on the listings sitemap after any bulk import, cleanup or plugin update, and keep listings, cities and categories in separate sitemaps so you can test them one at a time. The page indexing issues guide explains what Google does with each kind of URL.
Check every sitemap every day
The checker is a one-off test. DirectoryLens does the ongoing version from your Search Console connection: the Sitemaps tab in Google indexing re-reads every sitemap you submitted each day, follows indexes, unzips gzipped files, shows URL counts per sitemap over time, lists any sitemap it couldn’t load, and alerts you when a sitemap or the total loses 20% or more of its URLs within a week. Those sitemap URLs then feed daily checks with Google’s URL Inspection API, so you see how much of each page type Google has actually indexed.