Updated
“Excluded by ‘noindex’ tag” means Google crawled the page and found a noindex rule, either in a robots meta tag or in an X-Robots-Tag HTTP header, so it left the page out of its index. If you meant to hide the page, there is nothing to do. If you didn’t, find where the noindex comes from (the page’s HTML, the server’s headers, an SEO plugin, a CMS setting or a staging switch), remove it, and ask Google to recrawl.
What “Excluded by ‘noindex’ tag” means
A noindex rule tells search engines not to index a page. When Googlebot crawls the page and finds one, Google drops the page from search results entirely, even if other sites link to it (Google’s noindex documentation). Search Console lists those URLs under Excluded by ‘noindex’ tag; Google’s help page calls the same status “URL marked ‘noindex’”.
The rule can come from two places, and both count:
<!-- in the HTML <head> -->
<meta name="robots" content="noindex">
<meta name="googlebot" content="noindex">
# or as an HTTP response header
X-Robots-Tag: noindexFor Google to see the rule at all, the page must be crawlable. If robots.txt blocks it, the noindex is invisible and the URL can stay indexed from links. More in blocked by robots.txt.
Intended or a mistake?
| Usually correct to noindex | Usually a mistake |
|---|---|
Site search results: /search?q=plumber | Listing pages: /listing/joes-plumbing |
| Login, account, claim-listing and thank-you pages | City, category and category-in-city pages: /plumbers/austin-tx |
| Empty category-in-city pages, until they have listings | The home page |
| Thin tag and date archives | Blog posts and guides |
| Private or opted-out listings, staging copies | Whole sections after a migration or redesign |
Scan the example URLs in the report. If they are all from the left column, you are done. If any page type from the right column shows up, find the source.
Find where the noindex comes from
Work through these in order. The first two tell you what Google saw; the rest tell you who put it there.
1. URL Inspection: what Google saw
Inspect an example URL in the URL Inspection tool. Under Page indexing, Indexing allowed? says No and names where noindex was detected: the robots meta tag or the X-Robots-Tag header. Then click Test live URL and check the same field under Availability. If the live test no longer finds a noindex, the page was already fixed and you can request indexing. If it still does, keep going. View tested page shows the HTML and the HTTP headers Google received (URL Inspection help).
2. The page source and the headers
Check what your server sends, not what your browser shows after scripts have run. View the page source (not the element inspector) and search for noindex. For the header, which you can’t see in the source, use the Network tab of your browser’s developer tools, or curl:
curl -sI https://example.com/listing/joes-plumbing | grep -i x-robots-tag
curl -s https://example.com/listing/joes-plumbing | grep -i noindexIf curl shows nothing but Google still reports noindex, the rule may be added only for some visitors (a security plugin or CDN rule that treats bots differently) or only after JavaScript runs. Google’s JavaScript SEO basics warns that when Google finds a noindex tag it may skip rendering, so JavaScript that removes a noindex may not work. If you want the page indexed, keep noindex out of the original HTML.
3. Server, CDN and hosting rules
An X-Robots-Tag header usually comes from server configuration (.htaccess, an nginx add_header line), a CDN or edge rule, or a host that adds it to staging and preview domains. Headers are easy to copy from staging to production in a deploy and hard to spot, because nothing in the page shows them.
4. SEO plugins
- Yoast SEO: each content type and taxonomy has a “Show … in search results?” setting, and each post has “Allow search engines to show this … in search results?” in the Advanced tab of the Yoast box. Turning a whole content type off noindexes all of it (Yoast help).
- Rank Math: robots meta settings for each post type and taxonomy under Titles & Meta, a per-post Advanced tab, and a global “Noindex Empty Category and Tag Archives” option (Rank Math docs). That option matters on directories built on WordPress taxonomies, where a category with no posts can still be a page you want.
5. WordPress “Discourage search engines from indexing this site”
The checkbox under Settings > Reading (Search engine visibility). Since WordPress 5.3 it adds a robots meta tag with noindex to the site instead of a robots.txt block (WordPress core announcement). It is often ticked during a build and forgotten at launch.
6. Staging settings left on
Many launches copy the staging site over production with its “hide from search engines” setting still on: the WordPress checkbox, a plugin’s maintenance mode, a host’s staging header. If “Excluded by ‘noindex’ tag” jumps the week you launched or migrated, start here. Make “check indexing settings” a step on every launch checklist.
7. Website builders
- Wix: a site-wide “Let search engines index your site” toggle in SEO Settings, and a per-page setting to hide a page from search engines (Wix help).
- Squarespace: a per-page option in the SEO tab of page settings to hide the page from search results, and noindex through code injection (Squarespace help).
- Shopify: noindex is added through the theme’s
theme.liquidcode or theseo.hiddenmetafield (Shopify help), so look there and in installed apps.
8. Directory plugins and themes
Directory software often decides indexing per page: search results, empty categories or locations, private or pending listings, expired or unpaid listings. Those defaults are usually sensible, but check them against the URLs in the report. A plan level that hides listings from search, or a minimum-listings rule set too high, can noindex pages you want in Google.
How to fix it
- Remove the noindex from the template, plugin setting, header rule or CMS option that adds it. Fix it at the source, not page by page.
- Confirm with curl and with URL Inspection’s live test that Indexing allowed? now says yes.
- Make sure the pages are in your sitemap and not blocked by robots.txt.
- Click Request indexing for the most important pages. Google’s documentation notes it may take months for Google to revisit less important pages on its own.
- In the Page indexing report, open Excluded by ‘noindex’ tag and click Validate fix. How that works is in the page indexing issues guide.
When “excluded” is the right answer
A healthy directory always has pages here: search results, account and claim pages, and category-in-city pages that don’t have listings yet. Noindex is also the right tool for pages that exist for visitors but add nothing in search, such as thin zombie pages you aren’t ready to delete. Don’t use it to pick between duplicates; that is a job for canonical tags (see duplicates).
noindex, robots.txt, canonical or 404: which to use
Noindex is one of several ways to keep a URL out of Google, and using the wrong one is how many indexing problems start:
| You want to… | Use | Why |
|---|---|---|
| Keep a live page out of Google | noindex | Google crawls it, sees the rule and drops it, even if other sites link to it. |
| Stop Google crawling URLs that aren’t indexed and never should be | robots.txt | Saves crawling. It doesn’t remove URLs that are already indexed. See blocked by robots.txt. |
| Show one of several versions of the same content | canonical tag or redirect | Signals are consolidated onto one URL. Google advises against noindex for this. |
| Remove a page that no longer exists | 404 or 410 | Not a noindexed placeholder, and not a page that says “not found” at 200, which becomes a soft 404. |
| Keep content private | A login | Noindex and robots.txt are public instructions. Google’s robots.txt guide points to password protection for private content. |
| Hide an indexed page right now | Removals tool, then noindex | A temporary removal lasts about six months; the noindex keeps it out after that. |
Check noindex across page types with DirectoryLens
DirectoryLens’s Google indexing report shows Google’s status for each inspected URL, by page type, so a template that picked up a noindex stands out as a page type full of excluded URLs instead of a few rows among 1,000 examples. With the optional light crawl on, DirectoryLens reads the robots meta tag and X-Robots-Tag header of a sample of each page type every 30 days, and Thin and unused pages counts how many sampled pages carry a noindex. Its recommendations work the other way too: for category and city pages below your thin-page threshold with no impressions, it suggests noindex until they fill up.