Search Console guide

“Excluded by ‘noindex’ tag”: find where the noindex comes from

Google found a noindex rule on these pages. If you didn’t mean it, here is how to trace it to the tag, the header, the plugin or the setting that added it.

Explore the live demo →

5 days free · no card needed · ends on its own

Updated

“Excluded by ‘noindex’ tag” means Google crawled the page and found a noindex rule, either in a robots meta tag or in an X-Robots-Tag HTTP header, so it left the page out of its index. If you meant to hide the page, there is nothing to do. If you didn’t, find where the noindex comes from (the page’s HTML, the server’s headers, an SEO plugin, a CMS setting or a staging switch), remove it, and ask Google to recrawl.

What “Excluded by ‘noindex’ tag” means

A noindex rule tells search engines not to index a page. When Googlebot crawls the page and finds one, Google drops the page from search results entirely, even if other sites link to it (Google’s noindex documentation). Search Console lists those URLs under Excluded by ‘noindex’ tag; Google’s help page calls the same status “URL marked ‘noindex’”.

The rule can come from two places, and both count:

<!-- in the HTML <head> -->
<meta name="robots" content="noindex">
<meta name="googlebot" content="noindex">

# or as an HTTP response header
X-Robots-Tag: noindex

For Google to see the rule at all, the page must be crawlable. If robots.txt blocks it, the noindex is invisible and the URL can stay indexed from links. More in blocked by robots.txt.

Intended or a mistake?

Usually correct to noindexUsually a mistake
Site search results: /search?q=plumberListing pages: /listing/joes-plumbing
Login, account, claim-listing and thank-you pagesCity, category and category-in-city pages: /plumbers/austin-tx
Empty category-in-city pages, until they have listingsThe home page
Thin tag and date archivesBlog posts and guides
Private or opted-out listings, staging copiesWhole sections after a migration or redesign

Scan the example URLs in the report. If they are all from the left column, you are done. If any page type from the right column shows up, find the source.

Find where the noindex comes from

Work through these in order. The first two tell you what Google saw; the rest tell you who put it there.

1. URL Inspection: what Google saw

Inspect an example URL in the URL Inspection tool. Under Page indexing, Indexing allowed? says No and names where noindex was detected: the robots meta tag or the X-Robots-Tag header. Then click Test live URL and check the same field under Availability. If the live test no longer finds a noindex, the page was already fixed and you can request indexing. If it still does, keep going. View tested page shows the HTML and the HTTP headers Google received (URL Inspection help).

2. The page source and the headers

Check what your server sends, not what your browser shows after scripts have run. View the page source (not the element inspector) and search for noindex. For the header, which you can’t see in the source, use the Network tab of your browser’s developer tools, or curl:

curl -sI https://example.com/listing/joes-plumbing | grep -i x-robots-tag
curl -s  https://example.com/listing/joes-plumbing | grep -i noindex

If curl shows nothing but Google still reports noindex, the rule may be added only for some visitors (a security plugin or CDN rule that treats bots differently) or only after JavaScript runs. Google’s JavaScript SEO basics warns that when Google finds a noindex tag it may skip rendering, so JavaScript that removes a noindex may not work. If you want the page indexed, keep noindex out of the original HTML.

3. Server, CDN and hosting rules

An X-Robots-Tag header usually comes from server configuration (.htaccess, an nginx add_header line), a CDN or edge rule, or a host that adds it to staging and preview domains. Headers are easy to copy from staging to production in a deploy and hard to spot, because nothing in the page shows them.

4. SEO plugins

5. WordPress “Discourage search engines from indexing this site”

The checkbox under Settings > Reading (Search engine visibility). Since WordPress 5.3 it adds a robots meta tag with noindex to the site instead of a robots.txt block (WordPress core announcement). It is often ticked during a build and forgotten at launch.

6. Staging settings left on

Many launches copy the staging site over production with its “hide from search engines” setting still on: the WordPress checkbox, a plugin’s maintenance mode, a host’s staging header. If “Excluded by ‘noindex’ tag” jumps the week you launched or migrated, start here. Make “check indexing settings” a step on every launch checklist.

7. Website builders

8. Directory plugins and themes

Directory software often decides indexing per page: search results, empty categories or locations, private or pending listings, expired or unpaid listings. Those defaults are usually sensible, but check them against the URLs in the report. A plan level that hides listings from search, or a minimum-listings rule set too high, can noindex pages you want in Google.

How to fix it

  1. Remove the noindex from the template, plugin setting, header rule or CMS option that adds it. Fix it at the source, not page by page.
  2. Confirm with curl and with URL Inspection’s live test that Indexing allowed? now says yes.
  3. Make sure the pages are in your sitemap and not blocked by robots.txt.
  4. Click Request indexing for the most important pages. Google’s documentation notes it may take months for Google to revisit less important pages on its own.
  5. In the Page indexing report, open Excluded by ‘noindex’ tag and click Validate fix. How that works is in the page indexing issues guide.

When “excluded” is the right answer

A healthy directory always has pages here: search results, account and claim pages, and category-in-city pages that don’t have listings yet. Noindex is also the right tool for pages that exist for visitors but add nothing in search, such as thin zombie pages you aren’t ready to delete. Don’t use it to pick between duplicates; that is a job for canonical tags (see duplicates).

noindex, robots.txt, canonical or 404: which to use

Noindex is one of several ways to keep a URL out of Google, and using the wrong one is how many indexing problems start:

You want to…UseWhy
Keep a live page out of GooglenoindexGoogle crawls it, sees the rule and drops it, even if other sites link to it.
Stop Google crawling URLs that aren’t indexed and never should berobots.txtSaves crawling. It doesn’t remove URLs that are already indexed. See blocked by robots.txt.
Show one of several versions of the same contentcanonical tag or redirectSignals are consolidated onto one URL. Google advises against noindex for this.
Remove a page that no longer exists404 or 410Not a noindexed placeholder, and not a page that says “not found” at 200, which becomes a soft 404.
Keep content privateA loginNoindex and robots.txt are public instructions. Google’s robots.txt guide points to password protection for private content.
Hide an indexed page right nowRemovals tool, then noindexA temporary removal lasts about six months; the noindex keeps it out after that.

Check noindex across page types with DirectoryLens

DirectoryLens’s Google indexing report shows Google’s status for each inspected URL, by page type, so a template that picked up a noindex stands out as a page type full of excluded URLs instead of a few rows among 1,000 examples. With the optional light crawl on, DirectoryLens reads the robots meta tag and X-Robots-Tag header of a sample of each page type every 30 days, and Thin and unused pages counts how many sampled pages carry a noindex. Its recommendations work the other way too: for category and city pages below your thin-page threshold with no impressions, it suggests noindex until they fill up.

Questions

What does “Excluded by ‘noindex’ tag” mean?

Google crawled the page and found a noindex rule in a robots meta tag or an X-Robots-Tag HTTP header, so it didn’t index the page. It is correct for pages you meant to hide.

How do I find where a noindex comes from?

Inspect the URL in Search Console: Indexing allowed? says whether noindex was found in the robots meta tag or the X-Robots-Tag header. Then check the page source, the response headers (curl -I), your SEO plugin, CMS settings and server or CDN rules.

How do I check the X-Robots-Tag header?

Run curl -sI on the URL and look for an X-Robots-Tag line, or open the Network tab in your browser’s developer tools. URL Inspection’s live test also shows the headers Google received.

Why does Search Console say noindex when I can’t see it on the page?

It may be in an HTTP header rather than the HTML, added only for some user agents, or present in the original HTML before JavaScript changes it. Google may skip rendering when it finds noindex, so removing it with JavaScript may not work.

How long until Google indexes the page after I remove noindex?

Google has to recrawl it. Request indexing for important pages; less important pages can take much longer to be revisited.

Should I block noindexed pages in robots.txt?

No. Google has to crawl a page to see its noindex. If robots.txt blocks it, the URL can stay indexed from links.

Spot a noindexed template the day it happens.

Connect Search Console and DirectoryLens shows Google’s indexing status for every page type, every day.

Explore the live demo →

5 days free · no card needed · ends on its own