Indexing Issues Checklist

When a page is not indexed

If a page is not appearing in Google search results, the first question is whether it is indexed at all. Use the URL Inspection tool in Google Search Console to check. If the page is not indexed, work through this checklist in order. The causes are listed from most common to least common.

1. Check for noindex tags

Look for a noindex directive in the page's HTML head or HTTP headers.

In HTML: Look for <meta name="robots" content="noindex"> or <meta name="googlebot" content="noindex"> in the page source.

In HTTP headers: Check for an X-Robots-Tag: noindex response header. This is invisible in the page source and requires checking the server response headers.

Common causes of accidental noindex:

• CMS settings (WordPress, Shopify, and others have "discourage search engines" options that add noindex)
• Staging site settings that were not removed after launch
• A developer adding noindex during testing and forgetting to remove it
• A plugin or theme adding noindex to certain page types by default

Fix: Remove the noindex directive. Request re-indexing via URL Inspection in GSC.

2. Check robots.txt

If robots.txt blocks Googlebot from crawling the page, it cannot be indexed. Check your robots.txt file (yourdomain.com/robots.txt) for Disallow rules that match the page's URL path.

Common mistakes:

• Blocking an entire directory that contains important pages
• Using wildcards that match more URLs than intended
• Blocking CSS or JavaScript files that Google needs to render the page

Fix: Update robots.txt to allow crawling. Note that removing a robots.txt block does not immediately cause indexing. Google needs to re-crawl the page first.

3. Check canonical tags

If the page has a canonical tag pointing to a different URL, Google may index the canonical target instead of this page.

Check: View the page source and look for <link rel="canonical" href="...">. If it points to a different URL, Google will likely index that URL instead.

Common causes:

• CMS generating incorrect canonical tags for paginated or filtered pages
• A canonical tag pointing to a non-existent or redirecting URL
• Self-referencing canonical with the wrong URL format (HTTP vs HTTPS, www vs non-www)

Fix: Correct the canonical tag to either self-reference or point to the correct preferred URL.

4. Check if Google considers the content thin or duplicate

GSC may show the page as "Crawled - currently not indexed." This often means Google crawled the page but decided it does not add enough value to the index.

Possible reasons:

• The page has very little unique content
• The content is substantially similar to another page on your site or elsewhere on the web
• The page is a tag, category, or archive page with no unique content beyond links
• The page is auto-generated with templated content

Fix: Add substantial, unique content to the page. If the page genuinely has no unique value, consider whether it should exist at all.

5. Check for crawl errors

GSC > Pages report shows crawl errors. Common issues:

• Server errors (5xx): Your server is returning errors when Google tries to crawl the page. Check server logs and hosting.
• Soft 404: The page returns a 200 status code but Google detects that it behaves like a 404 (empty content, "page not found" message). Fix the content or return a proper 404.
• Redirect errors: The page redirects in a loop or through too many hops. Fix the redirect chain.

6. Check page discoverability

Google needs to find the page before it can index it.

Is the page in your sitemap? Check your XML sitemap for the URL. If it is missing, add it.

Is the page linked from other pages? A page with no internal links pointing to it may not be discovered through crawling. Add internal links from relevant existing pages.

Is the page new? New pages can take days to weeks to be crawled and indexed. If the page was published recently, wait. You can request indexing via URL Inspection to speed this up.

7. Check for rendering issues

If the page relies on JavaScript to render its content, Google may not see the content during crawling.

Check: Use the URL Inspection tool and click "View Tested Page" to see the rendered HTML. If the content is missing from the rendered version, you have a rendering problem.

Fix: Move critical content to server-side rendering. See the Rendering and JavaScript Reality page for details.

8. Check for domain-level issues

In rare cases, indexing issues affect the entire domain:

• New domain: Very new domains may take longer to get indexed. Be patient and ensure the site is technically sound.
• Domain penalty: Check GSC for manual actions. Check if the domain was previously used for spam.
• Domain configuration: Verify DNS is configured correctly and the domain resolves properly.

After fixing

After addressing the issue, request re-indexing via the URL Inspection tool in GSC. Monitor the page's status over the next 1 to 4 weeks. If the page is still not indexed after fixing the identified issue, work through the checklist again. Sometimes multiple issues compound.

Practical takeaway

Indexing issues have a finite number of causes. This checklist covers them in order of likelihood. Work through it systematically rather than guessing. Most indexing issues are caused by items 1 through 4 on this list.