Crawling vs Indexing vs Ranking
The three phases of how Google processes your site.
Most SEO problems trace back to confusion between crawling, indexing, and ranking. Each phase has different mechanics, different failure modes, and different fixes.
Three different processes
Crawling, indexing, and ranking are three separate processes that happen at different times, for different reasons, with different failure modes. Most SEO confusion comes from treating them as one thing.
Crawling
Crawling is discovery. Googlebot visits a URL, downloads the HTML, and follows the links it finds. That is it.
What triggers a crawl:
- Googlebot follows a link from a page it already knows
- A URL appears in your XML sitemap
- You request a crawl via Google Search Console
- Google decides to re-crawl a page it already knows (refresh crawl)
What can go wrong:
- Robots.txt blocks Googlebot from the URL
- The server is too slow or returns errors
- The page is buried too deep in the site structure (many clicks from the homepage)
- No internal or external links point to the page
How to check: Google Search Console > Pages report shows crawl status. Server logs show actual Googlebot visits.
A page being crawled does not mean it will be indexed. Crawling is just the first step.
Indexing
Indexing is processing. After Google crawls a page, it decides whether to add it to the index. This involves parsing the HTML, rendering any JavaScript, understanding the content, and evaluating whether the page adds value to the index.
What can prevent indexing:
- A noindex tag tells Google not to index the page
- Google considers the page a duplicate of another page
- Google considers the content too thin or low quality
- The page returns a soft 404 (200 status but no real content)
- Rendering fails and Google cannot see the actual content
How to check: Google Search Console > Pages report shows indexing status and reasons for exclusion. The URL Inspection tool shows whether a specific URL is indexed.
A page being indexed does not mean it will rank well. Indexing means Google has stored it. Ranking is a separate evaluation.
Ranking
Ranking is competition. When a user searches, Google evaluates all indexed pages that might be relevant and ranks them by how well they satisfy the query.
What affects ranking:
- Relevance to the query (does the content match what the searcher wants?)
- Content quality (is the content thorough, accurate, and original?)
- Authority (do other trusted sites link to this page or site?)
- Page experience (does the page load fast, work on mobile, and feel safe?)
- User satisfaction signals (do searchers engage with this result?)
How to check: Google Search Console > Performance report shows average position, impressions, and clicks for specific queries.
The diagnostic sequence
When a page is not performing in search, diagnose in order:
- Is it crawled? Check server logs or GSC crawl stats. If not crawled, fix discovery issues first.
- Is it indexed? Check GSC Pages report or URL Inspection. If not indexed, fix indexing issues.
- Is it ranking? Check GSC Performance for the target queries. If indexed but not ranking, the problem is competitiveness.
This sequence matters because fixes at each level are completely different. Improving content quality will not help a page that is not indexed. Fixing a noindex tag will not help a page that is not being crawled.
Takeaway
When something is wrong, always start by identifying which phase is failing. The fix for a crawling problem is never the same as the fix for a ranking problem. Diagnose first, then act.