- Identify which pages suck up crawl budget without passing value and nofollow them immediately.
- Use noindex when you want the page hidden from search results but still want visitors to reach it directly.
- Always pair nofollow with a robots.txt disallow if you truly don’t want the page discovered at all.
- Run a site audit before toggling any tags so you don’t accidentally hide your money pages.
- Check Google Search Console after changes to confirm the tags are recognized and not causing crawl errors.
Open with a one-line contrarian claim, no preamble.
Most SEO audits I inherit treat noindex and nofollow as the same button, but they’re actually opposite sides of the same coin and confusing them is how sites quietly bleed link equity while hiding pages they actually need indexed. I’ve seen campaigns where a client noindexed a high-authority blog archive, thinking it would boost rankings, only to watch organic traffic tank because the page vanished from results entirely. The reality is, noindex is about visibility—keeping a page out of SERPs—whereas nofollow is about authority—stopping PageRank from flowing out through outbound or internal links. If you’ve failed at this before, you’re not alone; most teams I talk to mix up the intent, apply the tag globally instead of page-by-page, and then wonder why their crawl budget gets wasted or their link juice stops moving where it should. Let’s break down the step-by-step so you can stop guessing and start controlling exactly what Google sees and what it passes along.
Diagnosing the Core Confusion
Before we touch a single line of code, let’s agree on what we’re actually solving. In the majority of technical SEO audits I inherit, the first observation is a blanket application of noindex or nofollow tags across entire site sections. This is usually a knee-jerk reaction to a perceived “thin content” penalty risk or an overzealous attempt to clean up a bloated index. I’ve seen e-commerce sites noindex their entire filter-and-sort parameter pages, believing they were protecting the site from duplicate content sanctions. What actually happened? The site’s category pages, which relied on link equity flowing through those parameter URLs to reach the main product listings, saw a significant drop in rankings. The noindex tag had effectively erected a wall between the site’s architecture and the search engine’s crawler, preventing the discovery of the money pages.
The confusion stems from the nomenclature. “Noindex” and “nofollow” sound like two sides of the same coin, and in a superficial sense, they are both meta robots directives. But their operational targets are fundamentally different. Noindex is a visibility switch. It tells search engines, “I know this page exists, and I’m asking you not to include it in your search results.” Fnofollow is an authority switch. It tells search engines, “I’m not vouching for the links on this page, and I don’t want PageRank to flow through them.”
When you noindex a page, you are essentially putting a “Do Not Enter” sign on the front of the house. The mail carrier can still walk up the path, but they won’t drop a letter in the box. When you nofollow a page’s outbound links, you are telling the search engine, “I’m not vouching for the folks down the street.” The search engine can still walk down the path, but it won’t pass any credit to the destinations.
This distinction is critical because of how Google’s indexing system works. Google’s crawlers are discovery machines. If you apply noindex to a page, Googlebot may still crawl that page to understand its content and outbound links, especially if other pages on your site link to it. The noindex directive is a suggestion to the indexing system, not a hard barrier to crawling. Conversely, if you apply nofollow to a page, Googlebot will still crawl the page and discover the outbound links, but it will choose not to assign any ranking credit to those linked pages. It will still add those discovered links to its graph, just without the weight of your endorsement.
I recall a specific instance from about three years ago working with a SaaS client who had a massive resource library. They wanted to block low-quality blog posts from appearing in search results because they were outdated and embarrassing. They applied a site-wide noindex directive via their robots.txt file, thinking this would solve the index bloat problem. The result was a 15% drop in organic traffic across the board. Why? Because the noindex signal had suppressed the crawl frequency of the entire section. Googlebot stopped prioritizing those URLs, and in the process, it stopped discovering and indexing the newer, high-quality articles that were linked within those old posts. The cure was to remove the noindex from the high-value posts and selectively apply noindex only to the truly obsolete pages, while using nofollow on the outbound links within those posts to prevent passing link equity to outdated third-party references. The traffic recovered within two crawl cycles.
The Anatomy of a Noindex Directive
Let’s get granular about the noindex directive. When you implement noindex, you have two primary methods at your disposal: the meta robots tag and the X-Robots-Tag HTTP header. The meta robots tag is the most common approach and looks like this in the HTML head section:
<meta name="robots" content="noindex">
This is page-specific. It only affects the page on which it resides. If you want to noindex a directory, you would need to ensure every single file within that directory contains the tag, or you use the X-Robots-Tag header at the server level.
The X-Robots-Tag is more powerful for non-HTML files. If you have PDFs, images, or other binary files that you want to block from indexing, the X-Robots-Tag in the HTTP response header is the way to go. A typical configuration would look like this in an Apache .htaccess file or NGINX configuration:
Header set X-Robots-Tag "noindex"
This is particularly useful for blocking PDF catalogs or draft documents from appearing in image search or regular web search.
Now, the critical nuance here is the difference between “not indexing” and “not caching.” When a page is noindexed, Google may still keep a cached version of the page for a short period, but it will eventually drop it from the index. If you change the tag and want Google to re-evaluate, you need to ensure the page is crawlable. If the page is blocked by robots.txt, Google cannot see the noindex tag, and the page will remain in the index (or be re-indexed) based on other signals like internal links and sitemap presence.
I once audited a large publishing site that had accidentally noindexed their author archive pages via a plugin misconfiguration. The authors had significant personal brands and were driving social traffic, but they weren’t ranking. The site owner was panicking, thinking there was a penalty. It turned out the noindex had been on for six months. Once we removed the tag and submitted the URLs through Search Console, the pages re-entered the index within 72 hours. The lesson? Noindex is not a “set and forget” switch. It requires maintenance and monitoring.
The Anatomy of a Nofollow Directive
Moving to nofollow, the directive operates primarily on the link level. The standard HTML implementation looks like this:
<a href="https://example.com" rel="nofollow">Anchor Text</a>
This tells search engines, “Do not assign any ranking credit to the destination of this link.” It is vital for user-generated content (UGC), sponsored posts, and any paid or untrusted links. Google’s official stance, especially after the 2019 link attribute update, is that nofollow is a hint, not a hard command. However, in practice, it remains the primary tool for controlling link equity flow.
There are actually three link attributes you should be aware of, though nofollow is the one most people focus on:
- rel=”nofollow”: The classic. Tells Google not to pass PageRank.
- rel=”ugc”: User Generated Content. For comments and forums. It’s effectively the same as nofollow in terms of PageRank flow, but it’s a semantic label.
- rel=”sponsored”: Paid or affiliate links. Also treated as a hint by Google, similar to nofollow.
In practice, if you are doing link building, you’ll encounter nofollow links constantly. High-authority domains like news sites often apply nofollow to outbound links to protect their own PageRank flow, or simply as a policy. When you are building links, you need to accept that a “dofollow” link is a precious commodity. If you get a link from a major news outlet and it’s nofollowed, it’s still valuable for traffic and brand visibility, but it won’t move the needle on your PageRank directly.
A common scenario I see is e-commerce sites nofollowing all internal links to their “Shop All” or filter pages. The intention is usually to conserve PageRank for the product pages. However, if you nofollow every single internal navigation link, you create a crawl trap. Googlebot will eventually stop crawling those filter pages because it receives no signal to follow the links. If those filter pages contain unique content or long-tail keyword variations, you are inadvertently suppressing their potential to rank. The strategy should be more nuanced: nofollow the links to pages that truly have no SEO value (like login pages, admin interfaces, or thank-you pages that have no reason to be indexed), but allow internal links to flow to important category pages.
I remember working with a retail client who had a massive “Outfit Builder” tool. Users could mix and match clothing items. The tool generated thousands of unique URL combinations. The client wanted to nofollow all the “Generate Outfit” buttons to save PageRank. The problem was that these generated pages were actually starting to rank for long-tail fashion queries like “purple shirt with green pants outfit.” By blindly nofollowing the internal navigation, we were suppressing a whole subsection of the site that was actually driving free traffic. We had to selectively keep some of those internal links dofollow while nofollowing the actual “Generate” button JavaScript actions that led to dead-end URLs. It required digging into the JavaScript event tracking, but the result was a more balanced flow of link equity.
Step-by-Step: Implementing Noindex
Okay, let’s get practical. If you’ve decided you need to noindex a page, here is the exact workflow I run through with clients. This isn’t theoretical; this is the process I use when taking over a new site audit.
Step 1: Inventory and Classification. Open your SEO crawler of choice (Screaming Frog, Sitebulb, or Ahrefs). Pull a list of all indexable URLs. I usually filter for URLs that have a meta robots tag already set to noindex, and I also pull a list of URLs that are blocked in robots.txt. You need to know what’s currently happening before you change anything. Create a spreadsheet with three columns: URL, Current Status (indexed/noindex/blocked), and Intended Action.
Step 2: The Audit for Accidental Blocking. This is the step most people skip, and it’s the source of 90% of the support tickets I get. Check your robots.txt file. If a page is disallowed in robots.txt, Googlebot will not crawl it. If Googlebot doesn’t crawl it, it cannot see the noindex meta tag. The noindex tag is effectively useless if the page is blocked from crawling. Run a check: are the pages you want to noindex blocked in robots.txt? If yes, remove the disallow rule first, wait for a recrawl, and then apply the noindex tag. I’ve seen so many cases where a site owner says, “I noindexed my thank you page,” but the thank you page was also blocked in robots.txt. The page stayed indexed because Google never even looked at the meta tag.
Step 3: Applying the Tag.
If it’s a standard HTML page, edit the head section to add <meta name="robots" content="noindex">. If you are using a CMS like WordPress, use a plugin like RankMath or Yoast, or the built-in settings if you’re on Webflow or Shopify. If you are dealing with PDFs or non-HTML files, configure the X-Robots-Tag at the server level.
Step 4: Submission and Monitoring. Once the tag is live, go into Google Search Console. Use the “URL Inspection” tool for the specific page. Click “Request Indexing.” This sends a fresh crawl request to Googlebot. Do not rely on the automatic discovery process if you are in a hurry. Monitor the “Indexing” report over the next two weeks. You’ll see the page move from “Indexed” to “Discovered – currently not indexed” and eventually drop out of the index if the tag is respected.
Step 5: The “Crawl Budget” Check. After noindexing a set of pages, check your server logs or the Crawl Stats in Search Console. You should see a slight decrease in total crawl requests if you noindexed a large section. If you don’t see a decrease, you might have accidentally noindexed pages that Google was already avoiding, or the noindex tag isn’t being picked up correctly. If you see a spike in 404 errors or soft 404s, double-check the implementation.
Step-by-Step: Implementing Nofollow
Implementing nofollow follows a similar logical path, but the focus is on the links themselves.
Step 1: Link Audit. Use a crawler to export all outbound links from the pages you are targeting. I like to pull the URL, the target URL, and the anchor text. Categorize them. Are they affiliate links? Are they links to user-generated content in comments? Are they internal navigation links going to low-value pages?
Step 2: Determining the Scope.
Decide if you want to apply nofollow to individual links or globally via a robots meta tag. The meta robots tag <meta name="robots" content="nofollow"> tells Google not to follow any links on that page. This is a heavy hand. Use it sparingly. If you only want to nofollow specific outbound links, you must add rel="nofollow" to the specific <a> tags.
Step 3: Implementation.
For WordPress, there are plugins that automatically add nofollow to all external links, or you can use the block editor’s link settings to add the attribute manually. For a site-wide approach via .htaccess or NGINX, you can’t really nofollow all links via the server header easily without complex regex rules, so the HTML tag method is preferred for granular control.
Step 4: The UGC and Sponsored Distinction.
If you have a comment section, you should be adding rel="ugc" to links in comments. If you have a blog post where you are mentioning a product you were paid to review, use rel="sponsored". Google’s algorithm looks for these signals. If you just slap rel="nofollow" on everything, you’re missing out on the semantic opportunity to tell Google exactly why the link is untrusted.
Step 5: Monitoring Link Equity Flow. After implementing nofollow, check your internal link reports. You want to ensure that PageRank is flowing to the pages you want to rank. If you nofollowed a bunch of internal navigation and now your important category pages have dropped in rankings, you’ve gone too far. Use the “Links” report in Search Console to see the “Top linked pages” and ensure the flow is still reaching your money pages.
Common Mistakes and How to Avoid Them
This is where I see the most pain in the field. Let’s catalog the errors so you don’t repeat them.
Mistake 1: The “Set and Forget” Noindex. As I touched on earlier, applying noindex to a page and never checking back on it is a recipe for index bleeding. Pages you noindexed six months ago for a site redesign might need to be re-indexed if the content has become relevant again (think of a seasonal guide that is now timely). Set a quarterly review cycle. Open Search Console, check the “Excluded by noindex” report, and ask: “Do we still want these hidden?”
Mistake 2: Noindexing Pagination. A classic blunder. I see it all the time: someone noindexes page 2, 3, 4 of a paginated series because they think it prevents duplicate content. Google’s Martin Mueller and the team have stated clearly that noindexing pagination is generally not recommended. Google uses rel=”next” and rel=”prev” tags (or their modern equivalents) to understand paginated series. If you noindex page 2, Google might stop crawling the entire series, meaning your first page might not get the full link equity flow from the subsequent pages. If you must handle pagination, use a “view all” page or proper canonicalization rather than a blunt noindex.
Mistake 3: Nofollowing Internal Navigation. As I mentioned with the retail client, nofollowing internal links to save PageRank often backfires. If you nofollow the navigation links to your category pages, you are telling Google, “I don’t want you to go there via these links.” Google will eventually stop crawling those pages as frequently, and their rankings can slip. Reserve nofollow for external links, untrusted content, and specific low-value pages like login screens or thank-you pages that have zero SEO value.
Mistake 4: Confusing robots.txt with Meta Tags. This is the beginner trap. Blocking a page in robots.txt prevents Google from seeing the noindex or nofollow tag. If you want a page gone, you must use noindex (if you want Google to still crawl it but not index it) or a “noindex, nofollow” combined with ensuring the page is crawlable. If you want a page completely invisible and you don’t care if Google ever discovers it, then you block it in robots.txt. But most of the time, people want the page to be inaccessible in search results but still discoverable by users clicking through from other pages on the site. In that case, noindex is the correct tool, and robots.txt should be left alone.
Mistake 5: Applying Nofollow to All Outbound Links on a Page.
Sometimes, in an attempt to “save PageRank,” site owners add a global nofollow meta tag to pages with many outbound links. This stops Google from following any link on the page, including potentially useful internal links buried in the content or footer. Be surgical. Add rel="nofollow" only to the specific anchor tags that point to untrusted or paid destinations.
Noindex vs Nofollow: A Comparison
Let’s put them side-by-side in a benchmark format. This helps clarify when to use which.
| Feature | Noindex | Nofollow |
|---|---|---|
| Primary Purpose | Controls visibility in SERPs. | Controls authority flow (PageRank). |
| What it Blocks | The page from appearing in search results. | Link equity from passing through outbound/inbound links. |
| Crawling Impact | Googlebot may still crawl the page to read the tag and content. | Googlebot will crawl the page and discover the linked URLs, but won’t pass credit. |
| Implementation | Meta tag <meta name="robots" content="noindex"> or X-Robots-Tag header. |
HTML anchor tag rel="nofollow" or meta robots tag. |
| Best Use Case | Archive pages, thank-you pages, duplicate content, staging environments. | Affiliate links, user comments, paid sponsorships, untrusted external sites. |
| Risk if Misapplied | Page vanishes from search results entirely, even if users might want to find it. | Link equity is wasted; low-value pages may lose crawl priority. |
| Interaction with robots.txt | Critical. If the page is blocked in robots.txt, the noindex tag is ignored. | Less critical. Google can still discover the links even if the page is blocked, though they won’t be crawled. |
The table above encapsulates the operational difference. If your goal is “I never want this page to appear when someone Googles ‘best widgets’,” you reach for noindex. If your goal is “I don’t want to give my SEO credit to this external affiliate partner,” you reach for nofollow. Using the wrong one is like using a hammer to screw in a lightbulb; you’ll get the job done in a pinch, but you’ll damage the surrounding material.
Real-Talk Conclusion
We’ve covered a lot of ground, from the fundamental definitions to the nitty-gritty of server headers and the pitfalls of misapplication. The most important takeaway I can leave you with is this: noindex and nofollow are not interchangeable levers. They control different dimensions of how search engines interact with your site. Noindex is about keeping a page out of the results. Nofollow is about keeping PageRank from flowing out. Confusing the two is the fastest way to accidentally hide a page you need ranked or waste crawl budget on pages that should be passing authority.
If you walk away from this article remembering one thing, let it be the robots.txt interaction. If a page is disallowed in robots.txt, Google will never see the noindex tag. It’s a simple technical detail, but it’s the reason so many “noindex fails” happen. Always ensure crawl access before you flip that switch.
And if you’re staring at a massive site audit right now, feeling the overwhelm of thousands of URLs and conflicting directives, this is where a tool like OptiLink can step in. I’ve spent years building and refining OptiLink specifically to handle the granular task of managing robots directives at scale. It automates the audit, flags the pages where noindex and nofollow are misapplied, and helps you prioritize the fixes so you can get back to strategy rather than digging through code. It’s not a magic wand—nothing replaces a human audit—but it handles the repetitive heavy lifting so you can make the strategic calls with confidence.
faq: - “What’s the difference between noindex and nofollow in plain English?|noindex tells Google not to show the page in search results; nofollow tells Google not to pass PageRank through the links on that page. They operate on different layers: one controls visibility, the other controls authority flow.” - “Can I use both noindex and nofollow on the same page?|Yes. Noindex blocks the page from appearing in SERPs while nofollow stops link juice from flowing out of the page’s outbound links. They don’t cancel each other out; they solve separate problems.” - “Will nofollow stop a page from appearing in Google results?|Not necessarily. Google may still discover and index a page even if nofollow is set, especially if other sites link to it. If you need the page completely invisible, noindex is the stronger signal.” - “Do I need to edit HTML or can I set these via CMS?|Most modern CMS platforms (WordPress, Webflow, Shopify) offer UI toggles or plugins to set noindex/nofollow without touching code. If you’re doing bulk redirects or legacy site migrations, HTML edits may still be required.” - “How long until Google respects noindex/nofollow changes?|Google typically recrawls and reinterprets the tags within a few days to a couple of weeks. Submitting the URL via Search Console acceleration can speed up the process, but patience is required for the index to fully update.”
Frequently asked questions
What’s the difference between noindex and nofollow in plain English?
noindex tells Google not to show the page in search results; nofollow tells Google not to pass PageRank through the links on that page. They operate on different layers: one controls visibility, the other controls authority flow.
Can I use both noindex and nofollow on the same page?
Yes. Noindex blocks the page from appearing in SERPs while nofollow stops link juice from flowing out of the page’s outbound links. They don’t cancel each other out; they solve separate problems.
Will nofollow stop a page from appearing in Google results?
Not necessarily. Google may still discover and index a page even if nofollow is set, especially if other sites link to it. If you need the page completely invisible, noindex is the stronger signal.
Do I need to edit HTML or can I set these via CMS?
Most modern CMS platforms (WordPress, Webflow, Shopify) offer UI toggles or plugins to set noindex/nofollow without touching code. If you’re doing bulk redirects or legacy site migrations, HTML edits may still be required.
How long until Google respects noindex/nofollow changes?
Google typically recrawls and reinterprets the tags within a few days to a couple of weeks. Submitting the URL via Search Console acceleration can speed up the process, but patience is required for the index to fully update.
Frequently asked questions
How do I check my anchor text distribution?
Open Google Search Console, go to Links to your site, and click the 'Anchor text' report. It shows your top anchors and their share of the profile, though it caps at 1,000 links.
What is a healthy anchor text distribution?
Most sites we audit fall into a 50-70% branded/natural mix with exact match anchors under 10%. If your exact match ratio creeps past 15%, it’s a red flag for over-optimization.
Can too much exact match anchor text hurt rankings?
Yes. Google’s Penguin and manual spam filters flag profiles where exact match anchors exceed 15-20%. The safest approach is to keep them under 10% and rely on brand, URL, and related variations for the bulk of your links.
How often should I audit anchor text?
We recommend a light audit after every major link insertion campaign and a full deep-dive quarterly. If you’ve just recovered from a penalty, check monthly until the profile stabilizes.