Shopify Robots.txt: How to Edit & Optimize for SEO

Key Insights

• Shopify auto-generates a working robots.txt for every store. Editing it is optional, and Shopify Support explicitly won’t help troubleshoot custom edits if something breaks.
• Since June 2021, stores can customize the file by adding a robots.txt.liquid template to the active theme, giving full control over crawl rules.
• The default file already blocks admin, checkout, cart, policy pages, and standard internal search. Most stores never actually need to touch it.
• Crawling and indexing are different processes. Blocking a page from being crawled doesn’t guarantee it won’t be indexed if Google finds it linked elsewhere.

A Shopify robots txt file tells search engine crawlers which parts of a store they’re allowed to visit. Every Shopify store ships with one already configured sensibly by default, which is exactly why most of what gets written about editing it overstates how often a merchant actually needs to. This guide covers what the default file already handles, the specific situations where a custom edit genuinely helps, and how to make changes safely without accidentally blocking the pages meant to rank, since a single misplaced rule can quietly remove an entire product category from search results.

What Shopify’s Default Robots.txt Already Does

Before touching anything, it’s worth knowing what’s already covered. Shopify’s default robots.txt is generated dynamically through the Liquid robots object and updated on Shopify’s side to reflect current SEO best practices, without any merchant intervention required:

For the majority of stores running a standard theme with no custom internal search implementation, this default configuration is complete. There’s genuinely nothing to add, and spending time customizing a file that’s already correct is time better spent elsewhere in a store’s technical SEO.

The Crawling and Indexing Distinction That Trips People Up

Before deciding whether to edit anything, it’s worth understanding a distinction that causes most robots.txt confusion. Crawling is when a search engine bot visits a page and reads its content. Indexing is when that page becomes eligible to actually appear in search results. These aren’t the same process, and they aren’t mutually dependent: a page can be indexed without ever being crawled if Google discovers it through an external link and has enough context to include it anyway, often showing up with no description at all since the content itself was never actually read.

When You Actually Need to Edit It

A custom robots.txt.liquid template earns its place in a handful of specific situations, not as a routine SEO task. The most common trigger is a third-party app changing how internal search works: Shopify’s default rules block its own standard search results path, but if an installed app renders search at a different URL, like /pages/search instead of the default, those results are no longer covered and can end up crawled and indexed as thin, duplicate content. Multi-domain stores using Shopify Markets are another legitimate case, since 2026 added support for domain-specific crawl rules through the request. host object, useful when different markets need different indexing behavior.

A third, less common but real scenario is blocking specific AI crawlers by name. As generative search tools have grown, some merchants choose to disallow specific bots from training on their product descriptions and images while still allowing traditional search engines full access, a distinction the default file doesn’t make on its own since it wasn’t written with AI crawlers in mind.

How to Add and Edit robots.txt.liquid

Customizing the file requires adding a new template to the active theme’s code. The process is straightforward, but it’s worth doing in a duplicate or development theme first rather than directly on the live storefront, since a syntax error in this file can affect how every crawler sees the entire store until it’s fixed:

The template supports a defined set of Liquid objects for outputting the default rules, and it’s strongly recommended to build on top of those objects rather than replacing the whole file with static plain text. Doing so means the custom file still benefits from Shopify’s ongoing default-rule updates instead of freezing at whatever was correct on the day it was written.

See how crawl changes affect your indexed pages  Explore DataDrop

The Mistake That Costs the Most

Removing a default rule is technically possible and rarely a good idea. The /policies/ block, for example, exists because legal boilerplate pages add no search value and can dilute a store’s perceived content quality if indexed at scale. Merchants sometimes remove default blocks trying to fix an unrelated visibility problem, not realizing the rule they’re deleting was protecting something else entirely.

The bigger mistake is treating robots.txt as a solution to indexing problems it can’t actually fix. Blocking crawling stops Googlebot from visiting a page, but it doesn’t stop that page from being indexed if it’s discovered another way, through an external link, for instance. A page can show up in results with no description at all, just a bare URL, precisely because Google knows it exists but was never allowed to crawl it and see what’s there. For pages that genuinely shouldn’t appear in search results, a noindex tag is the correct tool, not a robots.txt block.

A Shopify robots txt file is one of the rare pieces of technical SEO where the default setup is usually correct, and the main risk lies in editing something that didn’t need to be touched in the first place. Check the default behavior against the actual problem before writing a single custom rule, and when a genuine gap does exist, build the fix on top of Shopify’s Liquid objects rather than replacing them outright, so future default improvements still apply automatically.

Robots.txt controls crawling, but duplicate content issues often need a canonical tag fix instead. Our Shopify canonical URL guide covers when to reach for which tool.

Want a technical crawl audit of your store?  Book a Consult

Pro Tip 💡

Before editing robots.txt, check whether the problem you’re trying to solve is actually a crawling issue or an indexing issue. If a page is already indexed and showing up in results you don’t want, robots.txt won’t remove it; a noindex meta tag will, since Google has to be able to crawl a page to see and obey a noindex instruction.

FAQ

1. Do I Need to Edit My Shopify Robots.txt File?

Most stores don’t. Shopify’s default file already blocks admin, checkout, cart, and policy pages while allowing products, collections, and blogs to be crawled without restriction. Editing is only necessary for specific situations like custom internal search paths or multi-domain crawl rules.

2. How Do I Add a Custom robots.txt Liquid File?

In the theme code editor, locate the Templates folder, right-click to create a new file, and name it exactly robots.txt.liquid. It must use that exact filename and can’t be a JSON template.

3. Will Shopify Support Help If I Break My Robots.txt File?

No. Editing robots.txt.liquid is explicitly an unsupported customization. Shopify Support can’t assist with issues arising from custom edits, so changes should be tested carefully or done by a developer familiar with the syntax.

4. Can I Block AI Crawlers Like ChatGPT’s Bot Using Robots.txt?

Yes, this is one of the more common reasons merchants customize the file in 2026. A specific user-agent group can be added to disallow AI crawlers by name, separate from the rules governing traditional search engine bots.

5. What’s the Difference Between Robots.txt and a Noindex Tag?

Robots.txt controls whether a page gets crawled at all. A noindex tag tells a search engine, after crawling, not to include a page in results. Blocking a page in robots.txt can actually prevent Google from ever seeing a noindex tag placed on that same page.

6. Does Removing a Default Robots.txt Rule Hurt SEO?

It can. Default rules like the /policies/ block exist to keep low-value pages out of the index. Removing them without understanding why they were there in the first place risks exposing thin content that dilutes a store’s overall search quality signals.