At a glance
- Which Meta crawler does whatmeta-externalagent: collects pages for training language models.
- Edit robots.txt on ShopifyIn Online Store, Themes, Edit code, add a new template and choose robots.
- If they keep comingrobots.txt is a request, not a wall.
Which Meta crawler does what
- meta-externalagent: collects pages for training language models. Meta says it follows robots.txt. This is the usual one to block.
- meta-externalfetcher: fetches a page when a person asks Meta's assistant about it. It may not follow robots.txt, because a person asked for it.
- facebookexternalhit: builds link previews when your pages are shared. Blocking it breaks previews and can hurt ads that link to your store.
Edit robots.txt on Shopify
- In Online Store, Themes, Edit code, add a new template and choose robots. Shopify creates robots.txt.liquid with its default rules.
- Keep the default rules, and add a block for the crawler you want to stop: a User-agent line for meta-externalagent and a Disallow line for /.
- Save, then open yourstore.com/robots.txt to check the new lines are there.
If they keep coming
- robots.txt is a request, not a wall. Check your traffic reports for the user agent after a few days.
- To block at the network level you need a proxy or firewall in front of the store, such as Cloudflare, which Shopify supports on a custom domain with care. That is a bigger change; test checkout afterwards.
Updated 2026-11-01. General guidance, not advice for your situation; we scope each project in writing before any work starts.
Questions people ask first
Will blocking it hurt my Google ranking?
No. Google's crawlers are separate and the block names only Meta's.
Will it stop my products appearing in Meta's assistant answers?
It stops new pages being collected for training. Answers that fetch pages on request use a different crawler.
Can I undo it?
Yes. Remove the lines from robots.txt.liquid, or delete the template to go back to Shopify's default.
Tell us what you need built.
Tell us the process and the tools involved, or call. We reply with questions or a written scope.
hello@pushbuild.com