how to block Meta AI User Agent

How To Block Meta AI From Crawling Your Site? What Is Meta AI User Agent?

Last update on:

Meta AI has recently been widely discussed, with Mark Zuckerberg’s video announcing its latest release and features. Now, with many sites blocking AI crawlers such as GPTBot, Google-Extended, and Anthropic-AI (see the universal web crawler blocking report), especially in the News sector, you might be wondering how you can block Meta AI from accessing your site.

I did some research and was not able to find any official source from Facebook on the Meta AI user agent. There is a page on the Facebook developer’s guide with information about the crawlers that Facebook uses to browse a website when a link is shared between users:

  • facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
  • facebook external hit/1.1
  • facebookcatalog/1.0

However, these seem to be specific to Facebook only, and we don’t know if Meta AI uses the same User Agent.

So I went ahead and asked Meta AI itself about how we can block it from accessing a site via robots.txt, and this is the response:

How To Block Meta AI From Crawling Your Site? What Is Meta AI User Agent? 1
Asking Meta AI itself about its user agent

So, to block these user agents, you can add these lines to your robots.txt file:

User-agent: MetaAI
User-agent: Meta-AI
User-agent: Meta AI
User-agent: Meta.AI
User-agent: meta-ai
User-agent: metaai
Disallow: /

I then asked Meta AI if it could provide me with the source of this information, but none of the sources it provided mentioned anything about how to block Meta AI or what the Meta AI user agent is.

How To Block Meta AI From Crawling Your Site? What Is Meta AI User Agent? 2
None of the sources Meta AI provides mention anything about Meta AI User Agent

So I assume that what the Meta AI said about its user agent is just the AI making up things, doing guesswork, or related to AI hallucination, as it is more commonly referred to.

So, I went ahead and contacted Facebook about this, but I have not yet heard back from them. I will update this post as soon as I hear back from them (I will update this post as soon as I receive a reply.)

How To Block Meta AI From Crawling Your Site? What Is Meta AI User Agent? 3
What’s your Reaction?
+1
0
+1
0
+1
0
+1
0
+1
0
+1
0
+1
0

Author

  • Saeed Khosravi

    Saeed Khosravi is an SEO Strategist, Digital Marketer, and WordPress Expert with over 15 years of experience, starting his career in 2008. He graduated with a degree in MIB Marketing from HEC Montreal. As the Founder and CEO of Nexunom, Saeed, alongside his dedicated team, provides comprehensive digital marketing solutions to local businesses. He is also the founder and the main brain behind several successful marketing SAAS platforms, including Allintitle.co, ReviewTool.com, and Tavata.com.

    https://www.linkedin.com/in/saeedkhosravi/ [email protected] Khosravi Saeed
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments
Saeed Khosravi