meta-externalfetcher
meta-externalfetcher fetches pages when a person using Meta's assistant asks something that needs them, so a visit means someone is asking about the page right now.
Should you let it in?
Allow or block meta-externalfetcher
Usually, yes. Each visit is a person asking Meta's assistant something your page answers, right now. Blocking it means the answer is written without your page.
To ask it to stay out, add this to robots.txt. Whether it follows robots.txt is Meta's call; its documentation says.
User-agent: meta-externalfetcher
Disallow: /Is it really?
How to tell a real meta-externalfetcher from an impostor
Anyone can put “meta-externalfetcher” in a user agent. What can't be faked:
- Its operator doesn't publish a way to verify it, so a request claiming to be it can only be treated as a claim.
In Punchline
How Punchline counts meta-externalfetcher
Punchline labels a request that names meta-externalfetcher with how sure it is: verified or declared when the evidence checks out, a lower tier when it can't be checked, and impostor when the evidence contradicts it. Each visit is counted under Meta, with its purpose, so you can see whether meta-externalfetcher is bringing people's questions to your pages and which pages it reads. How it works · Known agents