# AI2Bot

> AI2Bot is Ai2 (Allen Institute for AI)'s crawler for collecting pages that may be used to train AI models. Here's how to recognize it and check a request really is it.

## AI2Bot facts

- **Operator:** Ai2 (Allen Institute for AI)
- **Type:** Crawler
- **Why it fetches:** Collects training data
- **User agent contains:** `AI2Bot`, `Ai2Bot-Dolma`
- **Verified by:** not verifiable
- **Operator's documentation:** [allenai.org/crawler](https://allenai.org/crawler)

## Allow or block AI2Bot

Your call. Blocking it keeps new pages out of the training data this crawler collects. Operators often answer live questions with a different agent (see the others from Ai2 (Allen Institute for AI) below), so you can block one and keep the other.

To ask it to stay out, add this to robots.txt: `User-agent: AI2Bot` then `Disallow: /`. Whether it follows robots.txt is Ai2 (Allen Institute for AI)'s call; [its documentation](https://allenai.org/crawler) says.

## How to tell a real AI2Bot from an impostor

Anyone can put "AI2Bot" in a user agent. What can't be faked is below.

- Its operator doesn't publish a way to verify it, so a request claiming to be it can only be treated as a claim.

## How Punchline counts AI2Bot

Punchline labels a request that names AI2Bot with how sure it is: **verified** or **declared** when the evidence checks out, a lower tier when it can't be checked, and **impostor** when the evidence contradicts it. Each visit is counted under Ai2 (Allen Institute for AI), with its purpose, so you can see whether AI2Bot is reading your site and which pages it reads. [How it works](https://punchlineanalytics.com/how-it-works) · [Known agents](https://punchlineanalytics.com/docs/agents)

[See which agents read your site](https://app.punchlineanalytics.com/signup)
