Install
Analyze a log export (free, offline)
Before installing anything: how much of your site's traffic is AI agents, from logs you already have. One command runs Punchline's classifier over a log export on your own machine, with no account and no network, and prints a report. Nothing from the export is kept or sent anywhere.
npx @punchline-analytics/analyze access.log --host example.com
It's also a Claude Code skill: npx @punchline-analytics/analyze --install-skill writes it into the current project, and then "analyze my AI agent traffic" in Claude Code runs the tool and explains the result for you.
The package is MIT-licensed on npm: @punchline-analytics/analyze.
Logs it reads
The format is detected from the file.
| From | What to export |
|---|---|
| Vercel | A log drain export or saved batch: JSON array or NDJSON with a proxy object per request. Runtime Logs in the dashboard can be downloaded as JSON. |
| Cloudflare | Logpush "HTTP requests", NDJSON (ClientRequestPath, ClientRequestUserAgent, ClientIP, …). Logpush to R2 works on free plans. |
| Apache, Nginx | The access log in combined format. Pass --host example.com, since the lines don't carry it. |
| Anything else | CSV or TSV with a header row naming the address, user agent and path or URL; host, method, status, referrer and time are used when present. Columns are matched by common names. |
A few thousand lines show the picture; a week is better. Several files can be passed at once, and - reads standard input.
What the report says
- Who visited. People, people sent by an AI assistant, verified or declared AI agents, crawlers, fetchers and scripts, suspected automation, and impostors, with requests and distinct address-and-user-agent pairs.
- Operators. Who runs the agents, how many of their requests were confirmed against the operator's published address lists, which agents were seen, and why they fetch: training, search index, fetching for a person, or acting for one.
- Impostors. Requests that used a known agent's name from an address its operator doesn't publish. Scrapers borrow names like GPTBot to get past bot rules; the report keeps them apart so the real operator isn't blamed or blocked.
- People sent by AI assistants, by assistant, from the referrer and
utm_source. - What agents read, by path, with who read it most.
- Worth doing. Whether agents asked for
/llms.txtor/robots.txt, whether any asked for markdown instead of HTML, and whether verified assistants got403or429from a bot rule. - By day, people against agents and crawlers.
--json prints the counts instead, for a script or an assistant. --rdns also confirms operator claims by reverse DNS, which is slower and needs the network.
What logs can't show
Logs carry the address, user agent, path, referrer and status, and that's enough for most of the rules. They don't carry Web Bot Auth signatures (rule R1), the headers every browser sends (R15), or anything from inside the page, and requests aren't grouped into sessions. So the report is a floor on agent traffic and can't say what happened after a fetch. A live install does all of that, on every request: pick one under Get started.
As a library
import { analyze, parseLogs, renderReport } from "@punchline-analytics/analyze";
const { requests, skipped } = parseLogs(text, { host: "example.com" });
const result = await analyze(requests, { skipped });
console.log(renderReport(result));
The classifier, registry and address snapshots are bundled at the version published; the registry grows with each release.
Ask the docs
Answers come from these docs only, written by Claude (Anthropic). Your question is sent to answer it, and isn't stored.