Who's knocking?
AI agents read your website every day: crawlers training models, AI search engines, and assistants fetching pages for someone who asked about you. Some of them are not who they say they are.
-
GET /pricing GPTBot/1.4 20.171.206.12verified IP in OpenAI's published ranges -
GET /docs/api ChatGPT agent signedverified Web Bot Auth signature from chatgpt.com -
GET /wp-login.php GPTBot/1.4 203.0.113.54spoofed Claims GPTBot, IP is not OpenAI's -
GET /blog/agents Googlebot/2.1 66.249.66.1verified Address resolves to googlebot.com and back -
GET / python-requests/2.32 198.51.100.7unverified Script with no published identity -
GET /pricing Chrome 141 referer chatgpt.comperson Clicked a link in a ChatGPT answer
What to fix
Who your robots.txt lets in
Checked at the root of your site for every agent we know, grouped by what the agent does.
Measure real agent visits
Create a free project, send us your traffic, and get a report of every agent that visited: who it was, whether it was verified, which pages it read, and how many people arrived from AI assistants.
Your keys
Save both now. They are shown only once and we cannot recover them.
- Send key
Only sends visits. Safe to put in your site's code.- Report key
Opens your report. Keep it private.
Connect your site
If your site runs through Cloudflare, this small Worker reports every agent in real time. It passes visitors to your site unchanged.
Works on any host, including cPanel shared hosting, and brings in your history. No code on your site.
Give this to Claude, ChatGPT or any agent with MCP. It can import your logs and read your report for you.
How we tell a real agent from an impostor
Anyone can write "GPTBot" in a request. We only call an agent verified when we can prove it, and only call it spoofed when we can prove the opposite. Strongest proof first:
-
Cryptographic signature
Agents such as ChatGPT agent sign their requests (Web Bot Auth). We check the signature against the operator's own published keys.
-
Published IP ranges
OpenAI, Anthropic, Perplexity, Google, Apple and Microsoft publish the addresses their agents use. A request claiming their name from elsewhere is spoofed.
-
Reverse DNS
For search crawlers that document it, the address must resolve to the operator's domain and back.
- verified
- Proven to be the agent it claims.
- unverified
- Claims an identity we cannot check either way.
- spoofed
- Proven not to be the agent it claims.
For AI agents
Knocklog is an MCP server. An agent can check a site, create a project for its user, import logs and read the report without a human filling in forms.
MCP endpoint https://api.knocklog.ai/mcp
Add it to Claude Code:
claude mcp add --transport http knocklog https://api.knocklog.ai/mcpTools: scan_site, classify_request, list_known_agents, create_project, get_install_plan, import_logs, get_agent_traffic.
Plain-text summary for agents: /llms.txt