Detect Known AI Agents and Crawlers with ExtractIP /agent API

Do you want to instantly know if an incoming connection comes from a real user or from an automated agent such as a web crawler or a popular AI assistant? The /agent API from ExtractIP gives you exactly that: it tells whether an IP belongs to a known agent (crawler, bot, or AI agent) and provides its service name.

This lightweight API is perfect for developers, analysts, and security teams who need clear visibility into the sources of their traffic.

API Response

When you query the /agent endpoint, you get a simple JSON response:

{
    "is_agent": false,
    "service_name": ""
}

• is_agent — true if the IP belongs to a recognized agent/crawler/AI assistant, otherwise false.
• service_name — the name of the detected service (e.g., Googlebot, Bingbot, OpenAI-Agent, Perplexity). Empty if no match is found.

What the API Can Detect

The /agent API identifies:

• Popular web crawlers (Googlebot, Bingbot, etc.)
• Commercial scrapers and indexers
• Known AI agents / assistants and their proxy services
• Automated monitoring and scanning tools

The database is updated regularly as new agents emerge.

Why It Matters

• Bot Filtering Keep unwanted crawlers and scrapers from overloading your servers.

• Access Policies Serve different content or apply rate limits to agents vs. real users.

• Security Detect aggressive crawlers, scanners, and automated tools that may probe your system.

• Analytics Accuracy Distinguish human traffic from automated traffic for cleaner data insights.

How to Use the API

CURL

curl "https://api.extractip.com/agent/1.2.3.4"

Or POST:

curl -X POST "https://api.extractip.com/agent" \
  -H "Content-Type: application/json" \
  -d '{"ip":"1.2.3.4"}'

JavaScript (fetch)

const ip = "1.2.3.4";
const res = await fetch(`https://api.extractip.com/agent/${ip}`);
const data = await res.json();
if (data.is_agent) {
  console.log("This is an agent:", data.service_name);
} else {
  console.log("Likely a real user");
}

Python (requests)

import requests

ip = "1.2.3.4"
url = f"https://api.extractip.com/agent/{ip}"

response = requests.get(url)
data = response.json()

if data["is_agent"]:
    print(f"This is an agent: {data['service_name']}")
else:
    print("Likely a real user")

Or using POST:

import requests

url = "https://api.extractip.com/agent"
payload = {"ip": "1.2.3.4"}

response = requests.post(url, json=payload)
data = response.json()

print(data)

Example Use Cases

• is_agent: true, service_name: "Googlebot" → allow indexing but reduce log noise.
• is_agent: true, service_name: "UnknownCrawler" → apply soft rate limits and collect additional headers.
• is_agent: false → treat as a normal user unless other risk signals appear.

Integration Ideas

1. Edge/CDN — apply caching or blocking policies at the edge based on agent detection.
2. API Gateway — throttle or reroute traffic depending on whether it’s from a crawler or a real user.
3. Logging & Analytics — mark records with agent=true for accurate reporting.
4. Fraud Prevention — use as one factor in your risk-scoring system.

Best Practices

• Combine /agent results with other signals (ASN, proxy, privacy checks) for robust decision-making.
• Use is_agent as an indicator, not as the only decision point.
• Update your traffic policies periodically — the list of known agents evolves quickly.

Get Started Today

Send an IP to:

https://api.extractip.com/agent/{ip_address}

or:

https://api.extractip.com/agent

The API returns clear JSON data you can use right away in your workflows. Try it for free at ExtractIP and start filtering out known AI agents and crawlers today!