Bot name
Last detected
IPs known
admantx-eusy01/3.2
2025-10-19
1
admantx-ussy04/3.2
2024-11-28
13
AdsBot-Google
2026-06-14
112
AdsBot-Google-Mobile
2025-10-24
7
AdsTxtCrawlerTP/1.2
2025-05-07
1
AhrefsBot/7.0
2026-06-24
6116
aiHitBot/2.9
2025-10-07
2
alexa site audit/1.0
2026-06-08
11
AliyunSecBot
2026-05-31
273
Amazon-Advertising-ad-standards-bot/1.0
2026-06-24
4
AmazonAdBot/1.0
2026-01-04
23
Amazonbot/0.1
2026-06-24
922
AnyEvent-HTTP/2.22
2025-03-03
1
AppEngine-Google
2026-03-08
103
Applebot/0.1
2026-06-24
2317
archive.org_bot
2026-06-15
30
AwarioBot/1.0
2026-06-22
13
AwarioSmartBot/1.0
2026-06-23
1
axios/0.21.4
2023-12-08
3
axios/0.27.2
2024-05-28
7
Baiduspider
2026-06-11
23
Baiduspider-render/2.0
2026-06-24
429
Baiduspider/2.0
2026-06-24
660
Barkrowler/0.9
2026-06-23
153
bidswitchbot/1.0
2026-05-16
240
Bidtellect/0.0.0.0
2026-06-23
2
bingbot/2.0
2026-06-24
1833
BingPreview/1.0b
2026-06-15
88
BitSightBot/1.0
2026-06-23
581
BLEXBot/1.0
2025-12-03
45
Bloglines/3.1
2026-06-11
11
Bravebot/1.0
2026-06-24
17
Brightbot 1.0
2026-02-12
3
Bytespider
2026-06-24
17384
CATExplorador/1.0beta
2026-03-20
6
CensysInspect/1.1
2026-06-24
1173
Centro Ads.txt Crawler/1.0
2026-06-23
16
centurybot/1.0
2025-01-08
2
ChatGLM-Spider/1.0
2024-08-07
2
ChatGPT-User/1.0
2026-06-24
1827
CheckMarkNetwork/1.0
2026-06-23
85
Claude-User/1.0
2026-06-24
71
ClaudeBot
2026-06-24
5260
ClaudeBot/1.0
2026-06-24
240
coccocbot-web/1.0
2026-06-24
144
Cocolyzebot/1.0
2026-05-18
3
companyspotter/2.0.0.0
2026-05-28
1
ContextAd Bot 1.0
2025-04-24
2
crawler_eb_germany_2.0
2026-05-21
10
Crawlson/1.0
2024-07-25
1
CriteoBot/0.1
2026-06-24
164
DataForSeoBot/1.0
2026-06-24
22
Dataprovider.com
2026-06-24
289
Discordbot/2.0
2026-05-19
3
DnBCrawler-Analytics
2026-06-15
104
DomainCrawler/1.0
2025-08-15
1
Domains Project/1.3.7
2024-10-06
2
DomainStatsBot/1.0
2026-06-13
5
Dormouse/1.0
2024-12-01
2
DuckAssistBot/1.1
2025-11-22
7
DuckDuckBot-Https/1.1
2026-04-04
29
DuckDuckBot/1.0
2025-04-06
11
DuckDuckBot/1.1
2026-06-24
36
DuckDuckGo-Favicons-Bot/1.0
2025-03-07
12
EmailWolf 1.00
2026-06-14
12
ev-crawler/1.0
2026-06-21
6
everyfeed-spider/2.0
2026-06-12
9
Example3/1.0
2026-05-24
42
facebookexternalhit/1.0
2025-12-17
5
facebookexternalhit/1.1
2026-06-24
5065
facebookscraper/1.0
2026-06-14
22
FAST-WebCrawler/3.8
2026-06-15
10
Feedfetcher-Google
2026-06-14
14
FriendlyCrawler/1.0
2024-01-11
1
Gabanzabot/1.1
2024-09-12
1
Gaisbot/3.0
2026-06-11
13
GeedoProductSearch
2025-12-03
27
GenomeCrawlerd/1.0
2026-01-28
132
Google-adstxt
2026-06-24
257
Google-Adwords-Instant
2025-03-03
2
Google-Apps-Script
2025-01-02
2
Google-Display-Ads-Bot
2026-06-24
74
Google-PageRenderer
2024-10-06
2
Google-Read-Aloud
2026-06-24
407
Google-Safety
2025-07-04
8
Googlebot-Image/1.0
2026-06-23
593
Googlebot-Mobile/2.1
2026-06-15
37
Googlebot-News
2026-06-15
16
Googlebot-Video/1.0
2026-06-10
9
Googlebot/2.1
2026-06-24
11043
GoogleOther
2026-06-22
952
GPTBot/1.0
2026-04-20
173
GPTBot/1.2
2025-11-10
945
Grammarly/1.0
2025-05-19
8
GrapeshotCrawler/2.0
2024-10-28
16
Gregarius/0.5.2
2026-06-22
13
Gulper Web Bot 0.2.4
2026-06-19
17
heritrix/3.3.0
2024-08-10
1
heritrix/3.4.0
2024-11-07
1
hstspreload-bot
2026-06-22
227
HTTP Banner Detection
2026-06-24
3
HuaweiCrawler
2026-05-27
12
IAS crawler
2026-06-22
1320
ias-or/3.3
2026-06-19
1457
ias-sync/3.3
2026-06-21
218
ias-va/3.3
2026-06-23
1452
iaskspider/1.0
2026-05-13
25
iaskspider/2.0
2026-02-26
30
Iframely/1.3.1
2026-05-27
42
ImagesiftBot
2025-07-24
249
INETDEX-BOT/2.0
2024-11-26
1
intelx.io_bot
2026-06-14
2
InternetMeasurement/1.0
2026-06-22
1074
keys-so-bot
2026-06-19
209
KhazandeBot/V2.6.2
2025-10-07
1
KStandBot/1.0
2026-05-13
1
KwHunter/2.0
2025-10-14
1
Leikibot/1.0
2025-10-20
7
LightspeedSystemsCrawler
2025-09-28
2
Linguee Bot
2026-01-18
16
LinkBloom/2.7.11
2025-10-15
6
LinkBloom/2.7.12
2025-10-29
2
LinkupBot/1.0
2026-06-23
5
Mail.RU_Bot/2.0
2025-03-17
19
Mail.RU_Bot/Favicons/2.0
2025-08-30
14
Mail.RU_Bot/Robots/2.0
2025-08-30
22
Maillocals Bot
2024-09-13
1
MarketingMiner v2.0
2024-07-06
1
MBCrawler/1.0
2026-06-24
2
MediaMathbot/1.0
2024-09-26
2
Mediapartners-Google
2026-06-24
3961
MegaIndex.ru/2.0
2024-11-30
3
meta-externalads/1.1
2025-10-20
1
meta-externalagent/1.1
2026-06-24
1369
meta-webindexer/1.1
2026-06-23
487
Microsoft.Data.Mashup
2025-10-29
2
MicrosoftPreview/2.0
2025-05-05
9
MiniCrawler/1.0
2025-10-26
1
MistralAI-User/1.0
2026-05-27
7
MJ12bot/v1.4.8
2026-06-24
898
ModatScanner/1.0
2025-07-24
8
MojeekBot/0.11
2026-06-20
1
MS Search 6.0 Robot
2025-09-27
2
MsnBot-Media/1.0b
2025-12-17
2
msnbot-media/1.1
2026-06-18
10
msnbot/0.11
2026-06-11
11
MyCrawler/1.0
2024-11-23
2
NetcraftSurveyAgent/1.0
2026-06-12
777
netEstate NE Crawler
2025-07-17
1
Nicecrawler/1.1
2026-04-08
12
node-fetch/1.0
2026-04-22
13
Nutch-1.21-SNAPSHOT
2025-01-06
5
OAI-SearchBot/1.0
2026-06-24
944
Pandalytics/2.0
2026-06-12
92
peer39_crawler/1.0
2026-06-19
26
PerplexityBot/1.0
2026-06-24
513
Pinterestbot/1.0
2026-04-23
6
Pixalate.com/1.0
2026-06-11
44
probe-image-size/7.2.3
2026-04-22
10
Project-Resonance
2026-06-09
9
Protopage/3.0
2026-04-02
1
Quantcastbot/2.0
2026-04-16
1
Qwantbot-prod/1.0
2025-01-15
13
Qwantbot-prod177007/1.0
2024-11-06
1
Qwantbot-prod19251/1.0
2024-08-12
1
Qwantbot-prod38503/1.0
2024-07-17
1
Qwantbot-prod577007/1.0
2025-01-15
9
Qwantbot-prod77007/1.0
2024-11-18
7
Qwantbot/1.0
2026-06-24
1614
Qwantbot/1.0_577007
2025-03-31
10
Qwantify-prod/1.0
2024-06-25
3
Qwantify-prod77007/1.0
2023-11-29
2
RecordedFuture Global Inventory Crawler
2026-06-23
6
RedekenBot/0.1
2024-04-28
1
RepoLookoutBot/v1.1.0
2024-10-04
3
robots-harvester/0.1
2025-10-30
1
S-Scanner/1.0
2025-11-17
3
ScraperBot/1.0
2026-04-06
41
Scrapy/1.7.4
2025-04-09
2
Scrapy/2.11.0
2025-12-04
2
Scrapy/2.11.2
2026-02-02
50
Scrapy/2.6.3
2025-05-19
2
Screaming Frog SEO Spider/17.1
2024-12-27
1
Screaming Frog SEO Spider/19.8
2024-04-30
1
Screaming Frog SEO Spider/20.2
2025-05-02
20
Screaming Frog SEO Spider/20.3
2024-12-19
2
Screaming Frog SEO Spider/20.4
2024-11-14
1
Screaming Frog SEO Spider/21.3
2025-02-06
8
Screaming Frog SEO Spider/8.1
2026-06-15
18
search-engine-indexer/1.0
2025-08-21
4
SeekportBot
2026-05-27
75
SemanticScholarBot
2024-03-12
1
SemrushBot-BA
2026-05-27
20
SemrushBot/6~bl
2026-01-23
1
SemrushBot/7~bl
2026-06-24
89
SeobilityBot
2026-03-24
2
serpstatbot/2.1
2026-06-02
28
SeznamBot/4.0
2026-06-24
118
Sidetrade indexer bot
2026-06-22
1
site-checker/1.0
2025-09-26
3
SiteCardBot/1.0
2025-10-23
1
SiteCheckerBotCrawler/1.0
2024-11-02
1
SiteOne-Crawler/1.0.7
2024-11-25
1
Slackbot 1.0
2026-04-09
22
Slackbot-LinkExpanding 1.0
2026-04-09
25
Snap URL Preview Service
2026-06-19
206
Snap-URL-Preview
2024-05-28
2
SnoopSecInspect/1.1
2025-01-14
1
Sogou Push Spider/3.0
2026-01-21
29
Sogou web spider/3.0
2026-05-15
31
Sogou web spider/4.0
2026-06-24
34
startmebot/1.0
2026-06-22
445
StartpagePrivateImageProxy/2.0
2025-01-08
1
StractBot/0.2
2024-08-31
2
SuperBot/4.4.0.60
2026-06-15
19
SurdotlyBot/1.0
2026-06-20
11
t3versionsBot/1.0
2025-09-14
6
TenMillionDomainsBot/1.0
2024-10-27
1
Timpibot/0.8
2025-03-24
11
Timpibot/0.9
2025-03-24
10
TinEye-bot/1.31
2024-11-30
1
trafilatura/1.11.0
2024-07-02
1
Twitterbot/1.0
2026-06-15
8
URLSuMaBot / 1.0
2024-08-05
1
VelenPublicWebCrawler/1.0
2026-06-14
53
VsuSearchSpider/1.0
2025-02-27
1
vuhuvRBT/2.0
2025-04-07
1
weborama-fetcher
2024-06-25
1
Website-info.net-Robot
2026-05-10
28
Websuse.de/1.0
2025-10-17
1
WebwikiBot/2.1
2025-09-14
2
WellKnownBot/0.1
2026-06-23
80
wordupsearchengine-1
2024-06-12
1
XaxisSemanticsClassifier/1.0
2025-10-28
1
Yahoo Ad monitoring
2024-10-20
2
Yahoo! Slurp
2026-06-08
66
Yahoo! Slurp China
2026-06-15
45
YandexBot/3.0
2026-06-24
1719
YandexFavicons/1.0
2026-06-22
282
YandexImages/3.0
2026-06-05
181
YandexMobileBot/3.0
2025-02-18
2
YandexNews/4.0
2026-06-15
15
YisouSpider
2026-06-16
39
YodaoBot/1.0
2026-05-15
27
ZaldamoSearchBot
2025-11-25
3
zgrab/0.x
2026-06-24
1879
ZoominfoBot
2026-06-11
208
