Bot name
Last detected
IPs known
admantx-adform/3.1
2022-07-18
1
admantx-eusyncbatch01/3.1
0
admantx-ussy03/3.2
2023-07-14
1
admantx-ussy04/3.2
2023-08-31
17
AdsBot-Google
2023-09-18
3
AdsBot-Google-Mobile
2021-12-17
2
adscanner/1.0
2022-05-11
3
AdsTxtCrawler/1.0
2023-07-12
66
AdsTxtCrawler/1.0.2
2023-09-22
742
AdsTxtCrawlerTP/1.2
2023-09-19
3
AhrefsBot/7.0
2023-09-22
638
aiHitBot/2.9
2023-09-22
3
AlexandriaOrgBot/1.0
2022-02-11
0
Amazon-Advertising-ad-standards-bot/1.0
2023-09-22
2
AmazonAdBot/1.0
2023-09-22
39
Amazonbot/0.1
2023-09-22
3
anthropic-ai
2023-08-21
41
AnyEvent-HTTP/2.22
2023-07-02
1
AppEngine-Google
2023-09-18
23
Applebot/0.1
2023-09-22
2414
aragna.net_bot
2023-04-11
1
archive.org_bot
2023-05-01
34
AwarioBot/1.0
2023-09-12
11
AwarioRssBot/1.0
2023-05-05
1
AwarioSmartBot/1.0
2022-04-23
3
Baiduspider-render/2.0
2023-09-10
466
Baiduspider/2.0
2023-09-21
568
Barkrowler/0.9
2023-09-22
279
bidswitchbot/1.0
2023-09-22
27
Bidtellect/0.0.1107.0
2023-01-09
0
Bidtellect/0.0.1125.0
2023-01-20
1
Bidtellect/0.0.1143.0
2023-02-06
1
Bidtellect/0.0.1157.0
2023-02-16
1
Bidtellect/0.0.1251.0
2023-05-25
1
Bidtellect/0.0.1259.0
2023-06-01
1
Bidtellect/0.0.1265.0
2023-06-25
1
Bidtellect/0.0.1273.0
2023-07-21
1
Bidtellect/0.0.1285.0
2023-08-08
1
Bidtellect/0.0.1290.0
2023-08-16
1
Bidtellect/0.0.1293.0
2023-08-17
1
Bidtellect/0.0.1296.0
2023-08-20
1
Bigsearch.ca/Nutch-1.0-dev
0
bingbot/2.0
2023-09-22
2027
BingPreview/1.0b
2023-08-25
614
Birdcrawlerbot/0.5
2023-07-05
1
bitlybot/3.0
2023-02-03
1
BitSightBot/1.0
2023-09-16
15
BLEXBot/1.0
2023-09-21
36
BlogBeatbot/2.5
2022-02-14
0
Bloglines/3.1
2023-09-01
3
BrightEdge Crawler/1.0
2023-09-13
2
Bytespider
2023-09-22
5297
CATExplorador/1.0beta
2023-04-21
11
CensysInspect/1.1
2023-09-22
255
Centro Ads.txt Crawler/1.0
2023-09-20
2
centuryb.o.t9
2022-09-20
3
certytags/1.0
2023-07-30
1
CheckMarkNetwork/1.0
2023-09-15
33
CloudFlare-AlwaysOnline/1.0
0
coccocbot-image/1.0
2022-07-17
36
coccocbot-web/1.0
2023-05-07
144
Cocolyzebot/1.0
2023-04-13
25
companyspotter/0.0.0.0
2022-09-17
0
companyspotter/0.1984.82.1324
2022-01-04
0
companyspotter/1.0.0.0
2023-05-22
2
crawler_eb_germany_2.0
2023-07-20
7
Crawling at Home Project
0
Crawlson/1.0
2023-01-09
12
CriteoBot/0.1
2023-09-22
189
CyotekWebCopy/1.9
2022-12-09
0
DataForSeoBot/1.0
2023-09-22
10
Dataprovider.com
2023-09-08
217
Discordbot/2.0
2023-06-20
3
DnBCrawler-Analytics
2023-09-20
16
dns-crawler/1.4.10.dev6
0
dns-crawler/1.4.10.dev7
0
Domains Project/1.3.7
2023-01-11
1
DomainStatsBot/1.0
2023-09-04
4
DongleEmulatorBot/0.1
2023-07-25
5
DuckDuckBot-Https/1.1
2023-07-03
14
DuckDuckBot/1.1
2023-09-16
5
DuckDuckGo-Favicons-Bot/1.0
2023-09-22
2
E. Orliac, G. Fourestey/1.0
0
EdgeWatch/1.1
2023-09-16
1
EmailWolf 1.00
2022-08-26
0
ev-crawler/1.0
2023-09-18
7
evc-batch/2.0
2022-02-11
1
Example3/1.0
2023-07-11
6
facebookexternalhit/1.0
2023-01-12
14
facebookexternalhit/1.1
2023-09-22
891
facebookexternalhit/1.2
2022-04-06
0
facebookscraper/1.0
2023-03-07
1
Feedfetcher-Google
2023-06-28
2
feedparser/6.0.10
2023-06-08
1
FemtosearchBot/1.0
2023-07-31
5
FindITAnswersbot/1.0
2023-05-02
1
FullStoryBot/1.0
2022-09-27
0
GigablastOpenSource/1.0
2022-12-21
4
Gluten Free Crawler/1.0
0
go-resty/2.7.0
2023-03-17
4
Google-adstxt
2023-09-22
376
Google-Adwords-Instant
2023-06-25
10
Google-Display-Ads-Bot
2023-09-18
36
Google-PageRenderer
2023-08-19
30
Google-Read-Aloud
2023-09-22
216
Googlebot-Image/1.0
2023-09-22
801
Googlebot-Mobile/2.1
2023-05-01
2
Googlebot-Video/1.0
2023-05-18
2
Googlebot/2.1
2023-09-22
6364
Grammarly/1.0
2023-09-12
4
GrapeshotCrawler/2.0
2023-09-22
16
Graydon bot/1.0
2023-02-06
3
Gregarius/0.5.2
2022-10-26
1
Gulper Web Bot 0.2.4
2023-08-31
3
GumGum-Bot/1.0
2022-03-04
0
heritrix/3.3.0
2023-04-04
1
hstspreload-bot
2023-09-21
8
HTTP Banner Detection
2023-09-14
6
HubSpot Crawler
2022-03-27
0
HubSpot Webcrawler
2022-12-29
6
ias-ie/3.3
2023-09-22
229
ias-ir/3.1
2023-03-23
334
ias-or/3.1
2023-03-23
186
ias-or/3.3
2023-09-22
191
ias-va/3.1
2023-03-23
410
ias-va/3.3
2023-09-22
360
Iframely/1.3.1
2023-08-10
25
Iframely/1.6.1
2022-03-25
0
IndeedBot 1.1
2021-10-24
1
INETDEX-BOT/1.5
2023-03-08
3
InfoTigerBot/1.9
2021-11-28
0
intelx.io_bot
2023-09-22
1
InternetMeasurement/1.0
2023-09-21
403
Keybot Translation-Search-Machine
2022-12-03
3
KomodiaBot/1.0
2022-07-02
10
KStandBot/1.0
2023-08-28
4
LightspeedSystemsCrawler
2023-09-04
5
Linespider/1.1
2023-07-25
39
LinkCityChecker/1.0
2021-11-28
0
linkdexbot/2.0
2023-09-22
5
LinkedInBot/1.0
2023-03-25
8
LivelapBot/0.2
2022-11-21
2
Lkx-TraversalHttpPlugin/0.0.1
2021-11-14
0
looid.com Search Engine/0.1
0
Mail.RU_Bot/2.0
2023-05-26
177
Mail.RU_Bot/Fast/2.0
2023-05-09
1
Mail.RU_Bot/Favicons/2.0
2023-09-10
14
Mail.RU_Bot/Img/2.0
2023-04-10
6
Mail.RU_Bot/Robots/2.0
2023-09-10
31
MarketingMiner v2.0
2022-12-13
0
masscan-ng/1.3
2023-06-22
1
masscan/1.0
2023-09-01
35
masscan/1.3
2023-09-22
114
MBCrawler/1.0
2023-09-21
5
MediaMathbot/1.0
2023-05-13
111
Mediapartners-Google
2023-09-22
501
Mediatoolkitbot
2022-03-07
4
MegaIndex.ru/2.0
2023-09-09
14
Microsoft.Data.Mashup
2023-08-02
2
MicrosoftPreview/2.0
2023-06-30
10
MixnodeCache/1.8
2021-11-22
1
MJ12bot/v1.2.4
2022-05-08
0
MJ12bot/v1.4.1
2022-04-17
0
MJ12bot/v1.4.2
2022-12-15
0
MJ12bot/v1.4.8
2023-09-22
930
MojeekBot/0.10
2022-05-06
1
MojeekBot/0.11
2023-09-22
1
MonitorBacklinksWP
2022-06-27
1
msnbot-media/1.1
2023-07-28
5
MxToolbox/Beta7
2023-07-03
1
My Nutch Spider/Nutch-1.18
2021-10-20
0
Neevabot/1.0
2023-05-31
2
NetcraftSurveyAgent/1.0
2023-09-07
52
netEstate NE Crawler
2023-09-14
1
NetpeakCheckerBot/3.6
2023-08-03
1
NetpeakCheckerBot/3.7
2023-08-20
3
Nicecrawler/1.1
2023-08-06
8
Nimbostratus-Bot/v1.3.2
0
node-fetch/1.0
2023-04-01
15
olbicobot/Nutch-1.15-SNAPSHOT
0
PagePeeker/3.0
2021-09-25
1
Pandalytics/1.0
2023-03-22
48
PartyflockLinkChecker/0.3
0
peer39_crawler/1.0
2023-09-21
23
PhantomBot/1.0
2023-04-15
1
PickyBot/0.6.0
2023-08-04
3
Pinterestbot/1.0
2023-09-16
4
Pixalate-Crawler
2022-03-29
0
Pixalate.com/1.0
2023-09-22
55
pmoz.info ODP link checker
2023-09-16
1
Project-Resonance
2023-03-27
2
Protopage/3.0
2023-09-21
2
PulsePoint-Ads.txt-Crawler/1.1
2023-07-04
10
Quantcastbot/1.0
2023-02-28
1
Quantcastbot/2.0
2023-08-27
1
Qwantify-dev/1.0
2023-09-22
15
Qwantify-dev119/1.0
2023-07-26
1
Qwantify-dev1540/1.0
2023-08-11
2
Qwantify-dev770/1.0
2023-08-10
1
Qwantify/1.0
2023-09-20
84
Qwantify/2.4w
2022-01-14
25
Qwarrybot/2.0
2022-01-14
0
RepoLookoutBot/v1.1.0
2023-05-23
14
ReverseEngineeringBot/0.1
2023-02-20
3
RobotsTxtParser-VIPnytt/2.1
2022-01-12
0
SabsimBot/3.0
2022-02-25
1
Scrapy/1.5.0
2022-09-11
1
Scrapy/1.7.3
2023-01-27
1
Scrapy/1.7.4
2022-10-05
0
Scrapy/2.4.0
2021-12-06
1
Scrapy/2.5.1
2021-11-15
0
Scrapy/2.6.1
2023-03-30
1
Scrapy/2.6.2
2022-09-11
0
Scrapy/2.6.3
2023-09-01
2
Scrapy/2.7.1
2023-02-05
2
Scrapy/2.8.0
2023-04-20
1
Scrapy/2.9.0
2023-07-30
2
Screaming Frog SEO Spider/10.1
0
Screaming Frog SEO Spider/11.2
0
Screaming Frog SEO Spider/11.3
0
Screaming Frog SEO Spider/12.2
0
Screaming Frog SEO Spider/12.3
0
Screaming Frog SEO Spider/12.6
0
Screaming Frog SEO Spider/13.2
0
Screaming Frog SEO Spider/14.1
0
Screaming Frog SEO Spider/14.2
0
Screaming Frog SEO Spider/14.3
0
Screaming Frog SEO Spider/15.2
2023-05-16
1
Screaming Frog SEO Spider/16.2
2021-11-16
0
Screaming Frog SEO Spider/16.3
2021-12-06
0
Screaming Frog SEO Spider/16.4
2021-12-21
0
Screaming Frog SEO Spider/16.5
2022-02-08
0
Screaming Frog SEO Spider/16.7
2023-05-02
1
Screaming Frog SEO Spider/17.2
2023-03-20
1
Screaming Frog SEO Spider/18.1
2022-12-22
0
Screaming Frog SEO Spider/18.2
2023-03-07
2
Screaming Frog SEO Spider/18.5
2023-06-28
2
Screaming Frog SEO Spider/19.0
2023-07-22
1
Screaming Frog SEO Spider/5.1
0
Screaming Frog SEO Spider/6.2
0
Screaming Frog SEO Spider/8.1
2023-05-29
4
Screaming Frog SEO Spider/9.4
2023-04-25
1
Seekport Crawler
2022-06-08
178
SeekportBot
2023-09-21
80
Semanticbot/1.0
2022-03-08
1
SemanticJuice
2021-10-25
0
SemrushBot-BA
2023-09-21
11
SemrushBot/6~bl
2023-03-18
1
SemrushBot/7~bl
2023-09-22
140
SemrushBot/7~bl-slave
2022-01-02
0
SenutoBot/1.0
2023-04-24
4
SEO Scanner/1.2.4
2022-05-11
0
SeohrefsBot/1.0.0
2023-07-23
1
SEOlizer/1.1
2022-08-10
1
serpstatbot/1.0
2021-11-09
4
serpstatbot/2.1
2023-09-14
27
SERPtimizerBot
2023-09-19
1
SeznamBot/3.2
2023-03-22
96
SeznamBot/4.0
2023-09-22
16
SeznamBot/4.0-RC1
2023-05-17
260
Sidetrade indexer bot
2022-12-02
1
SirdataBot
2023-09-22
201
SiteAuditBot/0.97
2022-03-03
0
SiteCheckerBotCrawler/1.0
0
Slackbot 1.0
2023-07-19
15
Slackbot-LinkExpanding 1.0
2023-07-19
16
Snap URL Preview Service
2023-09-22
74
Sogou head spider/3.0
2022-10-20
0
Sogou web spider/4.0
2023-05-25
361
special_archiver/3.1.1
2022-09-19
23
Squirrel-spider/2.0
2022-01-26
0
startmebot/1.0
2023-09-21
694
StartpagePrivateImageProxy/2.0
2023-06-21
1
Startpagina-Linkchecker/1.0
2023-07-04
2
SurdotlyBot/1.0
2023-09-20
75
t3versionsBot/1.0
2023-08-30
3
TBI-HttpOpenPlugin/0.0.2
0
TBI-HttpOpenPlugin/0.1.0
0
tchelebi/1.0
2023-09-17
3
Thither.Direct Bot/0.0.1
0
Timpibot/0.8
2023-08-31
4
TinEye-bot/1.31
2023-07-28
1
TombaPublicWebCrawler/3.5
2022-06-01
0
trendictionbot/0.5.0
2023-04-17
4
TTD-Content
2023-09-22
1760
Twingly Recon-Klondike/1.0
2023-01-01
0
Twitterbot/1.0
2023-06-25
15
UniversalFeedParser/5.2.1
0
VelenPublicWebCrawler/1.0
2023-09-12
41
VsuSearchSpider/1.0
2023-03-28
3
vuhuvBot/1.0
2022-12-16
1
webgains-bot
2023-08-11
2
weborama-fetcher
2023-06-27
18
webprosbot/2.0
2023-08-24
195
WebwikiBot/2.1
2023-08-28
3
WellKnownBot/0.1
2023-09-22
34
WhatStuffWhereBot
2022-09-20
0
woorankreview/2.0
2023-09-14
50
Wordupindexinfo1/0
2023-08-18
3
WP-URLDetails/6.1.1-alpha-54813
2022-12-23
0
Yahoo Ad monitoring
2023-06-21
4
Yahoo Link Preview
2023-04-23
1
Yahoo! Slurp
2023-06-21
36
Yahoo! Slurp China
2023-03-02
2
YahooMailProxy
2022-04-26
0
YandexAccessibilityBot/3.0
2022-02-17
14
YandexBlogs/0.99
2023-08-16
1
YandexBot/3.0
2023-09-22
1770
YandexFavicons/1.0
2023-09-15
177
YandexImages/3.0
2023-09-22
431
YandexMobileBot/3.0
2022-02-17
18
YandexNews/4.0
2023-05-01
2
YisouSpider
2023-09-20
250
Yooo! Search Engine Bot BETA
2023-06-17
1
ZaldamoSearchBot
2022-06-30
0
zgrab/0.x
2023-09-22
1258
zgtemple-bot
2021-12-22
0
ZoominfoBot
2023-09-21
126