Bot name
Last detected
IPs known
1st ZipCommander
2021-03-23
0
admantx-adform/3.1
2022-07-18
5
admantx-eusyncbatch01/3.1
0
admantx-ussy04/3.2
2023-01-26
17
AdsBot-Google
2022-11-18
0
AdsBot-Google-Mobile
2021-12-17
2
adscanner/1.0
2022-05-11
4
AdsTxtCrawler/1.0
2023-01-20
70
AdsTxtCrawler/1.0.2
2023-01-28
54
AdsTxtCrawlerTP/1.2
2023-01-27
1
AhrefsBot/2.0
2021-02-04
0
AhrefsBot/6.1
2021-03-11
157
AhrefsBot/7.0
2023-01-28
643
aiHitBot/2.9
2023-01-15
3
AlexandriaOrgBot/1.0
2022-02-11
0
Amazon-Advertising-ad-standards-bot/1.0
2023-01-27
1
AmazonAdBot/1.0
2023-01-28
18
Amazonbot/0.1
2022-08-20
9
AnyEvent-HTTP/2.22
2022-01-16
0
AppEngine-Google
2022-09-13
34
Applebot/0.1
2023-01-28
1733
archive.org_bot
2022-10-16
34
AwarioBot/1.0
2023-01-28
11
AwarioRssBot/1.0
2023-01-02
1
AwarioSmartBot/1.0
2022-04-23
3
Baiduspider-render/2.0
2023-01-27
605
Baiduspider/2.0
2023-01-28
677
Barkrowler/0.9
2023-01-24
270
bidswitchbot/1.0
2023-01-28
47
Bidtellect/0.0.1107.0
2023-01-09
0
Bidtellect/0.0.1125.0
2023-01-20
1
Bigsearch.ca/Nutch-1.0-dev
0
bingbot/2.0
2023-01-28
1541
BingPreview/1.0b
2023-01-28
1039
Birdcrawlerbot/0.5
2021-09-12
0
BLEXBot/1.0
2023-01-27
50
BlogBeatbot/2.5
2022-02-14
0
Bloglines/3.1
2022-05-29
0
CATExplorador/1.0beta
2023-01-24
10
CensysInspect/1.1
2023-01-27
179
centuryb.o.t9
2022-09-20
4
CheckMarkNetwork/1.0
2023-01-25
42
CISPA Web Analyzer
2021-09-15
0
CloudFlare-AlwaysOnline/1.0
0
coccocbot-image/1.0
2022-07-17
36
coccocbot-web/1.0
2023-01-28
144
Cocolyzebot/1.0
2022-08-24
33
companyspotter/0.0.0.0
2022-09-17
0
companyspotter/0.1984.82.1324
2022-01-04
0
companyspotter/1.0.0.0
2023-01-11
1
crawler_eb_germany_2.0
2023-01-17
1
Crawling at Home Project
2021-08-13
0
Crawlson/1.0
2023-01-09
30
CriteoBot/0.1
2023-01-28
130
CSIRT.CZ DNS crawler
2021-02-11
0
CyotekWebCopy/1.9
2022-12-09
0
DataForSeoBot/1.0
2023-01-27
5
Dataprovider.com
2023-01-27
208
Discordbot/2.0
2022-10-15
3
DnBCrawler-Analytics
2023-01-18
1
dns-crawler/1.4.10.dev6
0
dns-crawler/1.4.10.dev7
0
DNSResearchBot/2.1
2021-04-25
1
Domains Project/1.3.7
2023-01-11
1
DomainStatsBot/1.0
2023-01-27
4
DuckDuckBot-Https/1.1
2021-11-11
9
DuckDuckBot/1.1
2021-05-16
0
DuckDuckGo-Favicons-Bot/1.0
2023-01-28
2
E. Orliac, G. Fourestey/1.0
0
EmailWolf 1.00
2022-08-26
0
ev-crawler/1.0
2023-01-26
4
evc-batch/2.0
2022-02-11
2
facebookexternalhit/1.0
2023-01-12
19
facebookexternalhit/1.1
2023-01-28
655
facebookexternalhit/1.2
2022-04-06
0
Feedfetcher-Google
2022-11-07
1
feedparser/6.0.10
2022-09-10
0
FemtosearchBot/1.0
2023-01-16
4
FindITAnswersbot/1.0
2021-03-23
0
FlipboardProxy/1.2
2021-06-18
1
FullStoryBot/1.0
2022-09-27
0
GigablastOpenSource/1.0
2022-12-21
5
Gluten Free Crawler/1.0
0
Google-adstxt
2023-01-28
111
Google-Adwords-Instant
2022-09-28
5
Google-Display-Ads-Bot
2023-01-24
2
Google-PageRenderer
2023-01-13
4
Google-Read-Aloud
2023-01-28
180
Googlebot-Image/1.0
2023-01-28
595
Googlebot-Mobile/2.1
2022-03-15
0
Googlebot/2.1
2023-01-28
3004
Grammarly/1.0
2021-08-03
3
GrapeshotCrawler/2.0
2023-01-28
16
Graydon bot/1.0
2022-04-15
1
Gregarius/0.5.2
2022-10-26
1
Grover/Grover-1.18
2021-09-15
0
GumGum-Bot/1.0
2022-03-04
0
heritrix/3.3.0
2022-09-29
0
HTTP Banner Detection
2023-01-21
10
HubSpot Crawler
2022-03-27
0
HubSpot Webcrawler
2022-12-29
6
ias-ir/3.1
2023-01-28
208
ias-or/3.1
2023-01-28
157
ias-va/3.1
2023-01-28
341
Iframely/1.3.1
2023-01-26
9
Iframely/1.6.1
2022-03-25
0
IndeedBot 1.1
2021-10-24
5
INETDEX-BOT/1.5
2023-01-28
2
InfoTigerBot/1.9
2021-11-28
0
intelx.io_bot
2023-01-16
1
InternetMeasurement/1.0
2023-01-28
58
ivre-masscan/1.3
2021-05-12
0
KauaiSlowBot
2021-05-24
0
Keybot Translation-Search-Machine
2022-12-03
3
KomodiaBot/1.0
2022-07-02
11
KStandBot/1.0
2022-08-09
0
LanaiBotmarch
2021-03-19
0
LightspeedSystemsCrawler
2022-12-13
3
Linespider/1.1
2023-01-22
45
LinkCityChecker/1.0
2021-11-28
0
linkdexbot/2.0
2023-01-27
6
LinkedInBot/1.0
2023-01-25
4
LinkWalker/3.0
2021-06-03
1
LivelapBot/0.2
2022-11-21
2
Lkx-TraversalHttpPlugin/0.0.1
2021-11-14
0
lkxscan/v0.1.0
2021-05-13
0
looid.com Search Engine/0.1
0
Mail.RU_Bot/2.0
2023-01-28
177
Mail.RU_Bot/Fast/2.0
2021-09-08
0
Mail.RU_Bot/Favicons/2.0
2023-01-24
14
Mail.RU_Bot/Img/2.0
2021-07-29
3
Mail.RU_Bot/Robots/2.0
2023-01-27
24
MarketingMiner v2.0
2022-12-13
0
masscan-ng/1.3
2022-12-21
0
masscan/1.0
2023-01-26
47
masscan/1.3
2023-01-28
93
MBCrawler/1.0
2023-01-28
4
MediaMathbot/1.0
2023-01-28
36
Mediapartners-Google
2023-01-28
398
Mediatoolkitbot
2022-03-07
4
MegaIndex.ru/2.0
2023-01-27
14
Microsoft.Data.Mashup
2022-09-21
0
MicrosoftPreview/2.0
2022-10-19
0
MixnodeCache/1.8
2021-11-22
1
MJ12bot/v1.2.4
2022-05-08
0
MJ12bot/v1.4.1
2022-04-17
0
MJ12bot/v1.4.2
2022-12-15
0
MJ12bot/v1.4.8
2023-01-28
877
MojeekBot/0.10
2022-05-06
1
MojeekBot/0.11
2023-01-26
1
MonitorBacklinksWP
2022-06-27
1
msnbot-media/1.1
2023-01-26
2
MxToolbox/Beta7
2023-01-14
1
My Nutch Spider/Nutch-1.18
2021-10-20
0
Neevabot/1.0
2023-01-27
2
NetcraftSurveyAgent/1.0
2022-12-07
0
netEstate NE Crawler
2023-01-22
1
NetNewsWire/2.x
2021-04-16
0
NetpeakCheckerBot/3.6
2022-02-01
0
Nicecrawler/1.1
2023-01-25
1
Nimbostratus-Bot/v1.3.2
2021-07-02
36
node-fetch/1.0
2022-12-30
82
olbicobot/Nutch-1.15-SNAPSHOT
0
PagePeeker/3.0
2021-09-25
2
Pandalytics/1.0
2022-12-23
52
PaperLiBot/2.1
2021-06-18
0
PartyflockLinkChecker/0.3
0
PhantomBot/1.0
2023-01-27
1
Pinterestbot/1.0
2023-01-28
2
Pixalate-Crawler
2022-03-29
0
Plagger/0.x.xx
2021-05-03
0
pmoz.info ODP link checker
2022-10-24
1
Protopage/3.0
2021-10-15
0
Quantcastbot/1.0
2023-01-17
1
Qwantify/1.0
2023-01-28
20
Qwantify/2.4w
2022-01-14
40
Qwarrybot/2.0
2022-01-14
0
RepoLookoutBot/v1.1.0
2023-01-25
2
RobotsTxtParser-VIPnytt/2.1
2022-01-12
0
rogerbot/1.2
2021-07-13
0
RukiCrawler/1.0
2021-05-25
0
SabsimBot/3.0
2022-02-25
1
Scrapy/1.5.0
2022-09-11
1
Scrapy/1.7.3
2023-01-27
1
Scrapy/1.7.4
2022-10-05
0
Scrapy/2.2.1
2021-04-27
2
Scrapy/2.3.0
2021-02-04
0
Scrapy/2.4.0
2021-12-06
1
Scrapy/2.4.1
2021-07-28
0
Scrapy/2.5.1
2021-11-15
0
Scrapy/2.6.1
2022-03-30
0
Scrapy/2.6.2
2022-09-11
0
Scrapy/2.7.1
2023-01-25
1
Screaming Frog SEO Spider/10.1
0
Screaming Frog SEO Spider/11.2
0
Screaming Frog SEO Spider/11.3
0
Screaming Frog SEO Spider/12.2
0
Screaming Frog SEO Spider/12.3
0
Screaming Frog SEO Spider/12.6
2021-02-06
1
Screaming Frog SEO Spider/13.2
2021-04-25
1
Screaming Frog SEO Spider/14.1
2021-09-18
2
Screaming Frog SEO Spider/14.2
2021-04-01
0
Screaming Frog SEO Spider/14.3
2021-03-21
0
Screaming Frog SEO Spider/15.2
2021-06-08
0
Screaming Frog SEO Spider/16.2
2021-11-16
0
Screaming Frog SEO Spider/16.3
2021-12-06
0
Screaming Frog SEO Spider/16.4
2021-12-21
0
Screaming Frog SEO Spider/16.5
2022-02-08
0
Screaming Frog SEO Spider/16.7
2022-03-12
0
Screaming Frog SEO Spider/17.2
2022-10-25
0
Screaming Frog SEO Spider/18.1
2022-12-22
0
Screaming Frog SEO Spider/5.1
0
Screaming Frog SEO Spider/6.2
0
Screaming Frog SEO Spider/9.4
2022-02-12
0
Seekport Crawler
2022-06-08
258
Semanticbot/1.0
2022-03-08
1
SemanticJuice
2021-10-25
0
SemanticScholarBot
2021-02-19
1
SemrushBot-BA
2023-01-28
13
SemrushBot-BM/1.0
2021-09-05
0
SemrushBot-SA/0.97
2021-06-07
3
SemrushBot-SI/0.97
2021-07-30
1
SemrushBot/6~bl
2022-12-06
1
SemrushBot/7~bl
2023-01-28
80
SemrushBot/7~bl-slave
2022-01-02
0
SEO Scanner/1.2.4
2022-05-11
0
SEOlizer/1.1
2022-08-10
1
serpstatbot/1.0
2021-11-09
9
serpstatbot/2.0 beta
2021-05-18
9
serpstatbot/2.1
2023-01-17
2
SERPtimizerBot
2022-02-02
0
SeznamBot/3.2
2023-01-26
96
SeznamBot/4.0-RC1
2023-01-28
53
Sidetrade indexer bot
2022-12-02
1
SiteAuditBot/0.97
2022-03-03
0
SiteCheckerBotCrawler/1.0
0
Slackbot 1.0
2023-01-03
21
Slackbot-LinkExpanding 1.0
2023-01-03
25
Snap URL Preview Service
2023-01-13
2
Snarfer/0.x.x
2021-04-14
0
Sogou head spider/3.0
2022-10-20
0
Sogou web spider/4.0
2023-01-28
373
special_archiver/3.1.1
2022-09-19
26
Squirrel-spider/2.0
2022-01-26
0
startmebot/1.0
2023-01-27
680
Startpagina-Linkchecker/1.0
2023-01-28
4
SurdotlyBot/1.0
2023-01-23
121
t3versionsBot/1.0
2021-12-30
0
TBI-HttpOpenPlugin/0.0.2
0
TBI-HttpOpenPlugin/0.1.0
2021-02-13
4
TBI-WebScanner/0.0.1
2021-02-13
3
tchelebi/1.0
2023-01-18
2
Thither.Direct Bot/0.0.1
0
TinEye-bot/1.31
2022-12-17
0
TinyBotTestUA
2021-07-16
0
TombaPublicWebCrawler/3.5
2022-06-01
0
trendictionbot/0.5.0
2023-01-17
5
TTD-Content
2023-01-28
1145
TweetmemeBot/4.0
2021-02-24
1
Twingly Recon-Klondike/1.0
2023-01-01
0
Twitterbot/1.0
2023-01-27
11
UniversalFeedParser/5.2.1
2021-02-01
1
vebidoobot/1.0
2021-02-24
1
VelenPublicWebCrawler/1.0
2023-01-24
39
VsuSearchSpider/1.0
2021-10-22
0
vuhuvBot/1.0
2022-12-16
1
WebDataStats/1.0
2021-04-04
0
webgains-bot
2021-05-06
0
weborama-fetcher
2022-05-29
9
webprosbot/2.0
2023-01-18
2
WellKnownBot/0.1
2023-01-28
3
WhatStuffWhereBot
2022-09-20
0
woorankreview/2.0
2022-12-30
40
WP-URLDetails/6.1.1-alpha-54813
2022-12-23
0
Yahoo Ad monitoring
2022-11-13
0
Yahoo Link Preview
2022-12-09
0
Yahoo! Slurp
2023-01-12
44
Yahoo! Slurp China
2023-01-26
1
YahooMailProxy
2022-04-26
0
YandexAccessibilityBot/3.0
2022-02-17
15
YandexBot/3.0
2023-01-28
888
YandexFavicons/1.0
2023-01-28
94
YandexImages/3.0
2023-01-27
164
YandexMetrika/3.0
2021-02-24
0
YandexMobileBot/3.0
2022-02-17
20
YandexNews/4.0
2022-10-24
1
YisouSpider
2023-01-24
261
ZaldamoSearchBot
2022-06-30
0
zgtemple-bot
2021-12-22
0
ZoominfoBot
2023-01-28
17