Bot name
Last detected
IPs known
7Siters/1.08
2019-07-10
0
adidxbot/2.0
2020-06-01
0
admantx-adform/3.1
2020-12-06
24
admantx-eusyncbatch01/3.1
2020-12-18
2
admantx-sap/2.4
2019-07-13
0
admantx-usaspb/3.1
2020-03-15
0
admantx-ussy04/3.1
2020-10-18
2
admantx-ussy04/3.2
2020-12-01
3
AdsBot-Adsrvr
2019-10-29
1
AdsBot-Google
2019-12-27
0
AdsBot-Google-Mobile
2021-01-29
0
adscanner/1.0
2021-02-28
4
adscanner/1.0
2019-10-02
4
adscanner/1.1
2020-03-17
4
AdsTxtCrawler/1.0
2021-02-17
4
AffiliateLabz/1.0
2020-02-05
0
AhrefsBot/2.0
2021-02-04
0
AhrefsBot/5.2
2019-05-26
3
AhrefsBot/6.1
2021-02-27
680
AhrefsBot/7.0
2021-02-28
433
aiHitBot/2.9
2021-02-20
4
AlphaBot/3.2
2019-11-06
0
Amazonbot/0.1
2020-10-02
0
AppEngine-Google
2021-02-25
5
Applebot/0.1
2021-02-27
504
archive.org_bot
2021-02-24
1
archive.org_bot
2019-09-08
2
AwarioRssBot/1.0
2020-08-14
2
AwarioSmartBot/1.0
2020-11-24
0
Baiduspider-image+
2019-10-24
42
Baiduspider-render/2.0
2021-02-27
337
Baiduspider/2.0
2021-02-27
603
BananaBot/0.6.1
2020-07-11
0
Barkrowler/0.9
2019-07-04
8
Barkrowler/0.9
2021-02-26
17
bidswitchbot/1.0
2021-02-27
40
Bigsearch.ca/Nutch-1.0-dev
2019-05-18
0
bingbot/2.0
2021-02-28
1698
BingPreview/1.0b
2021-02-27
1480
bitlybot/3.0
2020-01-25
1
BLEXBot/1.0
2021-02-27
52
BorneoBot/0.7.1
2020-05-24
0
BorneoBot/0.9.1
2020-05-26
0
BuiltWith/1.0
2021-01-26
0
BuiltWith/1.1
2020-04-22
0
bumb.ly/0.01
2019-11-10
0
Bytespider
2021-02-27
321
CATExplorador/1.0beta
2021-02-24
11
CensysInspect/1.1
2021-02-27
41
centuryb.o.t9
2020-11-27
1
CheckMarkNetwork/1.0
2021-02-24
125
CipaCrawler/3.0
2020-02-08
5
CISPA Webcrawler
2020-07-06
0
Clarabot/1.4
2020-04-18
10
Cliqzbot/2.0
2019-08-13
44
Cliqzbot/3.0
2020-07-12
228
CloudFlare-AlwaysOnline/1.0
2020-02-23
1
coccocbot-image/1.0
2021-02-27
18
coccocbot-web/1.0
2021-02-24
23
Cocolyzebot/1.0
2021-02-26
36
Crawlson/1.0
2021-02-27
18
CriteoBot/0.1
2021-02-28
18
CSIRT.CZ DNS crawler
2021-02-11
0
CSS Certificate Spider
2019-08-01
0
CZ.NIC DNS crawler
2021-01-14
6
datagnionbot
2021-01-27
1
Dataprovider.com
2021-02-18
74
Discordbot/2.0
2020-12-21
4
dns-crawler/1.4.10.dev6
2020-08-20
0
dns-crawler/1.4.10.dev7
2020-08-27
0
dns-crawler/1.4.5
2020-06-25
0
dns-crawler/1.4.6
2020-07-09
0
dns-crawler/1.4.6.dev3
2020-06-25
0
dns-crawler/1.4.8
2020-07-16
0
DNSResearchBot/2.1
2021-02-14
1
DomainCrawler/3.0
2019-08-26
10
Domains Project
2020-01-20
0
Domains Project/1.0.1
2020-02-07
0
Domains Project/1.0.3
2020-03-09
0
Domains Project/1.0.4
2020-04-14
0
Domains Project/1.0.6
2020-04-21
0
Domains Project/1.0.8
2020-06-21
0
Domains Project/1.1.0
2020-08-05
0
Domains Project/1.1.1
2020-08-07
0
Domains Project/1.1.4
2020-08-09
0
Domains Project/1.2.1
2020-09-19
0
Domains Project/1.2.3
2020-10-18
0
Domains Project/1.2.4
2020-11-05
0
Domains Project/1.2.7
2020-11-27
0
Domains Project/1.3.2
2021-01-10
0
DomainSigmaCrawler/0.1
2019-11-24
0
DomainStatsBot/1.0
2021-02-13
6
DuckDuckBot-Https/1.1
2020-10-04
8
DuckDuckBot/1.0
2020-01-03
0
DuckDuckGo-Favicons-Bot/1.0
2021-02-28
3
E. Orliac, G. Fourestey/1.0
2019-08-27
0
eContext/1.0
2019-10-30
0
evc-batch/2.0
2021-02-27
4
facebookexternalhit/1.0
2020-10-14
9
facebookexternalhit/1.1
2021-02-28
776
facebookscraper/1.0
2020-07-03
0
FastCrawler/4.9
2019-06-05
0
FatBoyKimComBot
2019-11-02
0
Feedfetcher-Google
2021-02-21
1
FeedlyBot/1.0
2020-01-10
1
FemtosearchBot/1.0
2020-06-23
5
FlipboardProxy/1.2
2020-01-25
0
GarlikCrawler/1.2
2020-11-05
1
GigablastOpenSource/1.0
2020-10-15
4
Gigabot/1.1.1.2
2019-10-17
0
Gluten Free Crawler/1.0
2019-12-14
1
go-resty/2.0.0
2019-10-04
1
Google-Adwords-Instant
2021-02-04
4
Google-Read-Aloud
2021-02-19
3
Googlebot-Image/1.0
2021-02-27
527
Googlebot/2.1
2021-02-28
4209
Grahambot/0.1
2019-05-18
0
Grammarly/1.0
2019-11-09
0
GrapeshotCrawler/2.0
2021-02-27
49
Graydon bot/1.0
2020-09-05
1
Gregarius/0.5.2
2020-08-27
0
heritrix/3.4.0
2021-01-27
0
heritrix/3.4.0
2020-07-20
0
HTTP Banner Detection
2021-02-25
4
HuaweiWebCatBot/6.0
2020-10-27
1
HubSpot Connect/2.0
2019-10-19
0
HubSpot Webcrawler
2019-11-28
0
hypestat/1.0
2020-03-31
0
ia_archiver
2020-11-13
13
ia_archiver/1.0
2020-04-08
0
ias-va/3.1
2021-02-27
163
Iframely/1.3.1
2019-08-19
0
IndeedBot 1.1
2019-11-12
0
InoopaBot/1.0
2021-01-13
1
intelx.io_bot
2021-02-27
1
istellabot/t.1.13
2019-08-01
3
JobboerseBot
2020-01-28
7
Jooblebot/2.0
2019-06-28
0
K7MLWCBot/1.0
2020-02-06
0
Keybot Translation-Search-Machine
2020-08-09
0
Khazandebot/2.1
2020-08-02
0
KomodiaBot/1.0
2021-02-23
6
LightspeedSystemsCrawler
2021-01-21
4
Linespider/1.1
2021-02-27
33
linkdexbot/2.0
2021-02-27
5
LinkedInBot/1.0
2021-02-17
4
LinkpadBot/1.12
2019-07-28
1
LinkpadBot/2.3
2021-01-22
1
LinkWalker/3.0
2020-08-16
1
LivelapBot/0.2
2019-06-27
0
looid.com Search Engine/0.1
2019-08-30
1
Mail.RU_Bot/2.0
2021-02-27
154
Mail.RU_Bot/Favicons/2.0
2021-01-28
0
Mail.RU_Bot/Img/2.0
2020-01-20
13
Mail.RU_Bot/Robots/2.0
2021-02-23
17
masscan/1.0
2021-02-27
82
MBCrawler/1.0
2021-02-27
2
Mediapartners-Google
2021-02-28
354
Mediatoolkitbot
2020-09-02
2
MegaIndex.ru/2.0
2021-02-24
9
MetaInspector/5.6.0
2019-12-05
1
MixnodeCache/1.8
2021-02-22
3
MJ12bot/v1.4.7
2019-07-20
2
MJ12bot/v1.4.8
2021-02-28
575
MojeekBot/0.10
2021-02-19
1
MojeekBot/0.6
2019-09-28
1
MojeekBot/0.7
2020-06-27
1
MojeekBot/0.9
2020-07-01
0
MonitorBacklinksWP
2021-02-07
0
msnbot-media/1.1
2020-09-23
0
msnbot/2.0b
2021-01-02
642
MuckRack/1.0
2020-01-25
0
MxToolbox/Beta7
2020-02-02
0
netEstate NE Crawler
2021-02-27
3
NetpeakCheckerBot/3.2
2019-08-29
0
Nimbostratus-Bot/v1.3.2
2021-02-12
48
node-fetch/1.0
2021-02-27
28
olbicobot/Nutch-1.15-SNAPSHOT
2020-10-26
2
Onespot-ScraperBot/1.0
2020-01-14
0
PagePeeker/3.0
2020-07-09
0
PageThing.com
2021-02-19
0
Pandalytics/1.0
2020-12-09
34
Panscient/1.0
2019-12-15
0
PaperLiBot/2.1
2021-01-27
4
PartyflockLinkChecker/0.3
2019-10-30
1
pmoz.info ODP link checker
2021-01-11
1
Qwantbot/1.0
2020-10-14
2
Qwantify/2.4w
2021-02-05
67
Qwantify/Bleriot/1.1
2020-02-06
11
Qwantify/Bleriot/1.2.1
2019-10-03
0
Re-re Studio
2019-09-29
2
Researchscan/t12ca
2020-04-14
3
Researchscan/t12sns
2020-06-25
3
Researchscan/t13rl
2020-06-25
3
RyowlEngine/1.0
2020-01-22
0
SabsimBot/3.0
2020-11-24
0
Scrapy/1.4.0
2020-12-21
8
Scrapy/1.5.0
2020-12-12
3
Scrapy/1.5.1
2020-03-07
2
Scrapy/1.5.2
2019-07-18
1
Scrapy/1.6.0
2019-08-13
3
Scrapy/1.7.2
2020-11-11
4
Scrapy/1.8.0
2021-01-18
4
Scrapy/2.0.1
2020-04-24
0
Scrapy/2.2.1
2021-01-28
0
Scrapy/2.3.0
2021-02-04
0
Scrapy/2.4.0
2020-12-22
0
Screaming Frog SEO Spider/10.1
2020-01-09
0
Screaming Frog SEO Spider/11.2
2020-02-28
1
Screaming Frog SEO Spider/11.3
2019-10-02
0
Screaming Frog SEO Spider/12.2
2019-12-09
0
Screaming Frog SEO Spider/12.3
2019-12-15
0
Screaming Frog SEO Spider/12.6
2021-02-06
1
Screaming Frog SEO Spider/13.2
2020-11-09
1
Screaming Frog SEO Spider/14.1
2021-01-20
0
Screaming Frog SEO Spider/5.1
2020-04-20
0
Screaming Frog SEO Spider/6.2
2019-11-20
0
SecSpider/1.0
2020-11-05
0
Seekport Crawler
2021-02-25
133
Semanticbot/1.0
2020-09-02
0
SemanticScholarBot
2021-02-19
1
SemrushBot-BA
2021-02-26
23
SemrushBot-SA/0.97
2021-02-22
4
SemrushBot-SI/0.97
2019-09-30
0
SemrushBot/3~bl
2019-08-21
29
SemrushBot/6~bl
2021-01-18
70
SemrushBot/7~bl
2021-02-28
39
SEOlizer/1.1
2020-08-15
0
serpstatbot/1.0
2021-02-15
29
serpstatbot/2.0 beta
2021-02-02
7
SeznamBot/3.2
2021-02-28
144
SeznamBot/3.2-test1
2019-12-28
3
Sidetrade indexer bot
2021-02-22
1
SiteAnalyzerbot/2.0
2020-06-22
0
SiteCheckerBotCrawler/1.0
2019-10-16
0
SiteExplorer/1.1b
2019-05-20
1
Slackbot 1.0
2021-02-10
19
Slackbot-LinkExpanding 1.0
2021-02-10
40
SniffRSS/0.5beta
2019-05-18
0
Sogou web spider/4.0
2021-02-27
466
special_archiver/3.1.1
2020-09-19
5
startmebot/1.0
2021-02-26
948
Startpagina-Linkchecker/1.0
2021-02-18
2
Streamline3Bot/1.0
2019-09-29
1
SurdotlyBot/1.0
2021-02-20
113
TBI-HttpOpenPlugin/0.0.2
2020-08-22
0
TBI-HttpOpenPlugin/0.1.0
2021-02-13
36
TBI-WebScanner/0.0.1
2021-02-13
39
tchelebi/1.0
2021-02-25
1
The Knowledge AI
2020-10-19
5
Thither.Direct Bot/0.0.1
2019-06-14
0
TinEye-bot/1.31
2019-12-10
5
tracemyfile/1.0
2020-03-10
1
trendictionbot/0.5.0
2021-01-27
3
TTD-Content
2021-02-25
104
TweetmemeBot/4.0
2021-02-24
3
Twingly Recon
2019-05-18
0
Twitterbot/1.0
2021-02-18
9
UniversalFeedParser/5.2.1
2021-02-01
2
Uptimebot/1.0
2019-06-27
177
Vagabondo/4.0
2020-06-18
7
Validator.nu/LV
2020-03-21
5
vebidoobot/1.0
2021-02-24
4
VelenPublicWebCrawler/1.0
2021-01-12
22
vuhuvBot/1.0
2021-02-26
4
W3C_Validator/1.3
2020-03-21
4
weborama-fetcher
2021-02-26
11
woorankreview/2.0
2021-01-07
85
Yahoo Ad monitoring
2021-02-24
0
Yahoo Link Preview
2020-12-22
1
Yahoo! Slurp
2021-02-24
23
Yahoo! Slurp/3.0
2021-01-17
0
Yandex/1.01.001
2019-05-18
0
YandexAccessibilityBot/3.0
2021-01-19
17
YandexBot/3.0
2021-02-27
432
YandexFavicons/1.0
2021-02-27
6
YandexImages/3.0
2021-02-27
50
YandexMetrika/3.0
2021-02-24
0
YandexMobileBot/3.0
2020-09-10
34
YandexNews/4.0
2020-06-16
0
YisouSpider
2021-02-17
66