List of webbots

Download without IP addresses XML format JSON format CSV format

Download including IP addresses XML format JSON format CSV format

Bot name
Last detected
IPs known
 
1M7bot/1.0
2021-02-23
0
360Spider
2021-03-15
1
3w24bot/1.2
2019-07-16
0
3w24bot/1.3
2019-07-12
0
3w24bot/1.4
2020-03-29
0
7Siters/1.08
2019-07-10
0
AdAuth/1.0
2021-09-14
2
adidxbot/2.0
2020-06-01
0
admantx-adform/3.1
2021-06-06
24
admantx-eusyncbatch01/3.1
2020-12-18
2
admantx-sap/2.4
2019-07-13
0
admantx-usaspb/3.1
2020-03-15
0
admantx-ussy04/3.1
2020-10-18
2
admantx-ussy04/3.2
2020-12-01
3
AdsBot-Adsrvr
2019-10-29
1
AdsBot-Google
2019-12-27
0
AdsBot-Google-Mobile
2021-01-29
0
Adsbot/3.1
2021-03-15
29
adscanner
2019-09-06
2
adscanner/1.0
2021-10-14
4
adscanner/1.0
2019-10-02
4
adscanner/1.1
2020-03-17
4
AdsTxtCrawler/1.0
2021-10-13
26
AffiliateLabz/1.0
2020-02-05
0
AhrefsBot/2.0
2021-02-04
0
AhrefsBot/5.2
2019-05-26
3
AhrefsBot/6.1
2021-03-11
680
AhrefsBot/7.0
2021-10-21
555
aiHitBot/2.9
2021-10-04
4
AlphaBot/3.2
2019-11-06
0
Amazonbot/0.1
2021-10-19
9
AppEngine-Google
2021-10-15
26
Applebot/0.1
2021-10-21
1373
archive.org_bot
2021-10-21
14
archive.org_bot
2019-09-08
2
AteniumBot
2020-12-17
1
AwarioRssBot/1.0
2020-08-14
2
AwarioSmartBot/1.0
2020-11-24
0
Baidu Japan
2019-05-18
0
Baiduspider
2020-07-18
0
Baiduspider-image+
2019-10-24
42
Baiduspider-render/2.0
2021-10-20
689
Baiduspider/2.0
2021-10-19
830
BananaBot/0.6.1
2020-07-11
0
Barkrowler/0.9
2019-07-04
8
Barkrowler/0.9
2021-10-19
24
bidswitchbot/1.0
2021-10-21
61
Bigsearch.ca/Nutch-1.0-dev
2019-05-18
0
bingbot/2.0
2021-10-21
1822
BingPreview/1.0b
2021-10-21
1565
bitlybot/3.0
2020-01-25
1
BLEXBot/1.0
2021-10-21
82
BorneoBot/0.7.1
2020-05-24
0
BorneoBot/0.9.1
2020-05-26
0
BUbiNG
2020-09-13
0
BuiltWith/1.0
2021-01-26
0
BuiltWith/1.1
2020-04-22
0
bumb.ly/0.01
2019-11-10
0
Bytespider
2021-09-09
340
CATExplorador/1.0beta
2021-10-18
11
CCBot/2.0
2021-10-19
460
CensysInspect/1.1
2021-10-21
87
centuryb.o.t9
2021-06-14
3
CheckMarkNetwork/1.0
2021-10-21
141
Cincraw/1.0
2021-06-07
4
CipaCrawler/3.0
2020-02-08
5
CISPA Webcrawler
2020-07-06
0
Clarabot/1.4
2020-04-18
10
Cliqzbot/2.0
2019-08-13
44
Cliqzbot/3.0
2020-07-12
228
CloudFlare-AlwaysOnline/1.0
2020-02-23
1
CMS Crawler
2021-01-29
1
coccocbot-image/1.0
2021-10-20
36
coccocbot-web/1.0
2021-10-17
85
Cocolyzebot/1.0
2021-10-20
36
colly
2021-09-06
8
contxbot/1.
2019-10-20
0
Crawlson/1.0
2021-09-13
21
CriteoBot/0.1
2021-10-21
26
CSIRT.CZ DNS crawler
2021-02-11
0
CSS Certificate Spider
2019-08-01
0
CZ.NIC DNS crawler
2021-01-14
6
datagnionbot
2021-01-27
1
Dataprovider.com
2021-10-14
117
DataXu/1.0
2021-10-20
8
DF Bot 1.0
2021-03-03
1
Diffbot/0.1
2020-11-06
8
Discordbot/2.0
2021-08-04
4
dns-crawler/1.4.10.dev6
2020-08-20
0
dns-crawler/1.4.10.dev7
2020-08-27
0
dns-crawler/1.4.5
2020-06-25
0
dns-crawler/1.4.6
2020-07-09
0
dns-crawler/1.4.6.dev3
2020-06-25
0
dns-crawler/1.4.8
2020-07-16
0
DNSResearchBot/2.1
2021-04-25
1
DomainCrawler/3.0
2019-08-26
10
Domains Project
2020-01-20
0
Domains Project/1.0.1
2020-02-07
0
Domains Project/1.0.3
2020-03-09
0
Domains Project/1.0.4
2020-04-14
0
Domains Project/1.0.6
2020-04-21
0
Domains Project/1.0.8
2020-06-21
0
Domains Project/1.1.0
2020-08-05
0
Domains Project/1.1.1
2020-08-07
0
Domains Project/1.1.4
2020-08-09
0
Domains Project/1.2.1
2020-09-19
0
Domains Project/1.2.3
2020-10-18
0
Domains Project/1.2.4
2020-11-05
0
Domains Project/1.2.7
2020-11-27
0
Domains Project/1.3.2
2021-01-10
0
DomainSigmaCrawler/0.1
2019-11-24
0
DomainStatsBot/1.0
2021-10-20
6
DomCopBot
2020-08-10
0
DotBot/1.1
2021-08-19
61
DuckDuckBot-Https/1.1
2021-10-15
9
DuckDuckBot/1.0
2020-01-03
0
DuckDuckGo-Favicons-Bot/1.0
2021-10-21
4
E. Orliac, G. Fourestey/1.0
2019-08-27
0
eContext/1.0
2019-10-30
0
Elisabot
2021-07-21
6
evc-batch/2.0
2021-10-16
4
Exabot/3.0
2021-08-26
4
facebookexternalhit/1.0
2021-10-06
19
facebookexternalhit/1.1
2021-10-21
845
facebookscraper/1.0
2020-07-03
0
FastCrawler/4.9
2019-06-05
0
FatBoyKimComBot
2019-11-02
0
Faveeo/1.0
2020-01-25
0
Feedfetcher-Google
2021-02-21
1
Feedly/1.0
2021-08-03
4
FeedlyBot/1.0
2020-01-10
1
FemtosearchBot/1.0
2020-06-23
5
FlipboardProxy/1.2
2021-06-18
1
Gaisbot/3.0
2020-08-25
0
GarlikCrawler/1.2
2020-11-05
1
GigablastOpenSource/1.0
2021-05-29
5
Gigabot
2021-02-12
1
Gigabot/1.1.1.2
2019-10-17
0
Gluten Free Crawler/1.0
2019-12-14
1
go-resty/2.0.0
2019-10-04
1
Google-Adwords-Instant
2021-02-04
4
Google-Read-Aloud
2021-10-21
89
Googlebot-Image/1.0
2021-10-21
677
Googlebot/2.1
2021-10-21
4979
got
2021-10-17
33
Grahambot/0.1
2019-05-18
0
Grammarly/1.0
2021-08-03
3
GrapeshotCrawler/2.0
2021-10-21
49
Graydon bot/1.0
2021-10-20
1
Gregarius/0.5.2
2020-08-27
0
heritrix/3.4.0
2021-01-27
0
heritrix/3.4.0
2020-07-20
0
HTTP Banner Detection
2021-10-19
7
HuaweiWebCatBot/6.0
2020-10-27
1
HubSpot Connect/2.0
2019-10-19
0
HubSpot Webcrawler
2019-11-28
0
hypestat/1.0
2020-03-31
0
ia_archiver
2021-10-08
15
ia_archiver/1.0
2020-04-08
0
IAS crawler
2021-04-25
6
ias-ir/3.1
2021-10-21
128
ias-jp/3.1
2021-10-21
31
ias-or/3.1
2021-10-21
86
ias-sg/3.1
2021-10-21
14
ias-va/3.1
2021-10-21
217
Iframely/1.3.1
2019-08-19
0
IndeedBot 1.1
2021-08-02
4
InoopaBot/1.0
2021-01-13
1
intelx.io_bot
2021-10-09
1
istellabot/t.1.13
2019-08-01
3
Jaunt/1.5
2019-11-16
0
JobboerseBot
2020-01-28
7
Jooblebot/2.0
2019-06-28
0
K7MLWCBot/1.0
2020-02-06
0
Keybot Translation-Search-Machine
2020-08-09
0
Khazandebot/2.1
2020-08-02
0
KOCMOHABT
2021-10-18
36
KomodiaBot/1.0
2021-08-08
14
LightspeedSystemsCrawler
2021-09-25
4
Linespider/1.1
2021-08-28
38
Linguee Bot
2021-05-07
12
linkdexbot/2.0
2021-10-21
5
LinkedInBot/1.0
2021-02-17
4
LinkpadBot/1.12
2019-07-28
1
LinkpadBot/2.3
2021-01-22
1
LinkWalker/3.0
2021-06-03
2
LivelapBot/0.2
2019-06-27
0
looid.com Search Engine/0.1
2019-08-30
1
ltx71
2021-10-19
6
Mail.RU_Bot/2.0
2021-10-21
160
Mail.RU_Bot/Favicons/2.0
2021-10-21
8
Mail.RU_Bot/Img/2.0
2021-07-29
15
Mail.RU_Bot/Robots/2.0
2021-10-21
17
Mappy/1.0
2020-01-07
1
masscan/1.0
2021-09-24
96
masscan/1.3
2021-10-20
23
MauiBot
2021-06-07
15
MauiBot
2019-09-09
2
MBCrawler/1.0
2021-10-21
2
Mediapartners-Google
2021-10-21
464
Mediatoolkitbot
2020-09-02
2
meg/0.2
2020-11-20
2
MegaIndex.ru/2.0
2021-10-20
10
MetaInspector/5.6.0
2019-12-05
1
MixnodeCache/1.8
2021-10-18
3
MJ12bot/v1.4.7
2019-07-20
2
MJ12bot/v1.4.8
2021-10-21
666
MojeekBot/0.10
2021-10-20
1
MojeekBot/0.6
2019-09-28
1
MojeekBot/0.7
2020-06-27
1
MojeekBot/0.9
2020-07-01
0
MonitorBacklinksWP
2021-02-07
0
msnbot-media/1.1
2020-09-23
0
msnbot/1.0
2020-08-06
0
msnbot/1.1
2020-04-06
0
msnbot/2.0b
2021-01-02
642
MTRobot/0.2
2021-04-10
7
MuckRack/1.0
2020-01-25
0
MxToolbox/Beta7
2020-02-02
0
NameFo
2020-01-12
0
netEstate NE Crawler
2021-10-20
3
NetpeakCheckerBot/3.2
2019-08-29
0
Nimbostratus-Bot/v1.3.2
2021-07-02
48
niocBot
2020-03-16
0
niocBot
2020-04-13
0
Nmap Scripting Engine
0
node-fetch/1.0
2021-09-23
88
oBot/2.3.1
2021-10-17
41
olbicobot/Nutch-1.15-SNAPSHOT
2020-10-26
2
Onespot-ScraperBot/1.0
2020-01-14
0
PagePeeker/3.0
2021-09-25
2
PageThing
2021-07-05
2
PageThing.com
2021-05-04
4
Pandalytics/1.0
2021-09-16
48
panscient
2021-09-24
6
Panscient/1.0
2019-12-15
0
PaperLiBot/2.1
2021-01-27
4
PartyflockLinkChecker/0.3
2019-10-30
1
PetalBot
2021-04-09
2240
Picoweb_Bot
2019-11-20
0
pmoz.info ODP link checker
2021-07-14
1
proximic
2021-10-21
4607
psbot/0.1
2020-01-17
0
Qwantbot/1.0
2020-10-14
2
Qwantify/2.4w
2021-10-13
96
Qwantify/Bleriot/1.1
2020-02-06
11
Qwantify/Bleriot/1.2.1
2019-10-03
0
Re-re Studio
2019-09-29
2
Researchscan/t12ca
2020-04-14
3
Researchscan/t12sns
2020-06-25
3
Researchscan/t13rl
2020-06-25
3
Riddler
2021-06-28
3
RSSingBot
2019-10-11
0
RyowlEngine/1.0
2020-01-22
0
SabsimBot/3.0
2020-11-24
0
SafeDNSBot
2021-01-27
6
Scrapy/1.4.0
2020-12-21
8
Scrapy/1.5.0
2020-12-12
3
Scrapy/1.5.1
2020-03-07
2
Scrapy/1.5.2
2019-07-18
1
Scrapy/1.6.0
2019-08-13
3
Scrapy/1.7.2
2020-11-11
4
Scrapy/1.8.0
2021-01-18
4
Scrapy/2.0.1
2020-04-24
0
Scrapy/2.2.1
2021-04-27
2
Scrapy/2.3.0
2021-02-04
0
Scrapy/2.4.0
2020-12-22
0
Screaming Frog SEO Spider/10.1
2020-01-09
0
Screaming Frog SEO Spider/11.2
2020-02-28
1
Screaming Frog SEO Spider/11.3
2019-10-02
0
Screaming Frog SEO Spider/12.2
2019-12-09
0
Screaming Frog SEO Spider/12.3
2019-12-15
0
Screaming Frog SEO Spider/12.6
2021-02-06
1
Screaming Frog SEO Spider/13.2
2021-04-25
2
Screaming Frog SEO Spider/14.1
2021-09-18
2
Screaming Frog SEO Spider/5.1
2020-04-20
0
Screaming Frog SEO Spider/6.2
2019-11-20
0
SecSpider/1.0
2020-11-05
0
Seekport Crawler
2021-10-21
211
Semanticbot/1.0
2020-09-02
0
SemanticScholarBot
2021-02-19
1
SemrushBot
2021-10-19
6
SemrushBot-BA
2021-10-18
27
SemrushBot-SA/0.97
2021-06-07
4
SemrushBot-SI/0.97
2021-07-30
1
SemrushBot/3~bl
2019-08-21
29
SemrushBot/6~bl
2021-07-15
70
SemrushBot/7~bl
2021-10-21
40
SEOkicks
2021-10-03
14
SEOlizer/1.1
2020-08-15
0
serpstatbot/1.0
2021-10-19
29
serpstatbot/2.0 beta
2021-05-18
13
SeznamBot/3.2
2021-10-21
144
SeznamBot/3.2-test1
2019-12-28
3
Sidetrade indexer bot
2021-10-10
1
SiteAnalyzerbot/2.0
2020-06-22
0
SiteCheckerBotCrawler/1.0
2019-10-16
0
SiteExplorer/1.1b
2019-05-20
1
Slackbot 1.0
2021-10-15
28
Slackbot-LinkExpanding 1.0
2021-10-15
52
SMTBot/1.0
2021-10-09
7
SniffRSS/0.5beta
2019-05-18
0
Sogou web spider/4.0
2021-10-21
573
special_archiver/3.1.1
2021-10-21
13
SpiderLing
2019-10-21
0
startmebot/1.0
2021-10-21
1114
Startpagina-Linkchecker/1.0
2021-10-19
3
Streamline3Bot/1.0
2019-09-29
1
SurdotlyBot/1.0
2021-10-19
160
TBI-HttpOpenPlugin/0.0.2
2020-08-22
0
TBI-HttpOpenPlugin/0.1.0
2021-02-13
36
TBI-WebScanner/0.0.1
2021-02-13
39
tchelebi/1.0
2021-10-20
1
The Knowledge AI
2020-10-19
5
Thither.Direct Bot/0.0.1
2019-06-14
0
TinEye-bot/1.31
2019-12-10
5
TOBBOT
2021-01-13
1
tracemyfile/1.0
2020-03-10
1
trendictionbot/0.5.0
2021-09-20
3
TTD-Content
2021-10-20
155
Turnitin
2021-08-28
1
TurnitinBot
2020-05-16
3
TweetmemeBot/4.0
2021-02-24
3
Twingly Recon
2019-05-18
0
Twitterbot/1.0
2021-10-14
11
Twurly v1.1
2020-01-25
0
Typhoeus
2020-05-27
0
uipbot/1.0
2020-02-22
0
UniversalFeedParser/5.2.1
2021-02-01
2
Uptimebot/1.0
2019-06-27
177
Vagabondo/4.0
2020-06-18
7
Validator.nu/LV
2020-03-21
5
vebidoobot/1.0
2021-02-24
4
VelenPublicWebCrawler/1.0
2021-10-08
34
vuhuvBot/1.0
2021-10-17
4
W3C_Validator/1.3
2020-03-21
4
Wappalyzer
2019-11-03
0
WbSrch/1.1
2021-05-14
2
weborama-fetcher
2021-03-12
11
webtechbot
2020-06-14
0
WhizeBot
2021-01-14
0
woorankreview/2.0
2021-09-03
100
Wotbox/2.01
2019-10-06
0
XenForo/2.x
2020-12-30
0
XenForo/2.x
2021-10-07
2
Y!J-ASR/1.0
2020-03-10
0
yacybot
2021-04-22
1
Yahoo Ad monitoring
2021-02-24
0
Yahoo Link Preview
2020-12-22
1
Yahoo! Slurp
2021-10-05
40
Yahoo! Slurp/3.0
2021-01-17
0
YaK/1.0
2021-06-19
16
Yandex/1.01.001
2019-05-18
0
YandexAccessibilityBot/3.0
2021-10-06
21
YandexBot/3.0
2021-10-21
586
YandexFavicons/1.0
2021-10-16
29
YandexImages/3.0
2021-10-21
93
YandexMetrika/3.0
2021-02-24
0
YandexMobileBot/3.0
2021-10-07
40
YandexNews/4.0
2020-06-16
0
Yesslebot
2020-12-14
1
Yeti/1.1
2020-11-21
0
YioopBot
2020-08-21
0
YisouSpider
2021-09-19
160
ZoomBot
2020-06-16
2