List of webbots

Download without IP addresses XML format JSON format CSV format

Download including IP addresses XML format JSON format CSV format

Bot name
Last detected
IPs known
 
1M7bot/1.0
2021-02-23
0
360Spider
2021-03-15
1
3w24bot/1.2
2019-07-16
0
3w24bot/1.3
2019-07-12
0
3w24bot/1.4
2020-03-29
0
7Siters/1.08
2019-07-10
0
AdAuth/1.0
2021-04-08
2
adidxbot/2.0
2020-06-01
0
admantx-adform/3.1
2021-05-05
24
admantx-eusyncbatch01/3.1
2020-12-18
2
admantx-sap/2.4
2019-07-13
0
admantx-usaspb/3.1
2020-03-15
0
admantx-ussy04/3.1
2020-10-18
2
admantx-ussy04/3.2
2020-12-01
3
AdsBot-Adsrvr
2019-10-29
1
AdsBot-Google
2019-12-27
0
AdsBot-Google-Mobile
2021-01-29
0
Adsbot/3.1
2021-03-15
29
adscanner
2019-09-06
2
adscanner/1.0
2021-04-15
4
adscanner/1.0
2019-10-02
4
adscanner/1.1
2020-03-17
4
AdsTxtCrawler/1.0
2021-05-04
7
AffiliateLabz/1.0
2020-02-05
0
AhrefsBot/2.0
2021-02-04
0
AhrefsBot/5.2
2019-05-26
3
AhrefsBot/6.1
2021-03-11
680
AhrefsBot/7.0
2021-05-13
470
aiHitBot/2.9
2021-04-11
4
AlphaBot/3.2
2019-11-06
0
Amazonbot/0.1
2021-05-01
6
AppEngine-Google
2021-05-09
11
Applebot/0.1
2021-05-13
835
archive.org_bot
2021-02-24
1
archive.org_bot
2019-09-08
2
AteniumBot
2020-12-17
1
AwarioRssBot/1.0
2020-08-14
2
AwarioSmartBot/1.0
2020-11-24
0
Baidu Japan
2019-05-18
0
Baiduspider
2020-07-18
0
Baiduspider-image+
2019-10-24
42
Baiduspider-render/2.0
2021-05-11
453
Baiduspider/2.0
2021-05-12
663
BananaBot/0.6.1
2020-07-11
0
Barkrowler/0.9
2019-07-04
8
Barkrowler/0.9
2021-05-11
18
bidswitchbot/1.0
2021-05-12
41
Bigsearch.ca/Nutch-1.0-dev
2019-05-18
0
bingbot/2.0
2021-05-13
1781
BingPreview/1.0b
2021-05-13
1512
bitlybot/3.0
2020-01-25
1
BLEXBot/1.0
2021-05-13
76
BorneoBot/0.7.1
2020-05-24
0
BorneoBot/0.9.1
2020-05-26
0
BUbiNG
2020-09-13
0
BuiltWith/1.0
2021-01-26
0
BuiltWith/1.1
2020-04-22
0
bumb.ly/0.01
2019-11-10
0
Bytespider
2021-05-09
338
CATExplorador/1.0beta
2021-05-01
11
CCBot/2.0
2021-05-13
378
CensysInspect/1.1
2021-05-13
43
centuryb.o.t9
2020-11-27
1
CheckMarkNetwork/1.0
2021-04-15
128
Cincraw/1.0
2021-05-07
3
CipaCrawler/3.0
2020-02-08
5
CISPA Webcrawler
2020-07-06
0
Clarabot/1.4
2020-04-18
10
Cliqzbot/2.0
2019-08-13
44
Cliqzbot/3.0
2020-07-12
228
CloudFlare-AlwaysOnline/1.0
2020-02-23
1
CMS Crawler
2021-01-29
1
coccocbot-image/1.0
2021-05-12
25
coccocbot-web/1.0
2021-05-09
40
Cocolyzebot/1.0
2021-05-12
36
colly
2020-05-16
0
contxbot/1.
2019-10-20
0
Crawlson/1.0
2021-03-02
18
CriteoBot/0.1
2021-05-13
21
CSIRT.CZ DNS crawler
2021-02-11
0
CSS Certificate Spider
2019-08-01
0
CZ.NIC DNS crawler
2021-01-14
6
datagnionbot
2021-01-27
1
Dataprovider.com
2021-05-07
87
DataXu/1.0
2021-05-12
8
DF Bot 1.0
2021-03-03
1
Diffbot/0.1
2020-11-06
8
Discordbot/2.0
2021-05-02
4
dns-crawler/1.4.10.dev6
2020-08-20
0
dns-crawler/1.4.10.dev7
2020-08-27
0
dns-crawler/1.4.5
2020-06-25
0
dns-crawler/1.4.6
2020-07-09
0
dns-crawler/1.4.6.dev3
2020-06-25
0
dns-crawler/1.4.8
2020-07-16
0
DNSResearchBot/2.1
2021-04-25
1
DomainCrawler/3.0
2019-08-26
10
Domains Project
2020-01-20
0
Domains Project/1.0.1
2020-02-07
0
Domains Project/1.0.3
2020-03-09
0
Domains Project/1.0.4
2020-04-14
0
Domains Project/1.0.6
2020-04-21
0
Domains Project/1.0.8
2020-06-21
0
Domains Project/1.1.0
2020-08-05
0
Domains Project/1.1.1
2020-08-07
0
Domains Project/1.1.4
2020-08-09
0
Domains Project/1.2.1
2020-09-19
0
Domains Project/1.2.3
2020-10-18
0
Domains Project/1.2.4
2020-11-05
0
Domains Project/1.2.7
2020-11-27
0
Domains Project/1.3.2
2021-01-10
0
DomainSigmaCrawler/0.1
2019-11-24
0
DomainStatsBot/1.0
2021-05-12
6
DomCopBot
2020-08-10
0
DotBot/1.1
2021-03-18
43
DuckDuckBot-Https/1.1
2021-05-13
8
DuckDuckBot/1.0
2020-01-03
0
DuckDuckGo-Favicons-Bot/1.0
2021-05-13
4
E. Orliac, G. Fourestey/1.0
2019-08-27
0
eContext/1.0
2019-10-30
0
Elisabot
2020-11-03
3
evc-batch/2.0
2021-05-13
4
Exabot/3.0
2019-08-04
3
facebookexternalhit/1.0
2020-10-14
9
facebookexternalhit/1.1
2021-05-13
783
facebookscraper/1.0
2020-07-03
0
FastCrawler/4.9
2019-06-05
0
FatBoyKimComBot
2019-11-02
0
Faveeo/1.0
2020-01-25
0
Feedfetcher-Google
2021-02-21
1
Feedly/1.0
2021-05-13
4
FeedlyBot/1.0
2020-01-10
1
FemtosearchBot/1.0
2020-06-23
5
FlipboardProxy/1.2
2020-01-25
0
Gaisbot/3.0
2020-08-25
0
GarlikCrawler/1.2
2020-11-05
1
GigablastOpenSource/1.0
2021-05-12
5
Gigabot
2021-02-12
1
Gigabot/1.1.1.2
2019-10-17
0
Gluten Free Crawler/1.0
2019-12-14
1
go-resty/2.0.0
2019-10-04
1
Google-Adwords-Instant
2021-02-04
4
Google-Read-Aloud
2021-05-09
8
Googlebot-Image/1.0
2021-05-12
588
Googlebot/2.1
2021-05-13
4590
got
2021-05-11
7
Grahambot/0.1
2019-05-18
0
Grammarly/1.0
2019-11-09
0
GrapeshotCrawler/2.0
2021-05-13
49
Graydon bot/1.0
2020-09-05
1
Gregarius/0.5.2
2020-08-27
0
heritrix/3.4.0
2021-01-27
0
heritrix/3.4.0
2020-07-20
0
HTTP Banner Detection
2021-05-07
4
HuaweiWebCatBot/6.0
2020-10-27
1
HubSpot Connect/2.0
2019-10-19
0
HubSpot Webcrawler
2019-11-28
0
hypestat/1.0
2020-03-31
0
ia_archiver
2020-11-13
13
ia_archiver/1.0
2020-04-08
0
IAS crawler
2021-04-25
6
ias-ir/3.1
2021-05-13
83
ias-jp/3.1
2021-01-22
0
ias-or/3.1
2021-05-08
67
ias-sg/3.1
2021-04-27
1
ias-va/3.1
2021-05-13
170
Iframely/1.3.1
2019-08-19
0
IndeedBot 1.1
2021-04-10
1
InoopaBot/1.0
2021-01-13
1
intelx.io_bot
2021-03-10
1
istellabot/t.1.13
2019-08-01
3
Jaunt/1.5
2019-11-16
0
JobboerseBot
2020-01-28
7
Jooblebot/2.0
2019-06-28
0
K7MLWCBot/1.0
2020-02-06
0
Keybot Translation-Search-Machine
2020-08-09
0
Khazandebot/2.1
2020-08-02
0
KOCMOHABT
2021-04-27
25
KomodiaBot/1.0
2021-05-03
12
LightspeedSystemsCrawler
2021-04-14
4
Linespider/1.1
2021-05-10
36
Linguee Bot
2021-05-07
12
linkdexbot/2.0
2021-05-05
5
LinkedInBot/1.0
2021-02-17
4
LinkpadBot/1.12
2019-07-28
1
LinkpadBot/2.3
2021-01-22
1
LinkWalker/3.0
2020-08-16
1
LivelapBot/0.2
2019-06-27
0
looid.com Search Engine/0.1
2019-08-30
1
ltx71
2021-05-11
6
Mail.RU_Bot/2.0
2021-05-13
158
Mail.RU_Bot/Favicons/2.0
2021-03-08
2
Mail.RU_Bot/Img/2.0
2020-01-20
13
Mail.RU_Bot/Robots/2.0
2021-05-08
17
Mappy/1.0
2020-01-07
1
masscan/1.0
2021-05-07
87
masscan/1.3
2021-04-20
3
MauiBot
2021-04-29
9
MauiBot
2019-09-09
2
MBCrawler/1.0
2021-05-12
2
Mediapartners-Google
2021-05-13
400
Mediatoolkitbot
2020-09-02
2
meg/0.2
2020-11-20
2
MegaIndex.ru/2.0
2021-05-10
9
MetaInspector/5.6.0
2019-12-05
1
MixnodeCache/1.8
2021-05-11
3
MJ12bot/v1.4.7
2019-07-20
2
MJ12bot/v1.4.8
2021-05-13
603
MojeekBot/0.10
2021-05-11
1
MojeekBot/0.6
2019-09-28
1
MojeekBot/0.7
2020-06-27
1
MojeekBot/0.9
2020-07-01
0
MonitorBacklinksWP
2021-02-07
0
msnbot-media/1.1
2020-09-23
0
msnbot/1.0
2020-08-06
0
msnbot/1.1
2020-04-06
0
msnbot/2.0b
2021-01-02
642
MTRobot/0.2
2021-04-10
7
MuckRack/1.0
2020-01-25
0
MxToolbox/Beta7
2020-02-02
0
NameFo
2020-01-12
0
netEstate NE Crawler
2021-05-10
3
NetpeakCheckerBot/3.2
2019-08-29
0
Nimbostratus-Bot/v1.3.2
2021-02-12
48
niocBot
2020-03-16
0
niocBot
2020-04-13
0
Nmap Scripting Engine
0
node-fetch/1.0
2021-05-11
64
oBot/2.3.1
2021-05-05
36
olbicobot/Nutch-1.15-SNAPSHOT
2020-10-26
2
Onespot-ScraperBot/1.0
2020-01-14
0
PagePeeker/3.0
2021-04-23
2
PageThing
2021-05-11
2
PageThing.com
2021-05-04
4
Pandalytics/1.0
2021-03-16
38
panscient
2021-03-22
5
Panscient/1.0
2019-12-15
0
PaperLiBot/2.1
2021-01-27
4
PartyflockLinkChecker/0.3
2019-10-30
1
PetalBot
2021-04-09
2240
Picoweb_Bot
2019-11-20
0
pmoz.info ODP link checker
2021-01-11
1
proximic
2021-05-13
2765
psbot/0.1
2020-01-17
0
Qwantbot/1.0
2020-10-14
2
Qwantify/2.4w
2021-05-04
81
Qwantify/Bleriot/1.1
2020-02-06
11
Qwantify/Bleriot/1.2.1
2019-10-03
0
Re-re Studio
2019-09-29
2
Researchscan/t12ca
2020-04-14
3
Researchscan/t12sns
2020-06-25
3
Researchscan/t13rl
2020-06-25
3
Riddler
2021-01-03
3
RSSingBot
2019-10-11
0
RyowlEngine/1.0
2020-01-22
0
SabsimBot/3.0
2020-11-24
0
SafeDNSBot
2021-01-27
6
Scrapy/1.4.0
2020-12-21
8
Scrapy/1.5.0
2020-12-12
3
Scrapy/1.5.1
2020-03-07
2
Scrapy/1.5.2
2019-07-18
1
Scrapy/1.6.0
2019-08-13
3
Scrapy/1.7.2
2020-11-11
4
Scrapy/1.8.0
2021-01-18
4
Scrapy/2.0.1
2020-04-24
0
Scrapy/2.2.1
2021-04-27
2
Scrapy/2.3.0
2021-02-04
0
Scrapy/2.4.0
2020-12-22
0
Screaming Frog SEO Spider/10.1
2020-01-09
0
Screaming Frog SEO Spider/11.2
2020-02-28
1
Screaming Frog SEO Spider/11.3
2019-10-02
0
Screaming Frog SEO Spider/12.2
2019-12-09
0
Screaming Frog SEO Spider/12.3
2019-12-15
0
Screaming Frog SEO Spider/12.6
2021-02-06
1
Screaming Frog SEO Spider/13.2
2021-04-25
2
Screaming Frog SEO Spider/14.1
2021-01-20
0
Screaming Frog SEO Spider/5.1
2020-04-20
0
Screaming Frog SEO Spider/6.2
2019-11-20
0
SecSpider/1.0
2020-11-05
0
Seekport Crawler
2021-05-09
158
Semanticbot/1.0
2020-09-02
0
SemanticScholarBot
2021-02-19
1
SemrushBot
2021-05-13
5
SemrushBot-BA
2021-05-12
27
SemrushBot-SA/0.97
2021-05-10
4
SemrushBot-SI/0.97
2019-09-30
0
SemrushBot/3~bl
2019-08-21
29
SemrushBot/6~bl
2021-01-18
70
SemrushBot/7~bl
2021-05-13
39
SEOkicks
2021-05-13
14
SEOlizer/1.1
2020-08-15
0
serpstatbot/1.0
2021-05-04
29
serpstatbot/2.0 beta
2021-05-12
13
SeznamBot/3.2
2021-05-12
144
SeznamBot/3.2-test1
2019-12-28
3
Sidetrade indexer bot
2021-05-02
1
SiteAnalyzerbot/2.0
2020-06-22
0
SiteCheckerBotCrawler/1.0
2019-10-16
0
SiteExplorer/1.1b
2019-05-20
1
Slackbot 1.0
2021-04-29
25
Slackbot-LinkExpanding 1.0
2021-04-29
47
SMTBot/1.0
2021-05-09
4
SniffRSS/0.5beta
2019-05-18
0
Sogou web spider/4.0
2021-05-13
499
special_archiver/3.1.1
2020-09-19
5
SpiderLing
2019-10-21
0
startmebot/1.0
2021-05-11
986
Startpagina-Linkchecker/1.0
2021-05-07
2
Streamline3Bot/1.0
2019-09-29
1
SurdotlyBot/1.0
2021-05-11
126
TBI-HttpOpenPlugin/0.0.2
2020-08-22
0
TBI-HttpOpenPlugin/0.1.0
2021-02-13
36
TBI-WebScanner/0.0.1
2021-02-13
39
tchelebi/1.0
2021-05-01
1
The Knowledge AI
2020-10-19
5
Thither.Direct Bot/0.0.1
2019-06-14
0
TinEye-bot/1.31
2019-12-10
5
TOBBOT
2021-01-13
1
tracemyfile/1.0
2020-03-10
1
trendictionbot/0.5.0
2021-04-27
3
TTD-Content
2021-05-10
125
Turnitin
2021-05-03
1
TurnitinBot
2020-05-16
3
TweetmemeBot/4.0
2021-02-24
3
Twingly Recon
2019-05-18
0
Twitterbot/1.0
2021-05-08
9
Twurly v1.1
2020-01-25
0
Typhoeus
2020-05-27
0
uipbot/1.0
2020-02-22
0
UniversalFeedParser/5.2.1
2021-02-01
2
Uptimebot/1.0
2019-06-27
177
Vagabondo/4.0
2020-06-18
7
Validator.nu/LV
2020-03-21
5
vebidoobot/1.0
2021-02-24
4
VelenPublicWebCrawler/1.0
2021-04-08
23
vuhuvBot/1.0
2021-05-04
4
W3C_Validator/1.3
2020-03-21
4
Wappalyzer
2019-11-03
0
WbSrch/1.1
2021-05-09
2
weborama-fetcher
2021-03-12
11
webtechbot
2020-06-14
0
WhizeBot
2021-01-14
0
woorankreview/2.0
2021-01-07
85
Wotbox/2.01
2019-10-06
0
XenForo/2.x
2020-12-30
0
XenForo/2.x
2021-02-22
1
Y!J-ASR/1.0
2020-03-10
0
yacybot
2021-04-22
1
Yahoo Ad monitoring
2021-02-24
0
Yahoo Link Preview
2020-12-22
1
Yahoo! Slurp
2021-04-19
25
Yahoo! Slurp/3.0
2021-01-17
0
YaK/1.0
2021-05-10
16
Yandex/1.01.001
2019-05-18
0
YandexAccessibilityBot/3.0
2021-05-13
17
YandexBot/3.0
2021-05-13
435
YandexFavicons/1.0
2021-05-10
6
YandexImages/3.0
2021-05-13
50
YandexMetrika/3.0
2021-02-24
0
YandexMobileBot/3.0
2020-09-10
34
YandexNews/4.0
2020-06-16
0
Yesslebot
2020-12-14
1
Yeti/1.1
2020-11-21
0
YioopBot
2020-08-21
0
YisouSpider
2021-05-10
93
ZoomBot
2020-06-16
2