List of webbots

Download without IP addresses XML format JSON format CSV format

Download including IP addresses XML format JSON format CSV format

Bot name
Last detected
IPs known
 
1M7bot/1.0
2021-02-23
0
1st ZipCommander
2021-03-23
0
2ip bot/1.1
2022-09-01
0
360Spider
0
3w24bot/1.2
0
3w24bot/1.3
0
3w24bot/1.4
0
7Siters/1.08
0
AdAuth/1.0
2023-01-18
1
adidxbot/2.0
0
admantx-adform/3.1
2022-07-18
5
admantx-eusyncbatch01/3.1
0
admantx-sap/2.4
0
admantx-usaspb/3.1
0
admantx-ussy04/3.1
0
admantx-ussy04/3.2
2023-01-26
17
AdsBot-Adsrvr
0
AdsBot-Google
2022-11-18
0
AdsBot-Google-Mobile
2021-12-17
2
Adsbot/3.1
2021-03-18
11
adscanner
0
adscanner/1.0
0
adscanner/1.0
2022-05-11
4
adscanner/1.1
0
AdsTxtCrawler/1.0
2023-01-20
70
AdsTxtCrawler/1.0.2
2023-01-28
54
AdsTxtCrawlerTP/1.2
2023-01-27
1
AffiliateLabz/1.0
0
AhrefsBot/2.0
2021-02-04
0
AhrefsBot/5.2
0
AhrefsBot/6.1
2021-03-11
157
AhrefsBot/7.0
2023-01-28
643
aiHitBot/2.9
2023-01-15
3
AlexandriaOrgBot/1.0
2022-02-11
0
AlphaBot/3.2
0
Amazon-Advertising-ad-standards-bot/1.0
2023-01-27
1
AmazonAdBot/1.0
2023-01-28
18
Amazonbot/0.1
2022-08-20
9
AnyEvent-HTTP/2.22
2022-01-16
0
APIs-Google
2022-05-03
0
AppEngine-Google
2022-09-13
34
Applebot/0.1
2023-01-28
1733
archive.org_bot
2022-10-16
34
AteniumBot
0
AwarioBot/1.0
2023-01-28
11
AwarioRssBot/1.0
2023-01-02
1
AwarioSmartBot/1.0
2022-04-23
3
Baidu Japan
0
Baiduspider
0
Baiduspider-image+
0
Baiduspider-render/2.0
2023-01-27
605
Baiduspider/2.0
2023-01-28
677
BananaBot/0.6.1
0
Barkrowler/0.9
0
Barkrowler/0.9
2023-01-24
270
bidswitchbot/1.0
2023-01-28
47
Bidtellect/0.0.1107.0
2023-01-09
0
Bidtellect/0.0.1125.0
2023-01-20
1
Bigsearch.ca/Nutch-1.0-dev
0
bingadsbot
2022-06-04
0
bingbot/2.0
2023-01-28
1541
BingPreview/1.0b
2023-01-28
1039
Birdcrawlerbot/0.5
2021-09-12
0
bitlybot/3.0
0
BLEXBot/1.0
2023-01-27
50
BlogBeatbot/2.5
2022-02-14
0
Bloglines/3.1
2022-05-29
0
bnf.fr_bot
2021-10-30
0
BorneoBot/0.7.1
0
BorneoBot/0.9.1
0
BUbiNG
0
BuiltWith/1.0
0
BuiltWith/1.1
0
bumb.ly/0.01
0
BW/1.1
2023-01-24
10
Bytespider
2023-01-04
44
CATExplorador/1.0beta
2023-01-24
10
CCBot/2.0
2023-01-26
305
CensysInspect/1.1
2023-01-27
179
centuryb.o.t9
2022-09-20
4
CheckMarkNetwork/1.0
2023-01-25
42
Cincraw/1.0
2022-10-31
5
CipaCrawler/3.0
0
CISPA Web Analyzer
2021-09-15
0
CISPA Webcrawler
0
Clarabot/1.4
0
Cliqzbot/2.0
0
Cliqzbot/3.0
0
CloudFlare-AlwaysOnline/1.0
0
CMS Crawler
2021-01-29
1
coccocbot-image/1.0
2022-07-17
36
coccocbot-web/1.0
2023-01-28
144
Cocolyzebot/1.0
2022-08-24
33
colly
2022-12-30
32
companyspotter/0.0.0.0
2022-09-17
0
companyspotter/0.1984.82.1324
2022-01-04
0
companyspotter/1.0.0.0
2023-01-11
1
contxbot/1.
2022-02-01
5
crawler_eb_germany_2.0
2023-01-17
1
Crawling at Home Project
2021-08-13
0
Crawlson/1.0
2023-01-09
30
CriteoBot/0.1
2023-01-28
130
CSIRT.CZ DNS crawler
2021-02-11
0
CSS Certificate Spider
0
CyotekWebCopy/1.9
2022-12-09
0
CZ.NIC DNS crawler
0
DataForSeoBot/1.0
2023-01-27
5
datagnionbot
0
Dataprovider.com
2023-01-27
208
DataXu/1.0
2022-10-03
12
deepnoc
2022-01-05
0
DF Bot 1.0
2022-09-14
2
Diffbot/0.1
2022-08-25
3
Discordbot/2.0
2022-10-15
3
DnBCrawler-Analytics
2023-01-18
1
dns-crawler/1.4.10.dev6
0
dns-crawler/1.4.10.dev7
0
dns-crawler/1.4.5
0
dns-crawler/1.4.6
0
dns-crawler/1.4.6.dev3
0
dns-crawler/1.4.8
0
DNSResearchBot/2.1
2021-04-25
1
DomainCrawler/3.0
0
Domains Project
0
Domains Project/1.0.1
0
Domains Project/1.0.3
0
Domains Project/1.0.4
0
Domains Project/1.0.6
0
Domains Project/1.0.8
0
Domains Project/1.1.0
0
Domains Project/1.1.1
0
Domains Project/1.1.4
0
Domains Project/1.2.1
0
Domains Project/1.2.3
0
Domains Project/1.2.4
0
Domains Project/1.2.7
0
Domains Project/1.3.2
0
Domains Project/1.3.7
2023-01-11
1
DomainSigmaCrawler/0.1
0
DomainStatsBot/1.0
2023-01-27
4
DomCopBot
2022-10-30
1
DotBot/1.1
2022-11-19
70
DotBot/1.2
2023-01-28
9
DuckDuckBot-Https/1.1
2021-11-11
9
DuckDuckBot/1.0
0
DuckDuckBot/1.1
2021-05-16
0
DuckDuckGo-Favicons-Bot/1.0
2023-01-28
2
E. Orliac, G. Fourestey/1.0
0
eContext/1.0
0
Elisabot
2022-10-27
6
EmailWolf 1.00
2022-08-26
0
Embedly/0.2
2021-06-17
0
ev-crawler/1.0
2023-01-26
4
evc-batch/2.0
2022-02-11
2
Exabot/3.0
2022-01-09
1
facebookexternalhit/1.0
2023-01-12
19
facebookexternalhit/1.1
2023-01-28
655
facebookexternalhit/1.2
2022-04-06
0
facebookscraper/1.0
0
FastCrawler/4.9
0
FatBoyKimComBot
0
Faveeo/1.0
0
Feedfetcher-Google
2022-11-07
1
Feedly/1.0
2023-01-28
7
FeedlyBot/1.0
0
feedparser/6.0.10
2022-09-10
0
FemtosearchBot/1.0
2023-01-16
4
FindITAnswersbot/1.0
2021-03-23
0
FirmoGraph
2023-01-28
1
FlipboardProxy/1.2
2021-06-18
1
FullStoryBot/1.0
2022-09-27
0
Gaisbot/3.0
2022-06-20
1
GarlikCrawler/1.2
0
GeedoBot
2023-01-27
1
GigablastOpenSource/1.0
2022-12-21
5
Gigabot
2021-02-12
1
Gigabot/1.1.1.2
0
Gluten Free Crawler/1.0
0
go-resty/2.0.0
0
GoodBot
2021-04-19
0
Google-adstxt
2023-01-28
111
Google-Adwords-Instant
2022-09-28
5
Google-Display-Ads-Bot
2023-01-24
2
Google-PageRenderer
2023-01-13
4
Google-Read-Aloud
2023-01-28
180
Googlebot-Image/1.0
2023-01-28
595
Googlebot-Mobile/2.1
2022-03-15
0
Googlebot/2.1
2023-01-28
3004
got
2023-01-24
52
got/9.6.0
2021-12-15
0
Grahambot/0.1
0
Grammarly/1.0
2021-08-03
3
GrapeshotCrawler/2.0
2023-01-28
16
Graydon bot/1.0
2022-04-15
1
Gregarius/0.5.2
2022-10-26
1
Grover/Grover-1.18
2021-09-15
0
GumGum-Bot/1.0
2022-03-04
0
heritrix/3.3.0
2022-09-29
0
heritrix/3.4.0
0
heritrix/3.4.0
0
HTTP Banner Detection
2023-01-21
10
HuaweiWebCatBot/6.0
0
HubSpot Connect/2.0
0
HubSpot Crawler
2022-03-27
0
HubSpot Webcrawler
2022-12-29
6
hypestat/1.0
0
ia_archiver
2022-01-30
5
ia_archiver/1.0
0
IAS crawler
2022-12-11
4
ias-au/3.1
2023-01-26
4
ias-ir/3.1
2023-01-28
208
ias-jp/3.1
2023-01-28
50
ias-or/3.1
2023-01-28
157
ias-sg/3.1
2023-01-28
78
ias-va/3.1
2023-01-28
341
IDG/EU
2023-01-18
1
Iframely/1.3.1
2023-01-26
9
Iframely/1.6.1
2022-03-25
0
IndeedBot 1.1
2021-10-24
5
INETDEX-BOT/1.5
2023-01-28
2
InfoTigerBot/1.9
2021-11-28
0
InoopaBot/1.0
0
intelx.io_bot
2023-01-16
1
InternetMeasurement/1.0
2023-01-28
58
IonCrawl
2023-01-26
45
istellabot/t.1.13
0
ivre-masscan/1.3
2021-05-12
0
Jaunt/1.5
0
JobboerseBot
0
Jooblebot/2.0
0
K7MLWCBot/1.0
0
KauaiSlowBot
2021-05-24
0
Keybot Translation-Search-Machine
2022-12-03
3
Khazandebot/2.1
0
KOCMOHABT
2022-11-25
37
KomodiaBot/1.0
2022-07-02
11
KStandBot/1.0
2022-08-09
0
lanaibot
2021-05-18
0
LanaiBotmarch
2021-03-19
0
LightspeedSystemsCrawler
2022-12-13
3
Linespider/1.1
2023-01-22
45
Linguee Bot
2021-05-07
8
LinkCityChecker/1.0
2021-11-28
0
linkdexbot/2.0
2023-01-27
6
LinkedInBot/1.0
2023-01-25
4
LinkpadBot/1.12
0
LinkpadBot/2.3
0
LinkWalker/3.0
2021-06-03
1
LivelapBot/0.2
2022-11-21
2
Lkx-TraversalHttpPlugin/0.0.1
2021-11-14
0
lkxscan/v0.1.0
2021-05-13
0
loli_spider
2022-08-31
0
looid.com Search Engine/0.1
0
ltx71
2022-03-26
5
Mail.RU_Bot/2.0
2023-01-28
177
Mail.RU_Bot/Fast/2.0
2021-09-08
0
Mail.RU_Bot/Favicons/2.0
2023-01-24
14
Mail.RU_Bot/Img/2.0
2021-07-29
3
Mail.RU_Bot/Robots/2.0
2023-01-27
24
Mappy/1.0
0
MarketingMiner v2.0
2022-12-13
0
masscan-ng/1.3
2022-12-21
0
masscan/1.0
2023-01-26
47
masscan/1.3
2023-01-28
93
MauiBot
2021-06-07
15
MauiBot
0
MBCrawler/1.0
2023-01-28
4
MediaMathbot/1.0
2023-01-28
36
Mediapartners-Google
2023-01-28
398
Mediatoolkitbot
2022-03-07
4
meg/0.2
0
MegaIndex.ru/2.0
2023-01-27
14
MetaInspector/5.6.0
0
Microsoft.Data.Mashup
2022-09-21
0
MicrosoftPreview/2.0
2022-10-19
0
MixnodeCache/1.8
2021-11-22
1
MixrankBot
2023-01-23
1
MJ12bot/v1.2.4
2022-05-08
0
MJ12bot/v1.4.1
2022-04-17
0
MJ12bot/v1.4.2
2022-12-15
0
MJ12bot/v1.4.7
0
MJ12bot/v1.4.8
2023-01-28
877
MojeekBot/0.10
2022-05-06
1
MojeekBot/0.11
2023-01-26
1
MojeekBot/0.6
0
MojeekBot/0.7
0
MojeekBot/0.9
0
MonitorBacklinksWP
2022-06-27
1
MRGbot/1.0
2022-07-21
0
msnbot-media/1.1
2023-01-26
2
msnbot/1.0
2022-11-26
2
msnbot/1.1
0
msnbot/2.0b
0
MTRobot/0.2
2021-04-10
7
MuckRack/1.0
0
MxToolbox/Beta7
2023-01-14
1
My Nutch Spider/Nutch-1.18
2021-10-20
0
NameFo
0
Neevabot/1.0
2023-01-27
2
NetcraftSurveyAgent/1.0
2022-12-07
0
netEstate NE Crawler
2023-01-22
1
NetNewsWire/2.x
2021-04-16
0
NetpeakCheckerBot/3.2
0
NetpeakCheckerBot/3.6
2022-02-01
0
Nicecrawler/1.1
2023-01-25
1
Nimbostratus-Bot/v1.3.2
2021-07-02
36
niocBot
0
niocBot
0
Nmap Scripting Engine
0
node-fetch/1.0
2022-12-30
82
oBot/2.3.1
2023-01-26
36
olbicobot/Nutch-1.15-SNAPSHOT
0
Onespot-ScraperBot/1.0
0
Orbbot/1.1
2023-01-27
3
PagePeeker/3.0
2021-09-25
2
PageThing
2021-07-05
6
Pandalytics/1.0
2022-12-23
52
panscient
2022-12-17
24
Panscient/1.0
0
PaperLiBot/2.1
2021-06-18
0
Paracrawl
2021-02-27
0
PartyflockLinkChecker/0.3
0
PATHspider
2023-01-28
1
PetalBot
2023-01-28
2616
PhantomBot/1.0
2023-01-27
1
Picoweb_Bot
0
Pinterestbot/1.0
2023-01-28
2
Pixalate-Crawler
2022-03-29
0
Plagger/0.x.xx
2021-05-03
0
Plukkie/1.7
2021-11-06
0
pmoz.info ODP link checker
2022-10-24
1
Protopage/3.0
2021-10-15
0
proximic
2023-01-28
7252
psbot/0.1
0
Quantcastbot/1.0
2023-01-17
1
Qwantbot/1.0
0
Qwantify/1.0
2023-01-28
20
Qwantify/2.4w
2022-01-14
40
Qwantify/Bleriot/1.1
0
Qwantify/Bleriot/1.2.1
0
Qwarrybot/2.0
2022-01-14
0
Re-re Studio
0
RepoLookoutBot/v1.1.0
2023-01-25
2
req/v3
2022-09-11
0
Researchscan/t12ca
0
Researchscan/t12sns
0
Researchscan/t13rl
0
Riddler
2021-06-28
1
RobotsTxtParser-VIPnytt/2.1
2022-01-12
0
rogerbot/1.2
2021-07-13
0
RSSingBot
0
RukiCrawler/1.0
2021-05-25
0
RyowlEngine/1.0
0
SabsimBot/3.0
2022-02-25
1
SafeDNSBot
0
Scrapy/1.4.0
0
Scrapy/1.5.0
2022-09-11
1
Scrapy/1.5.1
0
Scrapy/1.5.2
0
Scrapy/1.6.0
0
Scrapy/1.7.2
0
Scrapy/1.7.3
2023-01-27
1
Scrapy/1.7.4
2022-10-05
0
Scrapy/1.8.0
0
Scrapy/2.0.1
0
Scrapy/2.2.1
2021-04-27
2
Scrapy/2.3.0
2021-02-04
0
Scrapy/2.4.0
2021-12-06
1
Scrapy/2.4.1
2021-07-28
0
Scrapy/2.5.1
2021-11-15
0
Scrapy/2.6.1
2022-03-30
0
Scrapy/2.6.2
2022-09-11
0
Scrapy/2.7.1
2023-01-25
1
Screaming Frog SEO Spider/10.1
0
Screaming Frog SEO Spider/11.2
0
Screaming Frog SEO Spider/11.3
0
Screaming Frog SEO Spider/12.2
0
Screaming Frog SEO Spider/12.3
0
Screaming Frog SEO Spider/12.6
2021-02-06
1
Screaming Frog SEO Spider/13.2
2021-04-25
1
Screaming Frog SEO Spider/14.1
2021-09-18
2
Screaming Frog SEO Spider/14.2
2021-04-01
0
Screaming Frog SEO Spider/14.3
2021-03-21
0
Screaming Frog SEO Spider/15.2
2021-06-08
0
Screaming Frog SEO Spider/16.2
2021-11-16
0
Screaming Frog SEO Spider/16.3
2021-12-06
0
Screaming Frog SEO Spider/16.4
2021-12-21
0
Screaming Frog SEO Spider/16.5
2022-02-08
0
Screaming Frog SEO Spider/16.7
2022-03-12
0
Screaming Frog SEO Spider/17.2
2022-10-25
0
Screaming Frog SEO Spider/18.1
2022-12-22
0
Screaming Frog SEO Spider/5.1
0
Screaming Frog SEO Spider/6.2
0
Screaming Frog SEO Spider/9.4
2022-02-12
0
SecSpider/1.0
0
Seekport Crawler
2022-06-08
258
SeekportBot
2023-01-27
8
Semanticbot/1.0
2022-03-08
1
SemanticJuice
2021-10-25
0
SemanticScholarBot
2021-02-19
1
SemrushBot
2023-01-26
5
SemrushBot-BA
2023-01-28
13
SemrushBot-BM/1.0
2021-09-05
0
SemrushBot-SA/0.97
2021-06-07
3
SemrushBot-SI/0.97
2021-07-30
1
SemrushBot/3~bl
0
SemrushBot/6~bl
2022-12-06
1
SemrushBot/7~bl
2023-01-28
80
SemrushBot/7~bl-slave
2022-01-02
0
SEO Scanner/1.2.4
2022-05-11
0
SEOkicks
2023-01-21
16
SEOlizer/1.1
2022-08-10
1
seolyt/1.1
2021-03-20
0
serpstatbot/1.0
2021-11-09
9
serpstatbot/2.0 beta
2021-05-18
9
serpstatbot/2.1
2023-01-17
2
SERPtimizerBot
2022-02-02
0
SeznamBot/3.2
2023-01-26
96
SeznamBot/3.2-test1
0
SeznamBot/4.0-RC1
2023-01-28
53
Sidetrade indexer bot
2022-12-02
1
SiteAnalyzerbot/2.0
0
SiteAuditBot/0.97
2022-03-03
0
SiteCheckerBotCrawler/1.0
0
SiteExplorer/1.1b
0
Slackbot 1.0
2023-01-03
21
Slackbot-LinkExpanding 1.0
2023-01-03
25
SMTBot/1.0
2022-09-15
24
Snap URL Preview Service
2023-01-13
2
Snarfer/0.x.x
2021-04-14
0
SniffRSS/0.5beta
0
Sogou head spider/3.0
2022-10-20
0
Sogou web spider/4.0
2023-01-28
373
special_archiver/3.1.1
2022-09-19
26
SpiderLing
2023-01-11
2
Squirrel-spider/2.0
2022-01-26
0
startmebot/1.0
2023-01-27
680
Startpagina-Linkchecker/1.0
2023-01-28
4
Streamline3Bot/1.0
0
SurdotlyBot/1.0
2023-01-23
121
t3versionsBot/1.0
2021-12-30
0
TBI-HttpOpenPlugin/0.0.2
0
TBI-HttpOpenPlugin/0.1.0
2021-02-13
4
TBI-WebScanner/0.0.1
2021-02-13
3
tchelebi/1.0
2023-01-18
2
The Knowledge AI
0
Thither.Direct Bot/0.0.1
0
TinEye-bot/1.31
2022-12-17
0
TinyBotTestUA
2021-07-16
0
TOBBOT
0
TombaPublicWebCrawler/3.5
2022-06-01
0
tracemyfile/1.0
0
trendictionbot/0.5.0
2023-01-17
5
TTD-Content
2023-01-28
1145
Turnitin
2023-01-17
3
TurnitinBot
0
TweetmemeBot/4.0
2021-02-24
1
Twingly Recon
0
Twingly Recon-Klondike/1.0
2023-01-01
0
Twitterbot/1.0
2023-01-27
11
Twurly v1.1
0
Typhoeus
2022-11-15
1
uipbot/1.0
0
UniversalFeedParser/5.2.1
2021-02-01
1
Uptimebot/1.0
0
URIBOT/1.1
2023-01-13
1
Vagabondo/4.0
0
Validator.nu/LV
0
vebidoobot/1.0
2021-02-24
1
VelenPublicWebCrawler/1.0
2023-01-24
39
VsuSearchSpider/1.0
2021-10-22
0
vuhuvBot/1.0
2022-12-16
1
W3C_Validator/1.3
0
Wappalyzer
0
WbSrch/1.1
2021-05-14
1
WbSrch/1.2
2021-03-24
0
WebDataStats/1.0
2021-04-04
0
webgains-bot
2021-05-06
0
weborama-fetcher
2022-05-29
9
webprosbot/2.0
2023-01-18
2
webtechbot
0
WellKnownBot/0.1
2023-01-28
3
WhatStuffWhereBot
2022-09-20
0
WhizeBot
0
woorankreview/2.0
2022-12-30
40
Wotbox/2.01
0
WP-URLDetails/6.1.1-alpha-54813
2022-12-23
0
XenForo/2.x
0
XenForo/2.x
2021-10-07
2
Y!J-ASR/1.0
0
yacybot
2023-01-27
2
Yahoo Ad monitoring
2022-11-13
0
Yahoo Link Preview
2022-12-09
0
Yahoo! Slurp
2023-01-12
44
Yahoo! Slurp China
2023-01-26
1
Yahoo! Slurp/3.0
0
YahooMailProxy
2022-04-26
0
YaK/1.0
2023-01-16
8
Yandex/1.01.001
0
YandexAccessibilityBot/3.0
2022-02-17
15
YandexBot/3.0
2023-01-28
888
YandexFavicons/1.0
2023-01-28
94
YandexImages/3.0
2023-01-27
164
YandexMetrika/3.0
2021-02-24
0
YandexMobileBot/3.0
2022-02-17
20
YandexNews/4.0
2022-10-24
1
Yesslebot
0
Yeti/1.1
2022-12-07
14
YioopBot
0
YisouSpider
2023-01-24
261
ZaldamoSearchBot
2022-06-30
0
zgrab/0.x
2023-01-28
143
zgtemple-bot
2021-12-22
0
ZoomBot
2022-09-10
1
ZoominfoBot
2023-01-28
17