|
Spider |
Hits |
1 |
Googlebot |
126,660 |
2 |
Mozilla/5.0 (compatible; AhrefsBot/7.0; http://ahrefs.com/robot/) |
77,037 |
3 |
Mozilla/5.0 (compatible; proximic; https://www.comscore.com/Web-Crawler) |
37,949 |
4 |
Google AdSense Robot |
33,948 |
5 |
Mozilla/5.0 (compatible; BLEXBot/1.0; http://webmeup-crawler.com/) |
29,449 |
6 |
'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Mobile Safari/537.36'Bytespider;https://zhanzhang.toutiao.com/ |
28,550 |
7 |
Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; http://www.grapeshot.co.uk/crawler.php) |
26,707 |
8 |
Yandex Robot |
22,420 |
9 |
CriteoBot/0.1 ( https://www.criteo.com/criteo-crawler/) |
15,640 |
10 |
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com) |
15,448 |
11 |
MJ12bot |
9,321 |
12 |
Mozilla/5.0 (compatible; Bytespider; spider-feedback@bytedance.com) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36 |
6,602 |
13 |
Sogou Spider |
6,431 |
14 |
Mozilla/5.0 (compatible; Barkrowler/0.9; https://babbar.tech/crawler) |
5,050 |
15 |
Mozilla/5.0 (compatible; SEOkicks; https://www.seokicks.de/robot.html) |
5,042 |
16 |
MaxPointCrawler/Nutch-1.17 (valassis.crawler at valassis dot com) |
3,747 |
17 |
peer39_crawler/1.0 |
1,434 |
18 |
YisouSpider |
1,192 |
19 |
netEstate NE Crawler ( http://www.website-datenbank.de/) |
1,021 |
20 |
Screaming Frog SEO Spider/17.1 |
937 |
21 |
Baidu Spider |
932 |
22 |
WebLog Expert |
604 |
23 |
Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; https://help.mail.ru/webmaster/indexing/robots) |
453 |
24 |
OWLer/0.1 (built with StormCrawler; https://ows.eu/owler; owl@ow-s.eu) |
445 |
25 |
WikiDo/1.1 (http://wikido.com; crawler@wikido.com) |
399 |
26 |
Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; https://zhanzhang.toutiao.com/) |
309 |
27 |
Yetibot |
254 |
28 |
Snap URL Preview Service; bot; snapchat; https://developers.snap.com/robots |
196 |
29 |
SeznamBot |
167 |
30 |
WPMU DEV Broken Link Checker Spider |
166 |
31 |
Mozilla/5.0 (compatible; VelenPublicWebCrawler/1.0; https://velen.io) |
146 |
32 |
IonCrawl (https://www.ionos.de/terms-gtc/faq-crawler-en/) |
135 |
33 |
EBIDAG/4.7 (EBID AG Crawler) Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36 |
100 |
34 |
Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Img/2.0; https://help.mail.ru/webmaster/indexing/robots) |
99 |
35 |
Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile/14E5239e YisouSpider/5.0 Safari/602.1 |
98 |
36 |
Perl LWP |
81 |
37 |
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.81 YisouSpider/5.0 Safari/537.36 |
74 |
38 |
BrightEdge Crawler/1.0 (crawler@brightedge.com) |
72 |
39 |
Mozilla/5.0 (compatible; ev-crawler/1.0; https://headline.com/legal/crawler) |
70 |
40 |
Mozilla/5.0 (compatible; Linespider/1.1; https://lin.ee/4dwXkTH) |
69 |
41 |
LightspeedSystemsCrawler Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US) |
69 |
42 |
Yahoo! Slurp |
56 |
43 |
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/602.4.8 (KHTML, like Gecko) Version/10.0.3 Safari/602.4.8 Spider |
48 |
44 |
Screaming Frog SEO Spider/18.1 |
48 |
45 |
Mozilla/5.0 (compatible) SemanticScholarBot ( https://www.semanticscholar.org/crawler) |
46 |
46 |
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider |
41 |
47 |
CheckMarkNetwork/1.0 ( http://www.checkmarknetwork.com/spider.html) |
41 |
48 |
Snap-URL-Preview (bot; snapchat; https://developers.snap.com/robots) |
40 |
49 |
IAS Crawler (ias_crawler; http://integralads.com/site-indexing-policy/) |
39 |
50 |
Screaming Frog SEO Spider/19.1 |
36 |
|
Subtotal |
459,918 |
|
Total |
460,207 |