Skip to content

Search engines

December 12, 2016

No surprise. Google is the number one in many aspects.

Looking in the numbers of September 2016, I can see: 3,828,299 pages scanned by the Google engine. At this number, you can add 909,422 more for mobile rendering. Then a total of just less than 5 millions pages scanned by Google on my website. I’m impressed.

The second crawler found on my website is… MJ12. I mean the Majestic Bot. Not a useful one, just a software which check for backlinks. Basically, there is no reason to accept it.

Then, you find Bing, the Microsoft search engine with 45,874 scanned pages! Too bad… Don’t be surprised to see so bad results in Bing when requesting it. They do not refresh their information. I suspect they scan more the “premium ones”. I mean, I do not challenge with some competitors like The Washington Post. Of course. I am just an obscure website. But, compared to Google, I found Bing not very fair. And, as you can expect, I do not receive visits from Bing.

Of course, I have more than 3 search engines but others are insignificant. Qwant, the french search engine has scanned about 1000 pages in 5 days, about 6,000 in a month. This is not ridiculous for this one because it is a fairly new search engine and I suspect they buy results from other famous search engines. But 6000 compared to 46000 (Bing), this is amazing. Because the newcomer could compete with a giant.

Note crawling the web means 2 things: having space to store the results. When it is text, it is quite simple. And a good bandwidth. Because scanning a page, it’s about 10 kilobytes. Scanning my website means for Google, loading about 500Mb a day. You can imagine the bandwidth you need to achieve this…

 

Advertisements

From → Webmaster

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: