Tuesday, August 28, 2007
Pay-per-crawl
One of the first things I understood after I created this blog was that Google is using their power (position) in a very insane way. To say in their words, they are being a little evil.
People think Google indexes the web, but really, they don't make too much (after they created the brand and the PageRank algorithm). Some studies say 30% of the web isn't indexed. That is, 1 of 3 pages you are looking for, is not present in any search engine.
Believe me, you have to work hard just to enter in a search engine (one month or more from your submission), and then, if some day they decide to include your website, you have to hire a SEO to make Google work properly, and appear in the first page when the user type your name... Google should be indexing the web in the right way, not you (by hiring a SEO for them), or Google should pay for it.
It is a vicious circle, you pay a SEO and Google get more people working for them to get the top-5 positions for the "britney spears" search entry. Then, that word becomes more competitive and you need another SEO to keep your position. It is ironic, everybody doing the Google's work while they give 20% to their employees to create Orkut for Brazilians and play ping-pong ;)
But if it is not enough, Google consumes your bandwidth (money) every time they crawl your site. Ok, you may think it isn't too much waste, they are just using a little part of your bandwidth... but, what would you think if 1 million of companies start crawling the web and your site, just like Google does. Clearly, that is different, because the abuse becomes obvious. On the other hand, you can't use a robot to consume their money (i.e. writing a program to auto-click adSense advertisements) because they may sue you. So, why can they waste my money but I can't waste theirs? Do you think it is fair? I know, I can write a robot.txt file to avoid the crawler, but why should I waste my time for them? Why are they assuming they can use my bandwidth?
I think it will be a problem in the long run. But don't panic, it won't happen for a while. Some day, if everything continues as it goes, people will notice that Google is not producing accurate results for their searches, because SEO's are manipulating them, and a fairer search engine will emerge. Google is becoming a advertised menu, and I don't want to search stuff there.
Let be sincere, Google is not producing the web, they are just getting money from your production and using your resources to do that! It could be a fair model when they were a start-up, but nowadays, that they are earning billions of dollars with your customers and your bandwidth, and you have to outsource a SEO for them, I think it can be considered evil, don't you?.
I can think in two solutions to this problem on the top of my mind: Google should pay-per-crawl your site, and they should crawl your site only if you specifically allow them (crawler should be disabled by default). But I will keep thinking, in Internet you always can do something to avoid the use of the power.
What do you think? May be, I am the only one thinking that this model is wrong.
Labels: Crawlers, Google, Internet, Technology
Wednesday, May 23, 2007
Himalia first result in Yahoo for: Model driven user interfaces
Himalia is being retrieved in the first place at least for the following search keywords in Yahoo:
- . model driven user interfaces
- . model driven user interface
- . model driven uis
This is the fist time in the history of the World ;) that any Search Engine retrieves Himalia as the first one in the result list for a search keyword not including: "Himalia", "Leonardo Vernazza" or "leovernazza". I always said that the Yahoo search engine was moving faster that the Google one... ;)
Let me know if there is any new keyword throwing traffic to Himalia.
