I’m sick of it. Slurp is Yahoo’s piece of crap robot. It’s the one sucking more of your bandwidth than MSN or Google put together. Check your logs, you’ll see.
Why do people let them? Because of the promise of a bunch of targeted traffic from a “big search engine!” Woooo. BFD.
I’ve had it with that “search engine.” I’ve banned Slurp from chewing huge chunks of my bandwidth every month for nothing in return. If any other site did what they do, they’d be getting sued left and right and banned by everyone as a scraping spammer.
Their results are horrific. They always have been. They are one of the biggest reasons Google is so popular. Google is the antithesis of Yahoo in every way that matters.
Yahoo’s answer to competition is to immediately go in the opposite direction. In this case, Google loves blogs, so Yahoo and their “engineers” fabricate a brilliant plan: “Let’s hate blogs!”
In a study, my own official study, of over 1500 sites in our network, all blogs, Yahoo has the least to do with traffic numbers of all three engines. It ain’t just me getting ignored. If you have a blog, you are NOT getting the kind of traffic from Yahoo that Google is probably sending you.
Content be damned!
“Google likes content, so we hate it” seems to be the prevailing wisdom over there.
If the Yahoos at Yahoo are bending you over each month for nothing, not even the most obvious rankings for your site, do what I did:
User-agent: slurp
disallow: /
That’s the best search engine optimization trick you’ll learn all week. Get rid of the dead weight. They don’t want people reading content any place else but on Yahoo.com – YOU are their competition, why should they reward you when they are trying to act like they never meant to be a search engine in the first place?
You can’t fire me, I quit!
Update 6/6/07
Now I am really puzzled. Some readers have done some searches today to see if I really rank that poorly in Yahoo.
Check out what they found (And remember that Yahoo has only sent me 262 visitors this month.)
Blog Marketing Tips #3 – #4
Go figure. You’d think I’d be rolling in visitors and knocking Google because I don’t have the same rankings over there. But if the traffic isn’t there, it isn’t there.