"The Search" an insiders response

The last chapter has not been written in the search game. Microsoft MSN Search is every bit as good as Google in terms of size, speed, and relevance. Microsoft has come from behind several times before. I wouldn't bet against them now. MSN Search is just getting started.

The Search by John Battelle is an accurate look at the 'Search Wars" of the late 90's. I was at AltaVista at the time. John got the facts correct, but there is more to the story...there always is. The book really proves the point of my previous post, "Innovate or Imitate".  AltaVista, Excite, and GoTo.com failed because of business and strategy decisions, not because of inferior technology.

These companies were first movers with excellent technology. AltaVista was the first real search engine, Yahoo was the first directory, Excite was the first search engine / portal, and GoTo.com was the first pay per click (PPC) search site.

The AltaVista experience is sad to remember. We should have been the "Google" of today. We were pure search, no frills, no consumer portal crap. DEC is guilty of neglect in its handling of AltaVista. Compaq put a bunch of PC guys in charge who relied on McKinsey consultants and copied AOL, Excite, Yahoo and Lycos into the consumer portal game. It should have been clear that being the 5th or 6th player in the consumer portal business wouldn't work. AltaVista spent hundreds of millions on acquisitions that never worked, and spent $100M on a brand advertising campaign. They spent NOTHING to improve core search. That was the undoing of AltaVista.

You need to remember the context of the time. It seemed like every week AOL was announcing a $50M deal to sell traffic. Yahoo was doing it too. The game was build traffic with search, keep them on your site with content, and sell traffic and "screen real estate" to sponsors for $20-$40M a pop up front. There was no proven search business model other than annoying banner ads that were not really contextual.

GoTo.com invented the paid search model where businesses bid against each other for certain search words. I remember the self righteous uproar at the time. Polluting search results with paid advertising masquerading as search results was blasphemous...UN-American. Well, it turned out to be a great idea, and users were smart enough to distinguish between regular results and paid results. In fact, they learned to like the paid results because they were contextual. Had GoTo.com stayed on course with their search destination site they may have emerged the winner.

Both AltaVista and GoTo.com (later called Overture) were in a perfect position to be the Google of today. Both made strategic mistakes that ultimately killed them.

Google deserves a lot of credit. They did four major things right. First, the PageRank algorithm was far superior to any other ranking/relevance method, and still is. Second, their use of cheap, scalable PCs and storage put them at a serious cost advantage over everyone else. Third, their implementation of paid search (AdWords) was simple, elegant, and very effective. Fourth, and maybe most importantly, they stayed the course on focused on search, while everyone else was trying to copy AOL and become a consumer portal. Great decisions.

The last chapter has not been written in the search game. There are many more decisions to be made. There is a long history of "first movers" being eclipsed by "fast followers".

Comments

[Gary]

MSN Search was late to the game but I agree that betting against them is ill advised.

Google has a major advantage that people have been using them as the de facto standard for searching. If you ask a person to look something up on the web they will almost always pull up Google or secondly Yahoo. You hear "Google It" becoming the generic phrase or brand name for performing a search. Google has deeply embedded themselves on websites and blogs everywhere through their AdSense program. Google has to be given credit for this.

It would be interesting to know where MSN Search gets most of their traffic from, what are the referring URLs. Is the traffic coming from MSN.com or is it coming from random URLs. If the traffic is coming from random URLs that would be a good indicator that people are seeking out MSN Search to perform searches but if the referring URLs are MSN.com or MSN.com sub-domains I would venture a guess that people are using MSN Search because it's in their face, it's on their browsers default home page. MS may win the people that have MSN as their home page but they need to find a way to get people to put MSN Search into their minds as the search to use.

MSN Search will also feel the effects of the different OS users, Windows, Mac and Unix(Linux, et al). People that are in the Unix camp will just not use anything that is MS.

Google, Yahoo and MSN Search return essentially the same set of results for a given search, it really boils down to MS finding a way to break the "brand naming" that Google is establishing before it gets too deeply entrenched in peoples minds.

It is interesting to note the similarities between Google - http://www.google.com and Yahoo Search - http://search.yahoo.com.

Sorry about the long comment.

[Will]

Great commentary. For a quick eye opener head over to dogpile.com, search for something and then open up the yahoo & google results side by side. Awfully close. Google is the de facto search due to user perception. I work with users concerning enterprise search, and everyone wants a 'Google' like search. But when I ask them how relevant are their results they get at google they cannot answer. It's all perception.

I think the next step in search is actually going to be away from the pure algo driven results and will again include human input. It has to, until an algo can put context around one or two words it will be impossible to deliver the results that people are coming to expect.

What interests me most about Don's post is the reference to how great Page Rank is and continues to be. The question I ask is, what is the next page rank? What's going to put page rank out of business? It's going to happen, it's a matter of who/what/when.
Exciting stuff.

[Mark]

One confusion here is concerning PageRank. There remains scant academic evidence that PR improves search, for example. It makes for a convenient soundbyte, though. If you go back to the original Google paper, you will note that structural analysis, URL weighting and other factors dominate. In general, I think there is a subclass of ranking events for which PR is valuable, but they are very much a minority. The other factor that G and Y (and presumably MSN) use to improve results is to identify noisy sites (and spam) using learning techniques--note that this predates the link spamming methods.