Business - Internet search to move in a new direction

Yahoo might be in news for all the wrong reasons. But Yahoo Inc research head Prabhakar Raghavan seems to believe that he has found the magic mantra to dethrone the internet search king—Google. His unconventional ideas include open source search model to cut into Google’s growing market share. Also a consulting professor at Stanford, he has earlier worked with Verity and IBM Research. Raghavan takes Pragati Verma through the complex maze of internet search and Yahoo’s plans to disrupt the search market and make room for more players and consumer choice. Excerpts:
Microsoft’s hostile bid to acquire Yahoo has dominated headlines? How did it impact employee morale in the research team?
People were obviously unsettled, when no one can comment on what would happen the next day. I advised my colleagues that if you are contributing to a profitable growth, there should be no sense of personal insecurity. Sometimes, the best move that you can make is to stand still.
You have taken the unusual decision of hiring researchers from fields other than computer sciences? How much has it helped?
Internet companies have traditionally researched on computing technologies. We were the first to hire micro-economists. The fact that Google and MSN followed us should tell you how much has the move worked. As computer science professionals, we could design systems that do huge damage to economic value. For instance, we need micro-economists to help us best place click-on ads. To figure out how satisfied is our userbase with a search result, we need social science and not just IT.
Our micro-economists suggested market-reserve pricing, basically using microeconomic theory to set minimum bids, based on the work of Roger Meyerson, who won the Nobel Prize in 2007. We rolled the suggestion into our advertising. By adjusting the reserve price, you could make a lot more money. While I can’t give you the money it made for Yahoo, I can tell you it paid many times over compared to the cost of research.
As we move into social networking domains, sociologists and cognitive psychologists are valuable and tell us how people behave. We need to understand how networks of people behave and influence each other. You might know of the Six degree theory of Duncan Watts. It builds on networks of acquaintanceships; there are six hops between any two people. He has done pioneering work on that. People like him give us insights into how network of users behave. There are theories that are hard to test in the physical world; it is possible to see how they unfold on the internet.
Yahoo has talked of doubling operating cash flow by 2010. Are you counting on these ideas to improve financial performance and dethrone Google as the search king?
Let’s first look at the realities of today’s search industry. Three companies dominate the market, giving users limited choice. Becoming a serious search player requires a massive capital investment of about $300 million. We are trying to remove all barriers to entry for software developers, who have ideas about how to improve search.
In May, we released a developer platform, called SearchMonkey, that lets programmers customise how our search results are displayed. Our new software, BOSS (build your own search software) takes Yahoo’s open strategy to the next level by providing Yahoo search infrastructure and technology to developers and companies to help them build their own search experiences. Essentialy, it is a self-service web services model for developers and start-ups, who can quickly build and deploy new search experiences.
BOSS offers two options for companies and developers and has partnered with top technology universities to drive search experimentation, innovation and research into next generation search. We are already working with select partners to build a more relevant brand or site-specific web search experience. We are also working with universities like Carnegie Mellon University, Stanford University, MIT and IIT Bombay.
This essentially means anyone can build their own search service. Aren’t you worried that an innovative startup could gain more than Yahoo?
To open up, you need an intellectual commitment and need to believe that you alone are not smart enough to invent everything. With just three companies dominating the space, you can only expect that much innovation. More participants will increase innovation. This will essentially disrupt the search market and make room for more players and provide consumers with more choice. And since we are not the market leaders, we don’t face the greatest risk.
According to comScore data, Google had a 62% share of the US search market in May, while we had 21% and MSN 9%. Our prediction models suggest that Google could lose a big chunk of its market share, as BOSS partners and players come in.
Open source initiatives will enable companies and developers to leverage Yahoo search infrastructure, to fuel customised and innovative search experiences. For instance, semantic search engine Hakia is using Yahoo’s BOSS technology to accelerate its advanced QDEX (query detection and extraction) technology. The Yahoo data is being combined with Hakia’s own crawled data, and then indexed and ranked by Hakia.
And of course, it will extend Yahoo’s advertising footprint across the web.
Can search starts ups like Cuil beat Google on their own?
I personally feel it is a nice set of ideas, but I am not sure if it will succeed in its current form. It’s actually too early to predict how they will perform in the long run. They have their secret sauce and so do many others. May the best secret sauce win!
We will be launching a new generation of search in two to three months. I can’t give you too many details now, but search is going to move in a completely new direction.
Users don’t really want to search. They want to spend time on their work, personal lives and entertainment. They come to search engines only to get their tasks done. We will move search to this new paradigm of getting the task done.