Mayer: The issue on "deep Web" content is that it's usually in databases and [Web] crawling isn't a great way of getting at a database. ... So we've been doing things like Google Base. Most databases allow people to do an XML feed off of them so you can do an XML output of your database and you can upload that database to Google and Google Base.
IDGNS: Are you making progress with that approach?
Mayer: Yes, literally hundreds of millions of items have been uploaded into Google Base. So we're making progress indexing that data, but we're not doing a good enough job surfacing that data in the search results. So we have it, and if you go to Google Base you can find it but it's hard to figure out, from the universal search aspect, when it should be blended into the main search results.
IDGNS: From an engineer's point of view, is working on search engine technology still "cool" and professionally stimulating?
Mayer: Search is still very much an unsolved problem. ... We're like six years into what's going to become a 500-year discipline. It's still very compelling and we have people joining Google every day who are really excited to work on search.
IDGNS: Three or four years ago, people were saying 'Google is great, it's a cash cow, but for people to switch to another search engine, there's no cost, in terms of effort or inconvenience.' But it turns out this hasn't happened. What's Google's view into this issue?
Mayer: There are two non-obvious outcomes around the stickiness question. One is that it's true: If someone builds a better search, users will probably move over [to that competing engine.] That keeps our employees, especially our engineers, really well motivated to make sure no one has better search. ... We have to prove to our users every day that we still have the best search.
The other non-obvious observation is that search is much stickier than most people realize. Intuitively, you know you can go to [other search engines] ... but people have a source they trust and it takes them a long time to feel like they're not giving anything up [by using another search engine.]
When you find a long lost friend or information about a medical condition that is hard to diagnose or [you experience] that "I'm feeling lucky" moment when this amazing Web site turns up first on Google, each of those moments stays in your mind. There's an attachment, a trust there that this tool really brought me what I needed. ... So the switching cost for search is a lot higher [than people think.]