Searching for Bing: How Microsoft's search engine has lost its way

In 2015, Microsoft’s Bing search engine achieved something it had never had before: relevancy. By notching a 20-percent share of U.S. search, according to comScore, it managed to impact Google, the brand that was literally synonymous with search.

In 2018, it seems like Bing’s willing to hand some of that success right back. Why? Because, as a habitual Bing user, I’ve noticed a deterioration in quality.

In the past I’ve argued that there were four good reasons to switch from Google to Bing. Since then, I’ve used Bing extensively, and it still remains my PC’s default search engine. I’ve sold colleagues on the benefits of Microsoft Rewards. Like anyone else, I’ve used Bing to hunt down news, search for social media posts on Twitter and elsewhere, research queries and trips, and more. But on more and more occasions, I find myself typing in Google.com within the search bar, then hunting down some tidbit or story that I know is there.

To be fair, Microsoft executives say they aren’t seeing any deterioration in quality, and attributed some of what I saw to glitches. (We’ve updated this story based on Microsoft’s responses to our original report.) My examples, which I’ve listed below, also highlight specific examples of Bing’s decline—which again, to be fair, represent my own subjective opinion through cherry-picked examples. See what you think in the examples below.

News search gets lost

Consider the same search on Bing and Google: “ThinkPad,” Lenovo’s iconic notebook series. As a journalist, I often click on the “news” tab of both search engines, with the idea that I want to see relevant news from both search engines, typically referred to as Google News and Bing News.

Immediately there are stark differences between the two. Bing News shows me just three results. Not only is it wasted space, but Google, by comparison, clusters several recent stories together.

Bing favors the sort of “endlessly scrolling” Web page design that eliminates unnecessary page clicks, a point in its favor. But just three entries down is a “trending on Bing” carousel with world news and politics, and nothing about the ThinkPad. That’s not what I’m looking for! Google News requires a click to jump to the next page, but everything appears relevant. When I first started researching this story a few weeks ago, Bing showed me an Atlanta crime story where a ThinkPad just happened to be one of the items stolen from a vehicle.

Generic searches for news stories also demonstrate stark differences between the two search engines. I frequently use search engines to research (and link) older stories. Entering “hachman snapdragon 850 pcworld” brings up the result I’m looking for within Google; in Bing, it doesn’t even appear on the first page of results. That’s a fundamental misstep, and one I’d swear wasn’t there several months ago.

hachman snapdragon 850 pcworld — For me, this is one of Bing’s big weaknesses: my inability to find stories I *know* exist, but can’t find.

Bing seems to be moving in the direction of providing multiple sources or opinions to confirm reports. But from my standpoint, it’s failing at its basic goal: providing relevant links.

In response, Jordi Ribas, the corporate vice president of Bing and AI products at Microsoft, said that the specific “snapdragon” query that I mentioned before was due to a power outage that occurred at one of the company’s data centers, and that there was an error in merging the document index. Now that Microsoft has fixed the error, Ribas said, the query now returns the expected result. (It does.)

Microsoft’s response: ‘independent judges’ confirm Bing’s success

As part of our conversation, Ribas pushed back on my claim that Bing’s overall quality had diminished. As part of the Bing’s internal testing process, he said, Microsoft spends millions of dollars on what the company calls “independent judges,” third-party testing agencies that are paid to look at anonymized search results from both Bing and Google. All told, more than 10,000 results are examined per month, Ribas said, selected randomly from search logs.

Each page is ranked as a win for one side—again, the testers don’t know which is which—or as a tie. Microsoft then tallies them up; a positive score indicates that more pages are being decided in favor of Bing, and a negative score shows that the competition, Google, is winning. These metrics show that Microsoft in fact is winning on the order of several points per month dating back to June 16, according to a graphic that Ribas shared via a Skype call.

“We’ve had an increase in market share over the years, a 10 percent increase in users,” Ribas said, referring to comScore numbers that put the company at about 24 percent of all U.S. search traffic. “If anything, we believe that Bing is getting better.”

At one point, Bing touted itself as the destination for social search, with a sidebar and various tools to connect to social networks. These days it has trouble keeping up with Google.

Consider the following search, where I wanted to see what the reactions had been to legendary announcer Don Criqui’s receiving the ax as the radio voice of my alma mater, University of Notre Dame (“Twitter Don Criqui”).

twitter don criqui — Bing offers up some inaccurate results, and generally ignores Twitter search. Google does a better job.

While Twitter would like you to believe that the best Twitter search lives on Twitter.com, Bing sort of flubs the whole thing outside of the lead result. Google at least gives you a decent effort. (Unfortunately, neither Bing nor Google can provide anything besides Criqui’s Facebook page.)

Image searches are hit-and-miss

One of the historical strengths of Bing is its image-search capabilities, which I revisited to see how the search engine is doing. It’s easy enough to pull up images of actress Nicole Kidman, even when the search is “Nicole Kidman at age 23.” (Though does Google pull up more images of a younger Kidman? Perhaps.) But what both Google and Bing are touting today is the intelligence of their search engines: How specific can they be? Here, Google pulls ahead: try “a snowy lake at night” and “a hat in a field.” While both Bing and Google nail the first, Bing fails badly at the second.

a hat in a field — I’m not sure what’s going on here, but Bing’s image search isn’t demonstrating much intelligence.

Generally, both Bing and Google do well when asked to search for images that you know already exist: “Marilyn Monroe and Groucho Marx,” for example, or celebrities in general: “DMX and puppy.” But again, if you’re asking Bing and Google for images that are (probably) being identified by AI, rather than metadata, Google seems to do a better job. Try associating some random objects together: like “baseball in tree.”

dmx and puppy — Not many problems here for either Google or Bing.

For those of you who would never search for something so detailed, that’s fine. But for me, I sometimes can’t help wondering what I’m missing.

You may disagree with me, but if I searched for “baseball ornament” I’d expect to see the pictures Bing found. I think Google’s results more closely match the original “baseball in tree” query.

Here, Ribas showed a few quick examples where Bing did well in displaying “random” combinations of objects in images, such as dogs and cats under a couch. Bing also allows you to search within an image, Ribas noted, highlighting a vase, for example, as a distinct object from the coffee table upon which it’s placed.

Flight data is bogged down by ads

Bing may have recently launched comprehensive services to improve the hotel booking experience by aggregating third-party data, but the company’s basic travel search still remains underpowered. Take a generic search, “flights to Hawaii.”

flights to hawaii — I’ve had to scroll down on Bing’s page to get past the superfluous ads.

You can make a basic assumption, as Bing does, that you’ll want to fly into Hawaii’s main airport, in Honolulu. Both Bing and Google establish your origin airport via your location, which is fine with me. But Bing does two things wrong here: One, it makes you click on a follow-up button to check flight information and fares; and two, it should provide a list or drop-down menu of islands or airports. Bing does allow you to select how many passengers you want to bring, however.

What Bing doesn’t quite seem to realize is that it’s competing with specialty travel search sites like kayak.com. Google’s website, Google.com/flights, seems to recognize this, and provides a wealth of data at your fingertips. On the other hand, Bing intelligently recommends certain flights as the best for a given set of criteria, as per this search for flights to New York City.

flights to jfk best flights — After asking Bing to find flights, or clicking through to Google’s flights page, you receive this result. Both search engines largely settle on the same “best flights,” though Google offers far more customization options.

Personalization’s benefits are what, exactly?

Another interesting element of the “ThinkPad” search within Bing News: The top results remain unchanged, regardless of whether I sign into Bing. As a journalist, I admit that personalized results aren’t always beneficial. If PCWorld editors are trying to assess the relative importance of a particular search term, we search anonymously. This is an anomaly, however, and it goes back to the old search conundrum: If I’m searching for “apple,” am I looking for a Red Delicious, or the computer company? If I’m allowing Bing or Google to capture my data, shouldn’t I benefit?

Interestingly, Google seems to give me the same top search results when searching for Apple whether logged in, or anonymous. But I noticed one difference: When logged into Bing, Bing surfaced several images of apples (the fruit) within its images tab. Google apparently knew (or guessed) that the Apple Inc. logo was what I was looking for. Both identified Apple Stores near me.

Really, this isn’t so much of a Bing-vs.-Google issue as it is just a question of what benefit you gain for trading off your privacy. I realize that personalized data is designed to influence ads, not search results. But isn’t it time we got a bit more out of the deal?

Bing’s redemption: answers

There is one search segment where Bing runs neck-and-neck, or even ahead of, Google. When asked a basic question (“how tall is the Eiffel Tower?” “How old was Lincoln?”), Bing acts less like a search engine and more like an encyclopedia, answering the question and then encouraging further exploration via related facts. Google does well here, too. Part of this involves using the sidebar area to the right of the main search results to provide more information.

For even an exceedingly generic search term (“Berlin”) Bing brings up things to do, facts, images, and news. Google does as well, though, and offers to plan a trip. While Bing comes off well in this comparison, I think Google goes a bit further in providing a user-friendly experience. (As well as e-commerce opportunities.)

Ribas also pointed out that Microsoft has spent effort in distinguishing itself in other niches, such as recipes, as well as summarizing e-sports results.

Still, I’d say that Microsoft’s management clearly believes that Bing’s road to success lies in this broad middle ground of frequently-searched terms, rather than in some of the more specific niches that I’ve highlighted. “Bing it” may never become a common verb. But if I’m turning more and more to Google, I just wonder how many other people will make Google their default search engine—for the first time in several years, I’m considering it myself.

This story has been updated at 12:34 PM on August 23 with comments from Jordi Ribas, the corporate vice president of Bing and AI products at Microsoft.