Is Bixby smarter than Google Assistant and Alexa?

Credit: Samsung

With each new Samsung Galaxy smartphone, Samsung continue to talk up the potential and capability of their assistant Bixby. With the arrival of the new Samsung Galaxy S10 and S10+, we decided to run Bixby through its paces and see just how smart it is.

As with our previous comparisons of Siri, Google Assistant and Amazon Alexa, we did this by playing Twenty Questions. This isn’t exactly a scientific method but it does seem like a fairly reasonable way to gauge the question-answering abilities of Samsung’s voice assistant.

Again - There's no strict marking criteria to the questions here. I literally pulled twenty queries that I felt like I'd get at least some kind of answer for. The goal isn’t just to evaluate whether the answers Bixby gives us are correct but also whether they understood the nature of the query itself. The idea here is to showcase Bixby’s ability to answer the kind of everyday questions that ordinary users might throw at them.

Hey Bixby

Credit: Samsung
  1. Who is the director of the Black Panther movie? “Black Panther was directed by Ryan Coogler” Correct!

  2. Who was the last Prime Minister of Australia? Initially, Bixby had some trouble with this one. The voice assistant kept mistaking my query for ‘Who is the lost Prime Minister of Australia’ and answering “Harold Holt”. Eventually, I had to rephrase my question to ‘Who is the Prime Minister of Australia’ and it told me that “Scott Morrison is the Prime Minister of Australia.” Tenuous but we’ll give Bixby the point.

  3. How many books are there in A Song of Ice and Fire? Bixby decided to tell me the weather (and in Farenheit no less). That’s a fail.

  4. How much RAM is there in the Samsung Galaxy S8? Bixby didn’t verbally respond to this one. Instead it pulled up a Google search with its answer: 6GB. This isn’t quite correct. There was a special limited edition of the smartphone with 6GB but the regular S8 only has 4GB. We’re not going to give it the point.

  5. How many bits in a byte? “That comes to 8 bits.” Correct!

  6. What is Quantum Computing? Bixby read up the Wikipedia definition for Quantum Computing, which we’re gonna say is correct.

  7. Is a plastic container microwave safe? Bixby pulled up a pretty legit answer from the web. We’re gonna give them the point.

  8. Who was the star of Malcolm the Middle? “Frankie Muniz.” Correct!

  9. What is the difference between waterproof and water resistant? Bixby pulled up a pretty legit answer from the web. We’re gonna give them the point.

  10. What is the population of Tasmania? “The population of Tasmania was 515,00 in 2014.” Correct!

  11. What events took place at the Winter Olympic Games? “I didn’t understand that.” No point.  

  12. How much sugar should I put in my coffee? Bixby pulled up a pretty legit answer from the web. That’s a point.

  13. How much will it cost to fly to Melbourne? “I can’t do that but I can look up the status of specific flights.” No point.

  14. Who wrote the song ‘Gold on the Ceiling’? We tried this query about a dozen times but always got redirected to Samsung Music to little avail. No points awarded.

  15. How much sunscreen should I wear when I go to the beach? Bixby decided to tell me the weather (and in Farenheit no less). That’s a fail.

  16. How do I fix a broken lamp? Bixby loaded up the SmartThings plugin and said “I can’t seem to do that.” No points.

  17. When is the new season of Jessica Jones due to start? Bixby pulled up several online sources that displayed the answer but did not read a snippet aloud. We’ll tentatively give it the point.

  18. Will peanut butter help my sore throat? “I didn’t understand that.” No point.

  19. What is Microsoft’s smart speaker called? Bixby loaded up the SmartThings plugin and said “Sorry, for now the function is not supported. I’ll keep learning to make your experience better.” No points.

  20. Who won the 2015 Oscar for Best Picture? “The 2015 Academy Award for Best Picture is Birdman.” Poorly phrased but correct.

Overall score - 11/20

How does it compare to Siri, Alexa and Google Assistant?

If we pull up the scores that we got when we ran Siri (7 points), Alexa (8 points) and Google Assistant (13 points) through the same queries last year, Bixby’s score of 11 correct answers puts it in second place. Obviously, the caveat here is that we haven’t re-tested those Assistants in the time since - and, like Bixby, every smart assistant out there has likely improved since then.

Still, it is a decent sign that some progress has been made when it comes to Bixby. Samsung’s assistant still lags behind the Google Assistant - but so does every smart assistant. It doesn’t have the breadth of core competencies that the Assistant has, nor the wealth of third-party Skills that the Alexa ecosystem offers. However, Bixby’s capability when it comes to Q&A is often surprisingly good.

I just wish it wouldn’t seemingly trip over itself in a rush to call upon its plugins so often.

Credit: Samsung


Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags BixbySmart AssistantsSamsung Bixby

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.
Fergus Halliday
Show Comments

Brand Post

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Jack Jeffries

MSI GS75

As the Maserati or BMW of laptops, it would fit perfectly in the hands of a professional needing firepower under the hood, sophistication and class on the surface, and gaming prowess (sports mode if you will) in between.

Taylor Carr

MSI PS63

The MSI PS63 is an amazing laptop and I would definitely consider buying one in the future.

Christopher Low

Brother RJ-4230B

This small mobile printer is exactly what I need for invoicing and other jobs such as sending fellow tradesman details or step-by-step instructions that I can easily print off from my phone or the Web.

Aysha Strobbe

Microsoft Office 365/HP Spectre x360

Microsoft Office continues to make a student’s life that little bit easier by offering reliable, easy to use, time-saving functionality, while continuing to develop new features that further enhance what is already a formidable collection of applications

Michael Hargreaves

Microsoft Office 365/Dell XPS 15 2-in-1

I’d recommend a Dell XPS 15 2-in-1 and the new Windows 10 to anyone who needs to get serious work done (before you kick back on your couch with your favourite Netflix show.)

Maryellen Rose George

Brother PT-P750W

It’s useful for office tasks as well as pragmatic labelling of equipment and storage – just don’t get too excited and label everything in sight!

Featured Content

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?