Apple's new Photos app will recognize 7 different facial expressions

A developer has uncovered how Photos will employ facial and object recognition to generate Moments and yield search results in iOS 10 and macOS Sierra.

Apple’s new update to Photos is one of the reasons we’re most excited about iOS 10, and now we’re learning more about how the app will integrate “Advanced Computer Vision.”

After Apple released iOS 10 and macOS to developers at WWDC, a developer discovered the list of hardcoded objects, scenes, and facial expressions that the Photos app will be able to recognize. According to developer Kay Yin, Photos will be able to recognize seven types of facial expressions: greedy, disgusted, neutral, screaming, smiling, surprised, and suspicious.

More impressively, the Photos app will also be able to detect 4,432 objects and scenes, from “alleyways” to “zucchinis.” This object and scene recognition is used by Photos to yield accurate search results in all languages.

“We do 11 billion computations per photo to be able to detect things like, there’s a horse, there’s water, there’s a mountain, and with this you can do really powerful searching on your device,” Craig Federighi, Apple’s senior vp of software engineering, said during the WWDC keynote.

Additionally, object and scene recognition is used to generate “Moments,” the new video montage feature in Photos that automatically stitches together media based on theme, date, and location metadata. According to Yin, Moments will be generated according to 33 different categories, like “day in history” and “last weekend.”

Why this matters: Apple’s improved Photos app in iOS 10 is now a lot like Google Photos: using face, object, and scene recognition so you can search your camera roll by person, location, or topic. Unlike Google Photos, however, Apple has made sure that Photos uses only machine learning to analyze the photos that are stored locally on your iPhone. Apple believes there doesn’t necessarily have to be a trade-off between machine learning and privacy. At WWDC, the iPhone-maker introduced the concept of “differential privacy,” which employs crowdsourcing to make a more useful product, without exposing any one individual’s data. And Apple seems to be relying on this concept in the new Photos app, the place where we store our most personal information.

Join the newsletter!

Error: Please check your email address.
Rocket to Success - Your 10 Tips for Smarter ERP System Selection

Tags iOS 10MacOS Sierraphotos

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Oscar Raymundo

Macworld.com
Show Comments

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Matthew Stivala

HP OfficeJet 250 Mobile Printer

The HP OfficeJet 250 Mobile Printer is a great device that fits perfectly into my fast paced and mobile lifestyle. My first impression of the printer itself was how incredibly compact and sleek the device was.

Armand Abogado

HP OfficeJet 250 Mobile Printer

Wireless printing from my iPhone was also a handy feature, the whole experience was quick and seamless with no setup requirements - accessed through the default iOS printing menu options.

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?