Apple's new Photos app will recognize 7 different facial expressions

A developer has uncovered how Photos will employ facial and object recognition to generate Moments and yield search results in iOS 10 and macOS Sierra.

Apple’s new update to Photos is one of the reasons we’re most excited about iOS 10, and now we’re learning more about how the app will integrate “Advanced Computer Vision.”

After Apple released iOS 10 and macOS to developers at WWDC, a developer discovered the list of hardcoded objects, scenes, and facial expressions that the Photos app will be able to recognize. According to developer Kay Yin, Photos will be able to recognize seven types of facial expressions: greedy, disgusted, neutral, screaming, smiling, surprised, and suspicious.

More impressively, the Photos app will also be able to detect 4,432 objects and scenes, from “alleyways” to “zucchinis.” This object and scene recognition is used by Photos to yield accurate search results in all languages.

“We do 11 billion computations per photo to be able to detect things like, there’s a horse, there’s water, there’s a mountain, and with this you can do really powerful searching on your device,” Craig Federighi, Apple’s senior vp of software engineering, said during the WWDC keynote.

Additionally, object and scene recognition is used to generate “Moments,” the new video montage feature in Photos that automatically stitches together media based on theme, date, and location metadata. According to Yin, Moments will be generated according to 33 different categories, like “day in history” and “last weekend.”

Why this matters: Apple’s improved Photos app in iOS 10 is now a lot like Google Photos: using face, object, and scene recognition so you can search your camera roll by person, location, or topic. Unlike Google Photos, however, Apple has made sure that Photos uses only machine learning to analyze the photos that are stored locally on your iPhone. Apple believes there doesn’t necessarily have to be a trade-off between machine learning and privacy. At WWDC, the iPhone-maker introduced the concept of “differential privacy,” which employs crowdsourcing to make a more useful product, without exposing any one individual’s data. And Apple seems to be relying on this concept in the new Photos app, the place where we store our most personal information.

Join the Good Gear Guide newsletter!

Error: Please check your email address.

Tags iOS 10MacOS Sierraphotos

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Oscar Raymundo

Macworld.com
Show Comments

Cool Tech

Crucial Ballistix Elite 32GB Kit (4 x 8GB) DDR4-3000 UDIMM

Learn more >

Gadgets & Things

Lexar® Professional 1000x microSDHC™/microSDXC™ UHS-II cards

Learn more >

Family Friendly

Lexar® JumpDrive® S57 USB 3.0 flash drive 

Learn more >

Stocking Stuffer

Plox Star Wars Death Star Levitating Bluetooth Speaker

Learn more >

Christmas Gift Guide

Click for more ›

Most Popular Reviews

Latest News Articles

Resources

GGG Evaluation Team

Kathy Cassidy

STYLISTIC Q702

First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.

Anthony Grifoni

STYLISTIC Q572

For work use, Microsoft Word and Excel programs pre-installed on the device are adequate for preparing short documents.

Steph Mundell

LIFEBOOK UH574

The Fujitsu LifeBook UH574 allowed for great mobility without being obnoxiously heavy or clunky. Its twelve hours of battery life did not disappoint.

Andrew Mitsi

STYLISTIC Q702

The screen was particularly good. It is bright and visible from most angles, however heat is an issue, particularly around the Windows button on the front, and on the back where the battery housing is located.

Simon Harriott

STYLISTIC Q702

My first impression after unboxing the Q702 is that it is a nice looking unit. Styling is somewhat minimalist but very effective. The tablet part, once detached, has a nice weight, and no buttons or switches are located in awkward or intrusive positions.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?