Pinterest’s AI Ups the Diversity Game
August 27, 2020
- Author: CTA Staff
Artificial intelligence is enabling Pinterest to power more inclusive search and recommendation functions.
In 2018, CES veteran Pinterest released a deep learning-enabled skin-tone range search function, building a more inclusive experience for Pinners.
The company recently introduced the latest iteration of skin tone ranges, better reflecting the diversity across its platform, including in its augmented reality technology, Try On. Particularly for users who browse the platform for beauty and style tips, the skin tone range technology provides a more personalized experience.
Enabled by artificial intelligence (AI), the new version of skin tone ranges takes on the identification challenges introduced by shadows, lighting, image variety and other factors.
Diverse Data Identifies Gaps
To evaluate its skin-tone range system, Pinterest used a diverse set of images that covered a range of skin tones. Error detection matrices identified if the system misclassified certain skin tones or images.
The performance measurements revealed biases in the first version’s algorithms based on facial recognition AI. The reliance on faces to predict skin tone limited the scope of the system, which was not able to analyze skin tone ranges in images including other body parts, such as a hand, and often misclassified darker tones.
Reteaching Artificial Intelligence
To mitigate the biases, the Pinterest team worked to label mass amounts of images that covered skin tones in different ways, including faces with sunglasses or facial hair, other body parts with accessories, and more recently faces with masks.
Pinterest engineers then reskilled the AI algorithms with these thousands of images, even making sure that the system could correctly classify images it had missed in the first version.
The team also made sure to categorize broader cases of faces in different positions, skin tones under different lighting and other more nuanced factors, helping the AI build a more robust database.
Leveraging Learnings to Diversify
Tools such as the skin tone ranges has helped Pinterest better understand the diversity of the platform’s content and how different people engage with the content. According to the Pinterest team, the first version of skin tone ranges helped them quadruple the skin tone range coverage of beauty and fashion content.
These measurements help the company diversify search results and recommendations to better fit their audiences.
