Facebook has revealed how vital artificial intelligence (AI) technology is to its fast-growing Marketplace platform, which is two years old on Tuesday.
Introduced in 2016, Marketplace is now used by one in three Facebook users in the US every month, and millions of others around the world. It is a place for Facebook users to buy and sell everything from cars and clothes to apartments and jetskis within their local communities. People can browse items, make an offer, and arrange a deal without paying any of the fees charged by the likes of eBay and Amazon.
The social media giant — currently under the spotlight after a huge security breach saw 50 million accounts compromised — says the entire Marketplace platform is powered by AI today and that more AI-enabled features are on the way.
"Whether someone is discovering an item to buy, listing a product to sell, or communicating with a buyer or seller, AI is behind the scenes making the experience better," wrote research scientists Lu Zheng, Rui Lui, and Marketplace product manager David Kim in a joint blog post on Tuesday.
To help buyers locate items, Marketplace relies on a content retrieval system and a detailed index that is compiled for every product — both leverage Facebook's AI-based computer vision and natural language processing (NLP) platforms in order to achieve their goals.
"Since the only text in many of these listings is a price and a description that can be as brief as 'dresser' or 'baby stuff,' it was important to build in context from the photos as well as the text for these listings," wrote Zheng, Lui, and Kim.
To accomplish this, Facebook built a multimodal ranking system that incorporates Lumos, Facebook's image understanding platform, and DeepText, Facebook's text understanding engine. "As a result, the index for each product includes layers for both the text and the images," said the authors.
In order to get a better idea of what kind of content buyers should be shown, Marketplace uses demographic data from their Facebook profile and keywords from searches they have made within Marketplace.
Computer vision software is also used to recommend visually similar products to buyers. For example, it's used to suggest tables that look similar to those the buyer has already viewed.
Facebook is also testing camera features that could use AI to recommend items users may wish to buy.
"Say you liked your friend's headphones and wanted your own; you could snap a photo of the headphones and Marketplace's AI technology could recommend similar listings for sale nearby," wrote Deb Liu, the VP of Marketplace, in a separate blog post.
Liu believes the Marketplace platform could even give us fashion and interior design recommendations in the future. "In the future, AI could help simplify tasks like completing an outfit or home design project," she said. "For example, you could upload a photo of your living room and get suggestions on furniture to buy based on your layout and size."
On the seller side, AI is used to automate listings and suggest what category and price sellers should use, based on other similar items. "This autosuggest model runs on each of the millions of for-sale listings on Marketplace and classifies each listing into one of the 29 top categories at post time," wrote Zheng, Lui, and Kim.
Facebook also uses machine translation software to translate item descriptions when they're not in a language the buyer understands.
When buyers and sellers start communicating via Messenger, an automated assistant powered by AI called M makes suggestions to help people communicate and get things done. For example, M will suggest replies like "Yes," "No," or "I think so" when someone asks if an item on Marketplace is still available.
Facebook initially touted M as a super smart assistant capable of helping users with a wide range of tasks and queries but it quickly scaled down the offering after realising AI still has a long way to go before that's possible.
"There was an experiment that was really ambitious to figure out what would people use a virtual assistant that basically had human-level intelligence before they had it," said Yann LeCun, chief AI scientist at Facebook, in an interview last month. "The experiment was designed to figure out how much of that can we automate and how much of it is kind of beyond."
