One of the greatest challenges confronting social media companies as they push towards their “privacy first” visions of our digital futures is how to maintain their surveillance empires as our communications are increasingly end-to-end encrypted. Facebook’s lifeblood is our most personal and intimate interests and communications. As those datapoints vanish behind the veil of encryption, how will the company continue to build the exquisitely detailed personalized behavioral and interest models that feed its advertising engine? The answer is likely to be the very on-device AI algorithms it is turning to for content moderation.
As Facebook rushes towards its touted “privacy first” future that prominently features end-to-end encrypted communications between users, one of the company’s greatest challenges is how to facilitate its continued mining and manipulation of its users when their interests and engagement are cloaked in the secrecy of encryption.
Communications metadata of the kind emphasized by the Edward Snowden NSA disclosures could make up some of this shortfall, allowing Facebook to see how often any pair of users communicate, but this only helps it understand user ties, not their interests.
To peer inside its users’ end-to-end encrypted messages, the most likely scenario is that Facebook will build upon its on-device content moderation initiative to add behavioral profiling to its on-phone content scanning algorithms.
As Facebook moves its content moderation algorithms onto end users’ phones, it will be scanning every message they send before it is encrypted and every message they receive after it is decrypted.
Once this infrastructure is in place, it would be trivial for Facebook to add a categorization model to its scanning pipeline that categorizes each message into its myriad advertising selectors. Such multiclass categorization models are fairly trivial to build and can be constructed for text, images, audio and video content, allowing Facebook to assign classification labels to everything flowing through a user’s account.
Facebook could do a number of things with these labels.
It could transmit these labels back to its own servers while truthfully assuring users that their communications are encrypted. The original messages would remain encrypted and inaccessible to Facebook, while the flow content classification labels that are what powers Facebook’s ad machine would continue uninterrupted back to its servers.
Alternatively, Facebook could simply store a running archive of content labels on the user’s device or encode them into an on-device model of some kind and externalize its ad algorithms to the user’s phone to incorporate these labels or model to make decisions about what ads to show the user. The ad selectors would then be sent to Facebook’s servers to request the appropriate ads. This would still result in a substantial amount of sensitive topical interest and behavioral information leaking to Facebook, but slightly less than simply streaming the complete label archive to its datacenter.
Outsourcing Facebook’s advertising engine to run directly on users’ devices would save Facebook an enormous amount of computing power and enable it to build ever more complex classification models now that it no longer has to worry about its own hardware resources to execute those models. As the onboard AI capabilities of smartphones grows rapidly, these models can become ever more complex.
Client side behavioral modeling could even be used to replace Facebook’s News Feed by having the user’s own phone run the profiling algorithms that build up a model of the user’s interests to find relevant content for them, again alleviating Facebook’s central datacenters of this task.
It would also solve Facebook’s privacy problem and insulate it from government demands for user data, since the company would no longer directly have access to user communications on its own servers. Much like Apple cannot remotely log onto a customer’s iPhone and download their private files, so too would this model protect Facebook from increasing government intrusiveness into user data.
Would this really meet Facebook’s needs?
Facebook the company is essentially an advertising engine. At the end of the day, it only needs to know enough about its users to keep them engaged and to show them relevant advertisements.
Facebook doesn’t actually need to store user data on its own servers. So long as its mobile application can classify each post and build up the necessary behavioral and interest models entirely on-device and simply send requests for relevant advertisements and content back to Facebook’s datacenter, the company can achieve all its goals with a phenomenal reduction in computing power and a dramatic increase in user privacy.
In fact, given that user content will no longer be leaving the phone, merely classification metadata, Facebook might even be more insulated from many data protection laws.
The company did not respond to requests for comment.
Putting this all together, Facebook’s move to the edge will help it not only continue to enforce its content moderation rules in an end-to-end encrypted world but will actually allow it to continue running its advertising behemoth with minimal modification.
In a world where content scanning can occur on-device, even encryption does little to secure our content from prying eyes.
