BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Facebook Just Gave Repressive Regimes The Ultimate Surveillance Blueprint

Following
This article is more than 4 years old.

Getty

Perhaps the most dangerous aspect of Facebook’s new initiative to move content moderation to the edge, running their algorithms directly on users’ phones, is that it gives the world’s repressive regimes a blueprint for the ultimate global surveillance network. By pushing governments to see surveillance not as an active process that requires identifying targets, intercepting their communications and centrally analyzing unimaginably large haystacks for the proverbial needle, but rather as a distributed process running directly on users’ own ever-present smartphones, Facebook has demonstrated that the future lies in the surveillance state.

Governments have long adapted to technological change by harnessing the latest communications modalities to surveil their citizens and gather intelligence on foreign nations. The infamous Edward Snowden disclosures in 2013 opened the public’s eyes to just how prevalent this surveillance really is and how deeply it reaches into our lives.

Yet, the Snowden disclosures also revealed how limited current global scale surveillance is, even with the NSA’s resources. The internet is simply too big to attempt to hoover up the totality of all global communications and the processing resources required to make sense of everything said and seen worldwide in realtime is beyond the ability of even the US Government.

Facebook’s new initiative to move its content matching and AI content moderation algorithms to the edge offers governments across the world a powerful new template for surveilling both their own citizens and Facebook’s two billion other users spanning the globe.

Suddenly, instead of having to compromise users’ devices or intercept their communications in transit and decrypt and process them, Facebook reminds us that users will be perfectly happy to allow companies to download filtering algorithms directly to their phones to process all of their communications on-device. This not only sidesteps the encryption debate by granting access to communications before they are encrypted for transmission and after they have been decrypted for display, but also avoids scalability concerns by offloading processing of each user’s communications to their own device.

While Facebook’s efforts are still at the research stage, the company’s presentation demonstrates that it has made considerable progress in proving out the feasibility of performing large scale production content moderation entirely on-device.

For governments, Facebook’s model offers a blueprint of the ultimate surveillance machine.

Rather than relying on Facebook, governments could require all cellphones used on their soil to install a monitoring application that authenticates them to the local cellular providers while also performing realtime monitoring of all communications that transit the device or which occur in its presence, turning their microphone and camera into a 24/7 mobile surveillance station.

Alternatively, governments could require device manufacturers to embed such surveillance filtering directly into dedicated firmware on the device that prevents the phone from being used on local networks without it.

Rather than the fixed government-funded telescreens of 1984, in Facebook's proposed future, the public itself would be purchasing and maintaining the very surveillance network used to repress it.

Imagine a world in which every smartphone monitors every word seen or said by its owner, alerting authorities at the first disloyal comment.

Add to that a world in which every phone's microphone is open 24/7, monitored by onboard AI algorithms that identify every voice heard nearby and transcribing every word it overhears, allowing the authorities to reach beyond the owner of a phone to everyone around them. What good is an encrypted voice call when microphones all around are listening in and transcribing every word?

Couple this with Facebook’s ability to map the realtime physical location of its users and create richly detailed maps of the movement habits of any given demographic or interest group.

The result is perhaps the most powerful surveillance network ever conceived.

Of course, could it be that Facebook itself helps pioneer how governments see this surveillance potential?

For a company that refuses to deny tracking the realtime movements of journalists and politicians, it is a minor stretch to see Facebook scanning the realtime conversations of its users to flag negative mentions of itself and listening to background conversations for intelligence that might be useful to its business operations.

After all, it went to great lengths earlier this year to remind the public that it reserves the right to utilize their GPS, camera and photo galleries for any purpose it deems fit.

Yet, Facebook’s efforts go far beyond surveillance. They provide the ultimate blueprint to countries wishing to silence speech they dislike. Why punish users after they have spread criticism of the government when you can delete those posts before they are ever published?

The company did not respond to requests for comment. Its silence when asked what safeguards it envisions to protect such on-device scanning from being repurposed by repressive governments reminds us that its sole focus is on profit, not the safety and privacy of its two billion users.

Putting this all together, whether governments turn to Facebook to outsource their domestic and global surveillance needs or whether they merely use Facebook’s model as a blueprint for their own on-device surveillance programs, Facebook is rapidly leading us towards a world in which privacy will be nothing more than a faint memory of a quaint bygone era.