As society contemplates the future of online content moderation, much of the emphasis to date has focused on censoring ideas rather than moderating how those ideas are expressed. Creating blacklists of unacceptable ideas and narratives creates a problematic space in which a small cadre of individuals in Silicon Valley are able to define what is acceptable for the public to see and say online. Such censorship is particularly corrosive to the functioning of democracy in which it is the expression of unpopular ideas that gives rise to societal movements, even if they run against the desires of those in power. At the same time, the expression of ideas, popular or obscure, can drive online toxicity by virtue of how even the most innocuous interactions are carried out. When it comes to the future of content moderation, a shift from censoring ideas to moderating words might lead to a far more civil and less hateful and toxic online discourse.
Beneath every democracy lies the free exchange of ideas, no matter how unpopular they may be at the time. The idea of women’s suffrage a century ago and the civil rights movement of half a century ago each represented at their beginning ideas that were unpopular with wide swaths of the population, especially the kinds of institutional elites that today wield near absolute power over the social discourse. Yet it was by giving national voice to these ideas and sparking a societal conversation that change was possible.
In today’s world, such momentous change could be halted with a mouse click. All it would take is Silicon Valley to decide such change would be harmful to society and add it to their “unacceptable speech” list to halt such a movement in its tracks by stopping society from being able to discuss it or even know it is happening.
It was just five years ago that this scenario was previewed when inadvertent algorithmic prioritization led Facebook to be awash with ALS Ice Bucket Challenge videos, while users had to turn to Twitter to track the latest developments in Ferguson. In short, a user on Twitter was saturated with live coverage of the events in Ferguson, while Facebook users were treated to a happy world of fun friendly fundraising videos. While Facebook’s deemphasis of Ferguson was not at all intentional, the incident serves as a reminder of just how powerful the algorithmic prioritization and moderation of social platforms really is and the ability of social platforms, if they desired, to entirely silence events or narratives with which they disagree.
What happens in the future as social platforms ramp up their automated content moderation efforts? Videos of riots and violent clashes with security services shared over social media have given vulnerable populations a voice in documenting their experiences. Yet such videos have also been at the center of spirited debates over how violence should be portrayed on social platforms, leading to complex and arbitrary decisions over what constitutes “documentation” of violence versus “promotion” of violence. In short, the context and intent of a post.
Governments throughout the world are eager for new tools to enforce public order and suppress viewpoints with which they disagree. As social platforms intervene ever more actively and aggressively in deciding what ideas their users are permitted to speak about, governments are watching carefully, eager to add their own lists of unapproved topics to those moderation guides.
At the same time, much of the digital world’s toxicity comes not from the expression of hateful ideas but rather from harmful everyday interactions.
Speaking with my own underrepresented colleagues, they often recount that overt explicit attacks on their gender, race and other demographics are less common as social platforms provide greater tools to report them and expanding digital trails increasingly allow the real-life identity behind a harasser to be discerned.
Instead, it is the near-constant belittling, demeaning and dismissive remarks, forceful and unrelenting professional attacks and near-stalkerish toxicity they face every day online. One remarked that when she and her male coauthor both independently shared links to their new journal article on social media, the male coauthor was lavished with praise for the work’s findings, importance and methodological rigor, while she received a deluge of attacks regarding the very same work, dismissing its findings, questioning its importance and attacking its methodology. None of those attacks mentioned her gender or race and thus there was little she could do to stop them.
Such attacks are rarely in violation of the terms of service of most social platforms, since they do not explicitly mention in any way the victim’s demographics or characteristics. Thus, while this relentless toxicity can have a severe impact on the emotional well-being of users and even drive them from social media entirely, there is typically little recourse for them to stop such attacks.
This underscores the problem of idea-based moderation. When platforms define lists of unacceptable ideas, beliefs or narratives, they narrowly constrain toxic behavior to be defined as attacks that explicitly self-identity as a hateful statement.
A deluge of posts from an entire community that call a user stupid and explicitly mention their demographics as the rational for that attack will typically be removed as a violation. Yet that same deluge of attacks that does not mention the user’s demographics will typically remain on most platforms.
In other words, it is trivial for bad actors to get around content moderation rules simply by omitting the reasoning behind their attack.
In contrast, word-based moderation focuses on the expression of ideas, rather than the ideas themselves.
An attack on a researcher’s paper that is in the form of a profanity-laden diatribe would be unlikely to be removed under idea-based moderation, but would be immediately removed under expression-based moderation. On the other hand, a clinical post raising a series of explicit methodological questions with clear evidence and citations to the paper and the broader literature would be permitted.
Putting this all together, today’s idea-based content moderation creates a dangerous slippery slope in which a handful of elites decides the views and events the world is permitted to talk about and raises the interests of governments eager to constrain debate. At the same time, focusing on ideas sidesteps the far broader issue of unattributed toxicity in which vulnerable communities are subjected to relentless attacks that, by virtue of not mentioning the rationale behind the attack, are typically permitted to continue unabated. Shifting to expression-based moderation would address these vulnerabilities and create a far more constructive and informed social dialog.
In the end, we must decide what we as a society wish our social media platforms to be: entertainment or enlightenment?
