Since Meta announced an end to third-party fact-checking, claiming it was freeing itself from “, social media users have questioned the value of content moderation. Is it an important tool for protecting the safety and efficiency of a platform, or a systematic method of censorship? In my view, content moderation merely represents a broken system – the insatiable requirement of human sacrifice for the sake of technological advancement, and its use as a veil to obscure the profit-driven motives of social media companies.
Content moderation has been embedded in the legal framework of online platforms for decades, as a way to prevent harmful behaviours from spreading on these sites, whether it is hate speech or exploitation. With platforms beginning to favour algorithmic detection over human-powered content moderation, feedback has focused on job losses and the potential harm posed by inefficient systems.
Advertisement
These concerns are well-placed: platforms that rely more heavily on algorithms are banking on computational capabilities that simply don’t exist yet to accurately detect complex behaviour and meaning. Using large language models to determine the difference between , for example, is complicated by the fact that this distinction is subjective and hinges on shifting social norms. Many users have weighed up these risks, along with the redundancies, and have concluded that human-based content moderation must be valued and sustained – these are the people who keep us safe.
However, we often forget there is collateral damage tied to this semblance of security. Moderators are the majority of us can hardly imagine. The “content” they engage with can include anything from insulting language to videos of graphic beheadings and child sexual abuse. They are seeing content like this day in, day out – often for hours on end – as part of their labelling assignments. We, the users, are somewhat shielded from this information, but only after it has traumatised often low-paid and vulnerable workers.
These jobs are already undercompensated and underappreciated; now, social media platforms like TikTok, Meta, and X are choosing to invest even less in their workforce. In doing so, they reveal something crucial about these sites that have come to demand so much of our time. When it comes to profitability, a safe and happy user base is no match for the immense potential to harvest data. The risks of poor content moderation – whether we see unsavoury images and text, or whether we are victims of online bullying – are only compounded by how much of ourselves we are already freely revealing.
For many sites, content moderation already involves both human and algorithmic labelling. As modelling develops, those algorithms will become more sophisticated, theoretically improving the user experience. But we must remember that behind social media platforms are profit-driven companies, and whatever is categorising our data, the implication remains the same: these firms collect an astronomical amount of information about us and the people with whom we interact, and they can use that information in almost any way, at any time.
Big Social Media has become so essential to our daily lives and our work that not only have we been willing to ignore the disenfranchised workforce behind the machine, we have also accepted our own vulnerability to influence and exploitation. Advancements in algorithmic moderation will not save us; they will only enable us to give more of ourselves away.
Jess Brough is a psycholinguist and researcher of language biases
