Why and how are machine learning algorithms driving polarisation?

By Professor Wendy Chun FBA

Transcript

For the past three decades, I've been researching new media. In particular, the impact of the internet on society, culture, and politics. Throughout my career, I've sought to work across divides and bring together the humanities and engineering by starting with what appear to be simple technical defaults and then showing how these defaults aren't simply technical, but also social and cultural.

Early on, I charted the remarkable rise of the internet as a mass medium to end mass media by revealing how and why a 1970s technical control protocol, transmission control protocol, internet protocol, TCP/IP, became 'cyberspace'. That is, how it was bought and sold in the mid to late 1990s as a utopian space that promised to dissolve all political problems from racism to monopoly capitalism by being conflated with 1980s dystopian cyberpunk fiction, which it barely resembled.

Not surprisingly, as we all know now, this conflation of technical control with freedom hasn't gone well and rewriting political problems as one's technology can and should solve rarely does. I've investigated how software, which engineers initially called everything that didn't matter, everything that wasn't hardware became accepted as the invisible essence of computation. And how this nothing, 'the software', came to encapsulate not only computation, but a worldview in which invisible codes determine the visible world. A worldview indeed inspired by early genetics.

I've revealed how new media, which seemed to thrive at the bleeding edge of obsolescence, remain through our habits. For example, Friendster is now long gone. I doubt many now know about Friendster, but friending remains. New media, I've stressed, are at their best wonderfully creepy.

Most recently, I've investigated how machine learning algorithms encode legacies of segregation, eugenics, and discrimination through axioms such as correlation and homophily. Now, to make this more concrete, I'll unpack the role of homophily. Recommender systems and social media networks in general presume homophily. They presume that similarity breeds connection, that birds of a feather flock together, that like acts like like. Apps therefore encourage you to buy, listen to, click on things that other people who've been determined to be like you have bought, listened to, or clicked on.

What's key is that your most valuable likes are ones that are slightly controversial or strange. Liking 'Harry Potter', for example, is so widespread that it's useless. But liking or hating the show 'Fleabag', now that's valuable. And early on, likes were weighted by controversy. The hate button was weighted more heavily than the like button because what's important are likes that divide, because you're placed into network neighbourhoods based on divisive likes. Networks segregate you into neighbourhoods with people who like what you like and hate what you hate, which means that echo chambers, echo chambers aren't an accident, they're a goal.

Now, we have to remember that the ties between homophily and segregation are deep and profound. Indeed, the very term 'homophily' came from a study of US biracial public housing projects, specifically a post-World War II study by the Columbia Bureau of Applied Sociological Research to study the impact of public housing on democratic engagement. These researchers focused on two projects. The first they named 'Craft Town', a cooperatively owned project of some 700 white families in New Jersey. And the second, they called 'Hilltown', a biracial low-rent project of about 800 families in western Pennsylvania.

What's intriguing is that they didn't just coin the term 'homophily'. They also coined the term 'heterophily' to describe friendships based on opposites attracting. And they didn't presume homophily to be always true. Indeed, they were sceptical that "birds of a feather always flock together". They asked, when does homophily hold and when does heterophily hold? And they found that status homophily in general wasn't absolute except in the cases of gender and race. To explain this, they hypothesised that status homophily was driven by what they called 'value homophily', the sharing of values.

To prove this, they concentrated on the over-selection of white liberals and white illiberals as friends in that biracial housing project, where they said a friend was one of your three closest friends, regardless of where they lived. This was different from most friendship surveys at the time, which focused on your three closest friends within a housing project. You were considered liberal if you believed that housing should be biracial and that folks in Hilltown got along, illiberal if you thought the opposite, and ambivalent if you didn't believe that housing should be biracial but thought the races in Hilltown did get along. Here, you see the Eureka moment, the moment they discovered homophily. As you see here, they actually threw out the responses of the black residents because they argued there were too few illiberal and ambivalent black residents with friends in the housing project.

So, they never considered liberals or illiberals as cross-racial categories. And they also ignored the largest category of white residents. The white ambivalents. They also never published the data or the report, and the actual report was submitted as evidence to support desegregation in the United States. It shows us that the over-election of white illiberals in real numbers was not statistically significant. Black and white residents did have friends and acquaintances of the other race. So, this divisive notion of homophily erases most of the world we live in by laundering hate into love. In this segregated world, how do you show love? By fleeing when others show up. And indeed, homophily has been used to model and justify white flight.

Intriguingly, homophily is usually justified in terms of comfort. You're allegedly more comfortable with people like you. But as Facebook whistle-blower Francis Haugen has shown, homophily actually produces agitation and greater stress. Facebook feeds became even more toxic when they were limited to family and friends. And at a certain level, this shouldn't surprise us, right? Think of family holidays and how explosive they are.

My point is that homophily, as it's currently embedded in networks, transforms masses into agitated clusters of comforting rage. You can think of this in terms of the classic physics experiment in which you take a magnet and put it under a piece of paper with iron filings. What emerges, as you see here, is a network with clusters of similarly charged filings at either pole. But these similarly charged filings also repel each other, but they remain together because they're attracted to because they love or hate what they oppose. Increasingly, these angry clusters are strung together like beads in a necklace to create angry majorities, because what holds these beads together are common scapegoats and enemies.

There's a reason why Rene Girard's theories of mimetic desire are so popular in the valley right now, and in his theory, similar people want what their models want. They share the same model, but this leads to conflict. And this conflict is only resolved by sacrificing and creating scapegoats, scapegoats whom they blame for the conflict rather than their similar desires. If this is so, what can we do? Well, what if we built networks that weren't based on homophily? Heterophily exists, opposites do attract. Electricity, for example, goes from negative to positive. Heterosexuality happens. And heterosexuality is almost impossible in a world of absolute gender homophily.

Heterophily, though, doesn't quite get us past this logic, and we've been exploring what happens when we start with these gaps. If you think about networks without gaps, there'd be no networks. Gaps make possible connections and nodes, but these gaps aren't empty. Like the example I discussed earlier, they're filled with data that's been deleted. They're filled with people's lives and experiences. And it's by studying these gaps that we can discover worlds that could have been and still are.


Wendy Hui Kyong Chun is a Professor in the School of Communication, and Director of the Digital Democracies Institute.

Subscribe to this podcast via your chosen service

Sign up to our email newsletters

Join our mailing list to explore the ideas and impact of the British Academy. Get updates on research, funding, policy, international collaborations, and events that bring the humanities and social sciences to life.