Algorithms of Oppression: How Search Engines Reinforce Racism

Collects a lot of criticisms of Google’s dominance, focused on the racist implications of taking the corpus as you find it, e.g., searching for “black girls” for a long time returned only porn on the first page. Only some people can live in a “filter bubble” where everything they see reinforces their own beliefs; racism and sexism come to people who don’t want them. Noble had to “take it as a given that any search I might perform using keywords connected to my physical self and identity could return pornographic and otherwise disturbing results,” and asks: “Why was this the bargain into which I had tacitly entered with digital information tools? And who among us did not have to bargain in this way?” Black Girls Code show more is nice, Noble argues, but it’s not black girls’ job to solve Silicon Valley’s racist exclusion and misrepresentation.

The problem is: what to do? Noble complains that Google directs searches to conglomerate news sources, but on YouTube that doesn’t happen and the results seem to be worse, leaning towards extremism and conspiracies, with a lot of racism. Past forms of information sorting were really bad too; Noble notes the history of racist Dewey Decimal System and Library of Congress classifications (not just history, though more contested now). She also discusses how Dylann Roof was radicalized by reading online, starting from Wikipedia and going from there—searches for “black on white crime” lead to white supremacist sites, rather than to neutral crime statistics that would reveal that most crime is intraracial. Could anything other than human moderation stop this pattern? I just don’t know; Noble suggests developing public search engines so that corporate motivations wouldn’t control the data collection/surveillance, but (1) they’d still confront the problems of dealing with a racist corpus, and (2) I’m not so hot on government surveillance either. Another suggestion is a black-friendly search engine, and there are some moves towards that, but I don’t think that solves the problem for people who don’t know to seek it out in the first place—or people like Roof.

The last chapter of the book focuses on a small business owner who cares for black hair, and whose business was harmed by two neoliberal blows—a decrease in the number of African-American students because of anti-diversity policies, and the rise of Yelp, which represented an increased cost—they’d only give her prominence/keep other hairdressers off her page if she paid, even if the other places didn’t specialize in black hair—and also presented particular difficulties for her reviews, inasmuch as she perceived that her customers were less likely to use Yelp in the first place than white people, so their reviews of her place might be the only reviews those customers left on Yelp and thus were more likely to look fake to Yelp. “Black people don’t ‘check in’ and let people know where they’re at when they sit in my chair. They already feel like they are being hunted; they aren’t going to tell The Man where they are. I have reviews from real clients that they put into a filter because it doesn’t meet their requirements of how they think someone should review.” Not that she was all that fond of all her customers—she also complained about people who came into her business to photograph the products she used, then order them online for less. Again, search engines aren’t the only problem she’s facing; it’s a constellation of economic and social changes of which search engines are only a part, perhaps a minor part, though it’s certainly worth pointing out that the small producers are the ones from whom wealth can still be extracted by these larger companies like Yelp. show less

Rating: 4.5* of five

The Publisher Says: Run a Google search for "black girls" - what will you find? "Big Booty" and other sexually explicit terms are likely to come up as top search terms. But, if you type in "white girls," the results are radically different. The suggested porn sites and un-moderated discussions about "why black women are so sassy" or "why black women are so angry" presents a disturbing portrait of black womanhood in modern society.

In Algorithms of Oppression, Safiya Umoja Noble challenges the idea that search engines like Google offer an equal playing field for all forms of ideas, identities, and activities. Data discrimination is a real social problem; Noble argues that the combination of private interests in show more promoting certain sites, along with the monopoly status of a relatively small number of Internet search engines, leads to a biased set of search algorithms that privilege whiteness and discriminate against people of color, specifically women of color.

Through an analysis of textual and media searches as well as extensive research on paid online advertising, Noble exposes a culture of racism and sexism in the way discoverability is created online. As search engines and their related companies grow in importance - operating as a source for email, a major vehicle for primary and secondary school learning, and beyond - understanding and reversing these disquieting trends and discriminatory practices is of utmost importance.

An original, surprising and, at times, disturbing account of bias on the internet, Algorithms of Oppression contributes to our understanding of how racism is created, maintained, and disseminated in the 21st century.

I RECEIVED A DRC FROM THE PUBLISHER VIA EDELWEISS+. THANK YOU.

My Review: The world, as the Internet has shaped it, took a promise of information access and educational opportunity unparalleled in human history and screwed it up to the point it reinforces the evils and stupidities it could so easily have alleviated.

The problem, it transpires, is both blindness..."*I* am no racist, or a sexist! Why, some of my best friends..." is not new, nor is it uncommon in any society...and neither is hubristic malevolence (Cambridge Analytica, for example). We're two decades in to a giant, uncontrolled social experiment. Voices like Author Noble's are still notable for their infrequence of prominence in the rarefied world of Congressional hearings and the European Union's creation of the GDPR.

The issues that Author Noble raises in this book need your attention. You, the searcher, are the product that Google and the other search engines are selling to earn their absurd, unconscionable, inadequately taxed profits. Every time you log on to the internet, Google knows...use other search engines, never click on any links, and Google still knows you're there. That's the Orwellian nightmare of it...like East Germany's Stasi, they're everywhere, in every website you visit. Unlike the Stasi, they are possessed of the capacity to quantify and analyze all the information you generate, and sell it to anyone who can use it. For you or against you, as long as the check clears, Google and its brethren couldn't care less.

(There are links to information sources in the blogged version of this review at Expendable Mudge Muses Aloud.) show less

Although a solid summary of the larger problem of subtle bias in search engine results, this book promised much more in the way of explaining the source of that bias. Although the author recognizes and criticizes "the idea that it is not the search engine that is the problem but, rather, the users of search engines who are. It suggests that what is popular is simply what rises to the top of the search pile," she does not provide any information to distinguish between the two alternatives. She assumes that the problem is Google, and not Google users, without providing any foundation for that assumption, and frames the entire book in terms of this undefended conclusion. For example, she sees no difference between page results and query show more auto suggestions, but my understanding at least is that users are much more influential in the latter than the former.

I had hoped for some actual technical discussion of how page results are generated, and what aspects are under the control of Google and which are merely aggregating the preferences of actual users, but that never comes. It does not help that, for a book published in 2018, the bulk of her examples are ancient, often from 2011. The Internet, and Google with it, has changed drastically in the last seven years, but her discussion barely recognizes the fact. I suspect the first section of the book, with the very outdated examples, was the core of her 2012 dissertation, which has here been included with very little change or update. Later chapters attempt to bring in more recent issues, but they feel like quick glosses meant to fill out the need to expand the dissertation to book length, and lack the more serious consideration that went into the earlier section. In all honesty, she should have written an entirely new, current book rather than attempt to update the older material. She has the background to do that, but she is trapped by the confines of trying to publish the dissertation as is.

With more technical expertise, and data more contemporary to the date of publication, this could have been a more exciting contribution. Failing that, it is still interesting on the general points, if dusty and outmoded in the details. show less

Safiya Umoja Noble’s Algorithms of Oppression is a book I waited for impatiently because I was familiar enough with her research to know it fit my reading interests and was important in my profession. (It jumped into the pages of Inside Higher Ed when a hasty Twitter questioning of the book's validity by a major tech organization’s official Twitter account based on nothing more than a quick glance at a publisher’s description led to a backlash. And many comments here at IHE, of course.) It’s the culmination of years of studying the ways algorithmic information systems – Google Algorithms of Oppression coverSearch in particular – represent people who are not white and not male. She noticed years ago, when shopping for her show more nieces, that black girls looking themselves up, would see lots of porn because that’s what Google thought you must be looking for. After she published an article about it, those search results changed. There’s no way of knowing what prompted the change, but every so often, when Google is called out for search results that are surprising – denial sites, for example, topping the list of results when searching for information about the Holocaust – they make some tweaks. This is not a systematic overhaul of how the algorithm works, it’s PR and brand protection, with a side of “oh heck, we didn’t expect that to happen.”

Noble unpacks the trouble with corporations that have no public accountability except to shareholders dominating our information landscape and, in particular, how problematic their systems are for women and people of color. The design of our most dominant information gateway poaches unpaid labor, imagines the world to be just like those who write the code to sell attention and adds, and gives us back a reflection of ourselves that is warped by not jumbling information together without context. Its dominance means journalists now have to make their stories more sensational to be found in the din, and that whole communities lose their connections as their own histories are crowded out. (There’s an excellent interview that shows how Yelp has affected one black woman’s business and how the system we may use casually to check out options actually demands constant payments from businesses to make their online profiles more visible while making networks of word-of-mouth less vital.)

This is a book librarians and anyone else who worries about the state of our information systems should check out. I’ll share a few of the many quotes I noted down to give you some flavor.

Algorithmic oppression is not just a glitch in the system but, rather, is fundamental to the operating system of the web (10).

Google’s enviable position as the monopoly leader in the provision of information has allowed its organization of information and customization to be driven by its economic imperatives and has influenced broad swathes of society to see it as the creator and keeper of information culture online, which I am arguing is another form of American imperialism that manifests itself as ‘gatekeeper’ on the web (86).

Algorithms are, and will continue to be, loaded with power (171).

Though in an epilogue, written after Trump’s election, Noble admits her solutions – strengthening the social institutions that are unlikely to get anything but decreased budgets and creating public options - aren't in the cards, she believes we need to change our information systems fundamentally.

Without public funding and adequate information policy that protects the rights to fair representation online, an escalation in the erosion of quality information to inform the public will continue . . . My hope is that the public will reclaim its institutions and direct our resources in service of a multiracial democracy. Now, more than ever, we need libraries, universities, schools, and information resources that will help bolster and further expand democracy for all, rather than shrink the landscape of participation along racial, religious, and gendered lines" (181, 186). show less

The problem with academic writing about the Internet is that it dates so quickly: unlike a popular nonfiction book, where you're frantically googling while reading to check on what's happened since the book came out, you're frantically googling to check on what's happened in the three years since the chapter you're reading went through peer review. I understand it's necessary, but it's intensely frustrating, especially when a book comes out in the awkward time frame this one did.

Still, there's some very good analysis here, on categorization and assumptions and the way systems that were created thoughtlessly can create and reinforce great harm. There's a lot to build on from here.

Most conversations about racism in America focus on individual acts and people. Was LuAnn wearing blackface as Diana Ross racist? When white people use the n-word ironically is it racist? Is Steve King of Iowa racist? (Yes, yes, and yes.) Unfortunately, those conversations consume so much of the air in the room, there is little energy to focus on the structures that reinforce and sustain racism, systems that guarantee that racist oppression and disparities would continue even if not one single racist epithet is uttered ever again anywhere in the world.

The problem is structural or systemic racism is usually invisible, which makes books like Algorithms of Oppression from Safiya Umoja Noble so critical to advancing justice. Noble happened show more to be google “black girls” and the autocomplete prediction that came up was appallingly pornographic. She wondered what kind of messages that sends, how the imprimatur of search, which many people mistake for research, implicitly endorses the results no matter how problematic they may be. This motivated years of research on those algorithms that have so much power to define and shape our world. These are instructions, code that is private and proprietary, code that operates with a kind of impunity. It’s code, how can code be racist?

But these sorts of errors have been part of our information systems all along. Consider the ubiquitous Dewed Decimal System or the Library of Congress classifications. When activists lobbied successfully to get the Library of Congress to stop classifying immigrants without documents as illegal aliens, Congress hurriedly introduced a law to prohibit political influence in the classifications, as though the original classifications were not political already. In another example, although most people in the world belong to other faiths and most of the texts in the world are from other faiths, eighty percent of the classification labels in the system are related to Christianity. Even the geography of Asia and Africa get the short end of the classification stick. No one sat down to create a racist classification system, it was just a matter of attention to and awareness of white culture and attention withheld and ignorance of non-white cultures and sources.

White supremacy is often a matter of simply being the default. By being the default, nothing needs to be said, it just is. Noble makes the critical point that bad search results are the result of decisions made in creating the algorithms that govern what they produce. That they can be modified for better results is evident by Google’s decision to downgrade the rank of porn results. That result ranking can be manipulated is evidenced by the big Search Engine Optimization industry, by Dan Savage’s cyber-prank santorum, and by Google’s own actions.

Noble does an excellent job making her point. She builds her case carefully and expands beyond the readily obvious to looking deeper, at card catalogs, at the Library of Congress, making the argument that if information retrieval is biased, the basic facts needed to redress inequity are not even available. When examining information classification, archival, and search algorithms through the lens of critical race theory, Noble strives to be dispassionate and analytical. It makes those chapters a bit of a struggle. Some might say she becomes “overly academic”, but that reminds me of this tweet.

ok hi what if for 2018 we agree to stop including “the prose is too academic” in reviews of academic books from academic presses by academics writing books that their fields require them to write

— free kellyanne (@DearSplenda) November 23, 2017

It’s actually not that there is too much academic jargon. There isn’t. It is that sometimes she writes dispassionately about something we know she cares about passionately. If we care for justice, we must be passionate about it, too. The thing is, when people write about race and gender and challenge the power structure, they will be attacked. Writing dispassionately, analytically, dare we say academically, is a necessary bulwark against those attacks.

Her passion comes through loud and clear in the final chapter. There is a riff on how technology will not save us that would make Gil Scott-Heron proud. She takes down a lot of popular shibboleths of the technocrats, the idea the colorblind is good, that racism has been solved, that the solution to racism is an individual practice, that we are postracial, that all we need to fix racism is fix the feeder system to education and industry. She presents the evidence and has receipts. This is a myth-busting book, busting the comfortable myth that technology and computers can’t be racist. They are made by people, of course they can be.

I received an e-galley of Algorithms of Oppression from the publisher through NetGalley.

Algorithms of Oppression at NYU Press
Safiya Umoja Noble author site

https://tonstantweaderreviews.wordpress.com/2018/04/08/9781479837243/ show less

The politics of technology design

The internet is not the neutral, unbiased warehouse of all things. Search, for example, is loaded with ingrained prejudices from our culture and history. Safiya Noble, a black feminist, was incensed when she searched ”black girls” on Google and came up with listing after listing of porn. All she wanted was some activities suggestions for her stepdaughter and her visiting cousins. She had to close the laptop lid before they saw the results.

If it weren’t bad enough that only porn resulted, look at Google’s autocomplete suggestions for a search on “women”:
-Women cannot: drive, be bishops, be trusted, speak in church
-Women should not: have rights, work, vote, box
-Women should: stay at home, be show more slaves, be in the kitchen, not speak in church
-Women need to: be put in their places, know their place, be controlled, be disciplined

The result is Algorithms of Oppression, a six year project to determine the extent of this poison, how it came to be, and what should be done. Noble found western society itself at the heart of it.

Google, incredibly, denies any responsibility. It says its algorithm operates on its own and they can’t train it. This is what we call a lie, as Google has managed to abide by all kinds of European directives against hate, Nazi products for sale, and the right to be forgotten. And magically, the black girls search results have been evolving too, as Noble shows in her many screenshots.

We like to believe that what rises to the top in search is whatever is most popular and most relevant. But we fool ourselves. There are classification systems at work, and Noble says blacks have been “contained and constrained“ by them. The search “beautiful” results in an endless page of photos of white women. Not Starry Night, Niagara Falls or the Taj Mahal, but white women. A search for “professor” brings photos of only white men. And a search for “unprofessional hairstyles for work” shows only women of color. As you might guess, “professional hairstyles for work” shows only white women.

And it’s not as if Google has customized the results according to Noble’s search history. She has spent several years using Google in her pursuit of a doctorate in black feminist studies. And this is how Google profiles her.

Basically, Google’s search algorithm represents the white male view of the world, she says, and brings up results to fulfill that need. Black community or society is simply not part of the equation, and therefore not part of the algorithm. Same goes for women.

Noble has a chapter on libraries, because librarians classify everything. They must of course, in order for anyone to do any sort of in-depth research. Yet the very act of classification is discriminatory. Irish Catholic, Korean American, black feminist – are all problems looking for homes. Everyone becomes an “objectified symbol“ to someone else. Leo Buscaglia spent his life ranting against this. Because of these labels, we think we know something about this person, he used to say, but we don’t at all. This same built-in bias shows up in online search. It is not in any way neutral.

As exhaustive as she has tried to be, Noble made no effort to stem the tide. Her screenshots do not also show results when Google’s Family Filter is on, and she never tried searching with negative terms to block the sex listings (Black girls –sex). It’s almost certainly true that most people can’t be bothered with either of these tactics, but Noble should have included their results.

Unfortunately, she concludes that Google search be federally regulated. This despite her entire book demonstrating the embedded, if not innate bias throughout every aspect of western society. It’s not an especially hopeful ending, and really just skirts the whole core issue.

We are nowhere near being postracial.

David Wineberg show less