AI Needs Black Feminism: Principles and Questions to Guide AI Development and Policy
Serena Oduro19:232021
AI trained on a history of redlining can multiply racism at scale
The neighborhoods redlined on a 1930s map of Oakland still echo in the data that AI uses to make decisions about finance and policing. Serena Oduro, Technology Equity Fellow at the Greenlining Institute, argues that statistical fairness fixes cannot undo that history on their own. So which questions should anyone building, buying or regulating AI ask before calling a system good for everyone?
Learn the Black feminist questions that test who an AI system really serves
Serena Oduro, Technology Equity Fellow at the Greenlining Institute, argues that AI trained on historical data inherits a history of racism, from redlining onward, and can multiply it at scale. She contends that statistical fairness fixes and intent-based anti-discrimination law are not enough, and offers Black feminism, with questions from her poetic guide, as a framework for building and regulating AI that serves those most often harmed.
“It's not just a mathematical problem. It's a problem of justice.”
“Liberation needs to lead for technology to actually be useful or else it'll just become another tool for oppression, as we've seen in other cases of algorithmic bias.”
What you will learn
Explain how historical data shaped by practices like redlining leads AI to reproduce and amplify racism.
Recognize real cases of algorithmic oppression in hiring, healthcare and public services.
Distinguish statistical fairness fixes from justice-oriented approaches to AI and AI policy.
Apply Black feminist questions to test whether an AI product or policy improves well-being for Black women and other marginalized groups.
Challenge claims that a technology is beneficial for everyone by asking who everyone really includes.
Key takeaways from the speaker
Examples of Algorithmic Oppression
An understanding of Black Feminism
How Black Feminism provides a great theoretical framework for equitable and liberating technology and technology policy
5 questions I present from my poetic guide that the audience can use to implement in any field of work/study they are in.
Details
Level
Beginner
Format
Talk
Technical depth
Introductory technical
For
All career stages
Length
19:23
Published
June 2021
Questions this talk answers
Why is AI trained on historical data not race neutral?
An algorithm finds patterns in past data and uses them to predict the future, and that data carries a history of racism such as redlining. As a result, AI can see patterns of racism and exacerbate them on a massive scale.
Why is statistical fairness not enough to fix algorithmic bias?
Oduro notes incompatibilities between legal and machine learning fairness approaches, and argues bias is a social problem. It is a question of what fairness means in a history of racism, not just a mathematical one.
Why does US anti-discrimination law struggle with AI?
US anti-discrimination law focuses on the intent of a human, while discrimination in AI is not really about intention, so the legal tools do not match how harm arises.
What does Black feminism add to AI development and policy?
Black feminism centers the combined oppression of race, gender and class that Black women face. Oduro argues it demands more liberatory outcomes and higher standards of justice than fairness that is not race aware.
How can I apply these ideas in my own work?
Ask how a solution increases individual and communal well-being for Black women or any historically marginalized group, and when something is said to benefit everyone, ask who everyone actually is.
About this talk
“AI Needs Black Feminism: Principles and Questions to Guide AI Development and Policy” will introduce the audience to why Black Feminism provides the theoretical foundations to thwart algorithmic oppression. I will present my poetic guide forthcoming from Meatspace Press, “Do we need AI or do we need Black Feminisms? A Poetic Guide” that provides questions that can guide not only developers, academics, and policymakers interested in algorithmic equity, but all creating or influencing technology who want to ensure that their work is not furthering racial inequality.
Fairness is a justice problem, not just math (53 s)
Who is everyone, really? (57 s)
Speaker 1: Okay, so I'm so excited to be presenting today. I'm Serena Oduro. I'm the Technology Equity Policy Fellow at the Greenlining Institute, which I'll talk about more in a second. And I'm also a writer. So part of what you're going to see today is kind of a mixture between my tech policy work at Greenlining. I mainly work on broadband issues and then also algorithmic equity.
So working on AI. This is specific about AI because that has been kind of the bread and butter of most of my work. And I'm just very excited to be at the Women's Tech Global Conference and to be presenting today on an issue that I think is very important and that I'm very passionate about. This presentation is called AI v. Black communism. Principles and questions to guide AI development and policy. And these questions were developed after a few months for myself in this field of AI, looking at, you know, issues of fairness, bias, bias, looking for solutions to promote justice for communities of color, the use of AI, and how AI also is often not used for just reasons. And I realized that for myself, Black feminism was a framework that I thought is very useful. And also for you all, even if you're not in the field of AI, and I will explain what I mean by AI, because that's a very broad topic. I'll define that. But I hope even if you're not in this field, even if you're just in tech,
or, you know, from different parts of technology, or even not that, that these questions could be useful for you, especially if you're trying to take a racial justice approach and really grounding your work from a specific standpoint. And I think Black feminism is a great standpoint. I'm so excited. I'm excited. I'm going to dive in. So first off, to explain the Green Lining Institute more. So Green Lining is a policy and research institute based in Oakland, California.
We envision a nation where communities of color thrive and race is never a barrier to economic opportunity. So we have two different policy areas, basically economic opportunity, and then also climate. And then within the economic opportunity, we also look at the role of technology. And that's what I focus on with technology policy. So looking at how the lack of access to broadband, and how the impacts of AI is affecting communities of color economic opportunities, which kind of frames some of the work we do, because we work a teeny bit less on like, facial recognition technologies, and more so like, when banks deploy AI, how does that impact people's or communities of color, and their economic opportunities.
So even within AI, there's a lot of room to have to do advocacy. So Green Lining, it's named opposing redlining, which is legal practice of denying services to communities of color. And so in this map behind, so again, Green Lining is based in Oakland, California, this map is from the 1930s of Oakland. And what you can see here with the red areas is those were areas with commonly Black communities, and they were denied financial instruments by banking institutions. So not having access to mortgage loans and other things kind of needed to, you know, buy a house and have access to the type of wealth needed.
And so the yellow, blue, and then the green areas in particular were green areas were more white suburban areas that had access to those type of financial instruments necessary to thrive economically. And so Green Lining is trying to oppose redlining through policy, and a lot of, you know, the discriminatory nature of redlining was used through policy, trying to use policy to actually counteract the effects.
And so a fact where you can see the harms of redlining is this, that even today, you know, for every dollar of wealth a white family has, the median Black family has eight cents. So, you know, redlining that happened from the 1930s, the effects of that, and the racial wealth gap have only continued and led to the racial wealth gap.
And so that's essential to our work, and when we get, when I get into AI, I'll kind of show how this connects, even though it can seem kind of separate. So greenlining is the affirmative and proactive practice of providing economic opportunities to communities of color. So again, how redlining was used to directly extract from and bar and oppress communities of color in order to, you know, put wealth towards white communities.
Greenlining is doing the exact opposite, trying to provide economic opportunities to communities of color. And so how does this connect to AI? Well, an algorithm uses past historical data predictions. And so the interesting thing that you see with the map of 1930s redlines Oakland to the racial wealth gap today is that all those data points that came from, you know, zip codes being barred from access to financial instruments and other forms of discrimination are the same types of data used to make predictions about the future.
And so what we're seeing, and people have seen this a lot, and I'll give some examples in the past few years, though it's been an issue for a bit, is that the data used for AI is biased. I mean, it has a history of racism and it has a history of racism and is not race neutral. And so when we say, let's just make a decision about policing or about the economy, about finances, et cetera, that's not neutral because the same data points used to make guesses about that, because all an algorithm does is, you know, processing massive amounts of data, finding patterns within that data and then making future predictions.
What it can actually do is see a history of racism, patterns of racism, and then proliferally exacerbate racism. And not only is that a problem because racism is a problem, but also AI can also be harder to fight against. I'll talk about the legality of fighting against AI and the difficulties that it presents. And so what you have is a system or technology that can exacerbate racial inequality and other forms of inequality on a massive scale due to the nature of what an AI model does and the racist history that we exist in.
And so another form for this is algorithmic bias. So an example of algorithmic bias is when Amazon wanted to make a hiring algorithm as a recruiting tool, but then it was biased against women because Amazon typically only hires or mainly is filled with men. And so the tool was actually biased against women because it saw, you know, women characteristics as, you know, not desirable for.
And there's also racial bias that will arise in medical algorithms. Again, using past data in the healthcare industry, which tends to be racist, it can then exacerbate and multiply racism through the algorithm. And then you have issues of, in Michigan, an employment agency made 20,000 false fraud accusations through an AI system that basically didn't work well. That was less about bias, but more about what I've noticed is the promises of AI and then what it can actually deliver.
Sometimes we're so excited by future technology that they didn't really prove to the agency that it worked well. And by they, I kind of mean the company that contracted that and that made that algorithm. And so you have the issues of, you know, AI that's not made well, AI that exacerbates and multiplies racism and other types and forms of discrimination.
And so I think people have grown. The need and call for tech companies to just regulate themselves or just to do good science isn't good enough. And so something that people I think have more realized is that policy is needed. And that's where I work in, in green lining. And policy is a combination of a lot of things, ethical standards, research, legal standards, industry practices and political barriers, which are pretty huge.
And not that this is an order of importance or grants-ness, but something that you see within AI is that legal standards, for example, the norms of anti-discrimination law in the US doesn't match well with the way that discrimination tends to arise in AI. Because our anti-discrimination laws are very focused on the intent of a human. But with AI, it's not really about intention.
And so, you know, there's kind of an issue there. Also, you have even research and industry practices that don't match well with policy. And so something that we've done is getting together, you know, data scientists, computer engineers, and different folks to be like, how can we regulate this well? But from this, I think that you do have to have a framework.
And that's why I'm going to talk about why Black feminism, to me, provides a framework to not just go towards AI that doesn't harm people, but AI that's truly beneficial. And I think Black feminism provides an important standpoint for that. And so here I talk about the technological realities, mutations, and norms. And basically what I want to highlight here is that in response to a lot of oppression that has happened due to AI, due to racism that's been exacerbated, that people have, in the AI community, tried to bring about different statistical approaches to fairness.
So can we measure fairness and, you know, calibrate in AI so that it creates fair outcomes and it's not racist? But one thing that this shows on the right side of the screen is that there's a lot of incompatibilities between the legal and machine learning fairness approaches. And I'd also just like to add that in addition to, from a legal perspective, some of the bias correction methods being discriminatory, oftentimes all of these issues really are social.
It's not just a mathematical problem. It's a problem of justice. It's a problem of what does fairness mean in a history of racism? And so that's why I think Black feminism is important because if you try to find a statistical approach without the approach of justice, how will communities of color actually benefit from the use of AI or be prevented from experiencing harm?
So Black feminism and technology. So I wrote Do We Need AI? Do We Need Black Feminism? A Poetic Guide, which was in response to a call for proposals from Meet Space Press about AI hype. And basically they were just talking about how there's been a lot of instances in which there's been promises for AI to be this grand solution.
And not that it doesn't always work, but oftentimes it's how we identify what is AI hype. And I believe that we really need a lens that really has the history of race and also sexism. And that's why I think Black feminism is really great for it. And so I wrote this poetic guide that proposes four questions in addition to some more kind of regular essay type writing that talks about the need of kind of discussing, you know, do we need AI or do we need Black feminisms? And I'll kind of get to my conclusion of that in the end.
But first to just give a bit of background of Black feminism, this is very high rooted. This is at a high level and this is actually the first paragraph of my writing. But on the left, you see, you know, Black feminism is rooted in the knowledge every system and technology has race, gender and class and blah. And so Black feminism is really about the Black female experience.
And often people, what I think is really powerful about it is as a Black woman, and this is why I was due to Black feminism myself, because I feel like it explains so much that I inherently feel that white women can't appeal to white systems of power, even though that they experience the discrimination of being a woman.
And kind of same with Black men, like they're Black and they experience anti-Blackness, but they can appeal to systems of patriarchy. And so oftentimes when they're fighting for liberation, it can be about womanhood or Blackness, but not often those who experience both, which is a unique form of oppression. And so Black feminism and I think the Black woman's standpoint really is searching for liberation and fighting against all those systems of oppression, as mentioned, you know, race, gender, class, and many other axes of oppression, because I can't just choose one.
You know, if I just choose my Blackness, that's not going to help me when I'm also experiencing sexism and sexism as a Black woman, which is even different from a white woman. And so I think this is important for technology because oftentimes even in discussions that I have been about justice and technology, we talk about race separately, you know, gender separately.
But it's like, what about when those come together? And she talked about, you know, Google search engine, how it reinforced racism and sexism against Black women by bringing up pornographic results whenever Black girls was searched. And I think that was a good example of the specific type of oppression that Black women face on the Internet and within technology, but that's sadly very understudied.
But if we want to fight against the algorithmic oppression that has arisen over the past many years, I do think that Black feminism provides and requires more liberatory demands and higher demands for justice than just fairness that's not race aware. Or even just fighting for racism, but not looking at that feminist intersection or the woman's intersection that the Black woman's standpoint presents.
And so some questions that I brought forth and posed in my poetic guide. In case individual and personal and communal well-being for Black women, is this algorithm less biased than a Black feminist? Should Black feminists be hyped? And why is this AI solution for Black women? And I wanted to post these questions because oftentimes the benefits and even drawbacks sometimes of AI is presented from a very race-neutral, gender-neutral standpoint.
It's like, you know, this AI solution is very beneficial, but something that a lot of critical race scholars within the AI field have talked about is that AI oftentimes, when you just say it's beneficial for everyone, that there's no such type of race-neutral standpoint in a history of racism such as in America. And so if you're saying it's beneficial for everyone, and so if you're saying it's beneficial for everyone, that oftentimes just means it's beneficial for those most in power and most privileged, which tends to be white men and folks close to that level of privilege.
And so for me, and so for me, I wanted to pose, why is this AI solution for Black women? How does it increase individual and personal and communal well-being for Black women? Because in the history of U.S. Racial oppression, that's not true when something's said to be beneficial for everyone, that often really means that it's not for Black women or for other minoritized peoples.
And so I think that this is an important question to pose. And I think it's good when developing AI or crafting AI policy, and I think these questions hold myself to a higher standard, because it makes me have to fight against that idea that policies can be race-neutral or that the development of technology can be race-neutral. So this is a question you can ask yourself, you know, if you're at a company and they said, you know, we're going to create this marketing plan, like, everyone will love it. Like, you can think, like, would Black women love it? Like, who's benefiting from this? Who is everyone?
Because often when you say everyone, that's not actually true. It's not everyone. And also, in this poetic guide, you know, I talk about on here, if Black women aren't mentioned, it means Black people will be harmed. But also, if any historically marginalized group is not mentioned, that means that this group will be harmed. What I like about this poetic guide is I really do think you could talk about different groups with these questions. You know, you can say, like, does this algorithm increase individual and personal and communal well-being for folks with disabilities?
Does it for those who don't have U.S. citizenship? There's many different axes of oppression which you could ask these questions. And I think it's important that we normalize, especially within technology, which can be seen as so neutral, but is actually so loaded and really carries and technology informs our our cultural norms, our social norms, as social media has changed so much that these questions can't be asked from a neutral standpoint.
That's not good enough anymore, especially if we want to follow through on, you know, the just use of technology that I think more people are pushing for. And so just in conclusion, and I have three minutes left and I'd like to see if there's any questions, but I wanted to show some of the stanzas that I put in.
So this is my whole poetic guide. It'll come out from Meet Space Press hopefully in July. So I definitely will post it on Twitter. You can follow me there. Also LinkedIn so that folks can read it, especially if you want to implement this in your work. You can follow me there and you can definitely see it. But I have poems like stanzas and then like stanzas and then blocks of text. And this one on the, my right, that says, do we need AI or do we need Black feminisms?
It's not saying we don't need AI, but as long as technology leads, Black women will fall behind. Minoritized people will fall behind. Liberation needs to lead for technology to actually be useful or else it'll just become another tool for oppression, as we've seen in other cases of algorithmic bias. And that's why I think Black feminism is an important tool because I think it blatantly calls out the need for justice in a way that it can really ignore because it can make many people uncomfortable under other frameworks.
And so this piece that I wrote was kind of on my own time. It wasn't just exactly green lighting. Like I said, I'm acting lighting and I'm also a writer myself, but it's interwoven with my own day job work very well, which I appreciate. So if you want to keep up with green lighting work specifically, Vincent Land, who's also a technology equity fellow, who's really awesome and you should follow on Twitter, published algorithmic bias explains how automated decision making becomes automated discrimination.
And also have a forthcoming report called governance seven policy recommendations for algorithmic oppression. And there you can follow more of our policy work. You know, I have a minute left. I tried to get through as much as I could. Like I said, again, to keep up with my policy and advocacy work, you can follow me on Twitter at Serino Duro and also on LinkedIn at Serino Duro.
I had a few things that's going to be published soon. So you can definitely follow that there. In addition to the poetic guide, there's going to be some other things. So please feel free to follow me if you'd like to see that as well. And thank you. I love seeing the comments. Thank you for those who attended and I hope you enjoy the rest of the Women's Tech Global Conference. I'm excited to see other speakers. Thank you.
you
Transcribed automatically; names checked against the speaker record. Select a time to jump there.
Article based on this talk
Why fixing AI bias takes justice, not just better math
Serena Oduro of the Greenlining Institute argues that Black feminism offers the questions AI builders and policymakers need to stop algorithms from multiplying old injustices.
In short
AI learns from historical data, and in a country shaped by practices like redlining, that data carries a history of racism that algorithms can reproduce and amplify at scale. Serena Oduro, Technology Equity Fellow at the Greenlining Institute, argues that statistical fairness methods and intent-based anti-discrimination law cannot solve this alone. She proposes Black feminism, and a set of questions from her poetic guide, as a framework for building and regulating AI that truly benefits the people most often harmed.
Key takeaways
Examples of algorithmic oppression
An understanding of Black feminism
How Black feminism provides a strong theoretical framework for equitable and liberating technology and technology policy
Questions from Oduro's poetic guide that the audience can use in any field of work or study
From redlined maps to biased data
Oduro grounds her argument in the history of her own organization. The Greenlining Institute, a policy and research institute in Oakland, California, takes its name in opposition to redlining, the practice of denying financial services to communities of color. She shows a 1930s map of Oakland where predominantly Black neighborhoods were marked red and denied mortgage loans and other financial instruments, while whiter suburban areas were not. The effects persist: as the speaker cited, for every dollar of wealth a white family has, the median Black family has eight cents.
The link to AI, she explains, is that algorithms process massive amounts of past data, find patterns and make predictions about the future. The same data points that came from zip codes being barred from financial access are the kinds of data used to make predictions today. Calling such a decision neutral, she argues, ignores where the data came from. Worse, the harm can be hard to challenge: "And not only is that a problem because racism is a problem, but also AI can also be harder to fight against." (5:40)
What algorithmic oppression looks like
Oduro walks through several cases. Amazon built a recruiting tool that turned out to be biased against women, because it learned from a workforce made up mainly of men. Medical algorithms trained on past healthcare data have shown racial bias. In Michigan, an employment agency made 20,000 false fraud accusations through an AI system that simply did not work well, a case she sees as less about bias and more about the gap between AI's promises and what it delivers.
"Sometimes we're so excited by future technology that they didn't really prove to the agency that it worked well." (7:21)
Why fairness math and current law fall short
Oduro notes a growing recognition that asking tech companies to regulate themselves, or simply to do good science, is not enough, and that policy is needed. But policy itself runs into mismatches. US anti-discrimination law, she explains, is built around the intent of a human decision maker: "Because our anti-discrimination laws are very focused on the intent of a human. But with AI, it's not really about intention." (8:10)
The AI community has responded to harms with statistical approaches to fairness, trying to measure and calibrate models for fair outcomes. Oduro points to incompatibilities between legal and machine learning fairness approaches, and goes further: some bias correction methods can themselves be discriminatory from a legal perspective, and the underlying issues are social. "It's not just a mathematical problem. It's a problem of justice." (10:02) Without a justice lens, she asks, how will communities of color actually benefit or be protected?
Black feminism as a framework for technology
Oduro wrote her poetic guide, "Do We Need AI or Do We Need Black Feminisms? A Poetic Guide", in response to a call from Meatspace Press about AI hype. Black feminism, she explains, is rooted in the experience of Black women, who face a distinct form of oppression at the intersection of race and gender. White women experience sexism but can appeal to white systems of power; Black men experience anti-Blackness but can appeal to patriarchy. Black women cannot choose one axis to fight on.
She argues this matters for technology because discussions of justice often treat race and gender separately. She cites the example of Google search returning pornographic results when "Black girls" was searched, a specific harm she says remains understudied. "But if we want to fight against the algorithmic oppression that has arisen over the past many years, I do think that Black feminism provides and requires more liberatory demands and higher demands for justice than just fairness that's not race aware." (13:14)
Questions that test who "everyone" really means
At the heart of the guide are questions such as how an algorithm increases individual, personal and communal well-being for Black women, and why a given AI solution is for Black women. Oduro poses them because AI's benefits are usually framed from a race-neutral and gender-neutral standpoint, and in a history of racism, she argues, benefit for everyone often means benefit for those most privileged.
She offers a practical test for any workplace: if a team claims a new plan will be loved by everyone, ask whether Black women would love it and who is really benefiting. "Because often when you say everyone, that's not actually true. It's not everyone." (15:39) The questions also travel: they can be asked about people with disabilities, people without US citizenship, or any historically marginalized group.
Her closing stanza does not reject AI. "Liberation needs to lead for technology to actually be useful or else it'll just become another tool for oppression, as we've seen in other cases of algorithmic bias." (17:17)
Watch the talk
"AI Needs Black Feminism: Principles and Questions to Guide AI Development and Policy" by Serena Oduro, 19 minutes: https://www.womentech.net/video/serena-oduro-ai-needs-black-feminism-principles-and-questions-guide-ai-development-and-policy
About the speaker
Serena Dokuaa Oduro is a technology equity advocate and poet, and Technology Equity Fellow at The Greenlining Institute. In that role, she aims for algorithms to uplift communities of color.
Questions this talk answers
Why is AI trained on historical data not race neutral?
Why is statistical fairness not enough to fix algorithmic bias?
Why does US anti-discrimination law struggle with AI?
What does Black feminism add to AI development and policy?
How can I apply these ideas in my own work?
Drafted with AI from the talk transcript. Editor review pending.
Frameworks and ideas mentioned
Black feminism
Black woman's standpoint
Greenlining
Statistical fairness approaches in machine learning
Resources mentioned
Do We Need AI or Do We Need Black Feminisms? A Poetic Guide (Meatspace Press)
Algorithmic Bias Explained: How Automated Decision-Making Becomes Automated Discrimination (Greenlining Institute report)
Forthcoming Greenlining Institute report with seven policy recommendations on algorithmic governance
Cite this talk
Serena Oduro. "AI Needs Black Feminism: Principles and Questions to Guide AI Development and Policy." WomenTech Network, 2021. https://www.womentech.net/video/serena-oduro-ai-needs-black-feminism-principles-and-questions-guide-ai-development-and-policy