Dr. Carmen Martínez - "Your Chatbot is too sexy"": Stolen Female Capital in the Era of AI
Why a helpful, kind AI assistant keeps ending up with a woman's voice
A woman VP explains that her company picked a female voice for its assistant because testers called it helpful, kind and compassionate, qualities she says the women on her team share. Anyone building conversational AI risks scaling that stereotype. Dr. Carmen Martínez, an applied linguist automating a call center with conversational AI, traces where such choices come from and asks what designers should do before giving AI a body.
Learn how capital theory exposes gender bias in AI personas and three steps to counter it
In short
Dr. Carmen Martínez, a conversational AI expert and applied linguist, argues that gender biases learned in society are transferred into tech products, especially voice assistants given female personas. Drawing on Bourdieu's forms of capital and Catherine Hakim's theory of erotic capital, she shows how female capital, including emotional labor, is commodified, and urges teams to reflect and use inclusive design before giving AI a body.
“They are innate, but in humans, they are mainly learned. They are learned through our social interactions and from our exposition to media and social media and culture in general.”
“but i ask you now what is natural and instinctive about uh humans”
What you will learn
- Explain what biases are and how people learn them through social interaction, media and culture.
- Distinguish Bourdieu's economic, cultural, human and social capital from Catherine Hakim's erotic capital and its seven components.
- Spot how emotional labor stereotypes lead teams to give service-oriented AI a female voice.
- Question research that presents a preference for feminine service providers as natural or instinctive.
- Apply three checks before giving an AI product a persona: reflect, examine the gendered meaning of the service, and design inclusively.
Details
- Level
- Beginner
- Format
- Talk
- Technical depth
- Non-technical
- For
- All career stages
- Length
- 18:31
- Published
- June 2021
Questions this talk answers
Why do so many intelligent personal assistants have female voices?
Teams link service and emotional labor with women; one VP justified her choice by saying testers described the voice as helpful, resourceful, kind and compassionate, qualities she attributed to the women on her team.
What is erotic capital?
A concept from British sociologist Catherine Hakim (2010) describing a mix of visual, aesthetic, social, sexual and physical assets, with seven components ranging from beauty and social skills to fertility and motherhood.
What happens if you give an AI product a persona without reflection?
According to the speaker, it replicates and amplifies existing social biases and the dynamics of gender and power.
How can designers counter gender bias in AI personas?
Think historically from a gender perspective about what a service means when paired with a gender, then use inclusive design to break the canon and offer a variety of interaction styles.
Are biases innate or learned?
The speaker says biases are innate, but in humans they are mainly learned through social interactions, media, social media and culture.
Topics
Highlights
- Why the assistant got a woman's voice (86 s)
- Even the research is biased (71 s)
- Three steps before giving AI a body (76 s)
So yes, your chatbot is too sexy. Stolen female capital in the era of AI. This presentation with a little bit of a weird name. Before I go to the content, I just would like to review what is Flixbass. Flixbass is obviously the company I'm working in and it's a mobility provider launched in Germany in 2013. That is offering trips by bus and training that they are comfortable, sustainable and green. We are operating in more than 30 countries. We have more than 2000 destinations and a total of more of 350,000 connections every day. Very recently, we raised a G series of funding that's insanely will take into the next level of expansion. Myself, yes, and already it all who I am. I'm Karen Martinez. I work as a conversational expert at Flixmobility Tech that is one of the companies under the umbrella of Flixbass. I work in the automation of the call center using conversational artificial intelligence and by training, I'm a doctor of applied linguistics with a specialization in conversation analysis and ethnography of communication. Today, the agenda is basically to review, to start from the beginning and to
the discussion. So biases against women, what they are and how are they transferred to the tech products that we are creating in the industry. For that, I'm going to introduce you to the theories of human capital and erotic capital so we can see the process of appropriation of the female image and the commodification of it from some empirical objective point of view.
And at the end, I will give you some examples of bias in intelligent personal assistants and some general takeaways. This session is not dedicated to have questions. I don't have the time, but right after this session, I'm going to move with Cornelia with my colleague to the boost of Flixbass. So if you have any question, just make me feel free of following me to Flixbass boost after that.
Okay, so let's take it from the beginning. What are biases and what are biases against women? A bias is a disproportionate weight in favor or against an idea, a group or a belief. They are close-minded, they are prejudicial and they are unfair. They are innate, but in humans, they are mainly learned. They are learned through our social interactions and from our exposition to media and social media and culture in general.
Flixbass is a form of bias against women that translates into prejudices and discrimination and basically puts women and girls into a very negative social framing that stops their normal and fair development. Flixbass is a form of bias. Flixbass is a form of bias against women. Flixbass is a form of bias against women against women. Flixbass is a form of bias against women against women against women.
position and has also emotional disengagement so if in their case we would be speaking about women we would be speaking about this humanization but they are just software agents and so in the case of the artificial reality they are providing visual entertainment so what is happening is that the patriarchal discourse has again appropriated the female capital to men's desires and commodified and how can we see how the value has been appropriated well there is two theories in sociology that actually helps us to understand the process better so bordo in the 80s came out with the theory of human capital in which he said that the value that we assign in society to individual it has basically four factors everybody has an economical capital that is the sum of assets and
economic gains that you have in terms of money or properties after everybody has cultural capital that is the amount of knowledge you have about information that is socially valuable like art cinema opera literature general culture and so on everybody has human capital that basically translates into your education qualifications in the form of degrees work trainings and basically all the knowledge and ability that you can exchange in the job market for a salary and people have also social capitals that is the ability or not to belong to a certain network of people that could be translated into financial gains if i think that many of you are female founders so you are very much aware of how important you are to be able to be able to be able to deal with your education and to be in the right cycle in order to have business ventures and also these social networks can
help you in society to influence to have more influence and power let's put as an example the movement of black life matters or today no women in touch and so in 2010 basically a british sociologist uh dr catherine hakeen uh came out with the idea of erotic capital because she found she found that basically uh the way in which we are giving value to people in our society it cannot be only explained by the theories of board you so uh she came out with the idea that everybody has also erotic capital that is uh a combination of uh a series of assets that are visual aesthetic social social sexual and also physical and erotic capital is not uh it doesn't belong only to women they belong
equally to men and to women but historically uh women are supposed to exploit it more and also because in the case of women high levels of erotic capital are related into better access into marriage markets and therefore uh social mobility and financial gains uh there are seven components of erotic capital the first one is beauty uh real beauty is uh universally valorized and very short in supply therefore is very much demanded the second one is sexual attractiveness sexual attractiveness or sex appeal like the classical one of uh marilyn monroe is basically the power of attraction that your femininity or your masculinity has over other people and also permeates all your social relationships in the daily life real sexual attractiveness real uh sex appeal is very short in
supply and therefore is very highly demanded after that we have social skill in interaction that is the capacity and the know-how of how to behave properly in all kind of social interactions and as subcomponent of social skills in interaction we have the capacity for emotional labor and the capacity of emotional labor it has been given to women uh traditionally in our cultures also because uh emotional care is part of motherhood but it has been reinforced and this is the reason because the professions that require high levels of of emotional labor like nurses or fly attendants or customer care service they are classically uh assigned it and they are expected fulfilled by women okay the rest of the component of the erotic capital they are not so
complex we have lively next that's there are is a combination of physical energy and a good mental shape sexual uh competence that is the uh combination of uh uh abilities that make you a sexually competent partner social presentation that is the amount of effort of money that you point if you put into makeup and clothes in order to convey a certain social image and after that we have fertility and motherhood that is the capacity of procreation you have and in the case of motherhood the capacity that your body your female body has to uh create a life out of itself and this is the the only part of the erotic capital that belongs exclusively only to women okay so again i go back to the
original idea of the presentation there is a female there is a female human capital that has been appropriated by a patriarchal discourse basically going through these components that i have presented to you through from bordeaux and from bordeaux and from hakim and basically appropriating what is most convenient to commodify to other men but these ideas by having this product presence in our markets and these ideas in our culture permeates everybody and basically women are also a part of this pro of this problem and in our daily work in our daily life we also uh help to transfer these biases not only further inside of our society the women that they are um active agents in the tech industry they are also reinforcing them in artificial intelligence or in general in software products so what i'm gonna show you is the classical
example of bias in the voice industry that is the one that i belong and where people think that there is a innate somehow preference for female voices intelligent personal assistants so what i'm showing you next is basically the transcription of of the interview of a vp a woman vp of a very big corporation in which she's explaining why they choose a female voice for the intelligent personal assistant they created and since saying the the following thing literally so she said we actually tested a lot of voices and did a lot of research and many of the customers that were part of this test actually prefer the voice that we originally selected for the intelligent personal assistant of course everything is biased from the beginning and she continues explaining the following thing and it's also because they describe the
voice as helpful resourceful kind and compassionate of course it's an intelligent personal assistant so it's a service provider so therefore we need the voice of a woman and she's actually going deeper uh explaining about those stereotypes are present in her organization these are also some of the qualities we find in our team at the company a lot of women have these qualities on the team all right i guess that they don't require it from men because they are just men and so of course because it's an assistant that is providing a service and is expected to give a lot of emotional work she's also saying that we also wanted the intelligent personal assistant to be a personal friend a companion a colleague a family member and so we wanted to find a voice that
that was warm and inviting to engage with and that's how we came to select the boys okay so it has to be a woman and actually if we try to do and to go to the through the same process that they went through and to try to find insight in the research we will find that also the research is completely biased and and this two authors in 2003 they repeat the same of course because customers create a perspective uh perception of natural and instinctive affection when a woman delivers a service service provision should be feminine but i ask you now what is natural and instinctive about uh humans i mean like we are humans because we are not animals is because we have cultural standards and society and standards in our society
and this second author in 2007 basically gets into another conclusion that customers like to receive services from people from the opposite gender but if we go even deeper into this bias we will not find information but people with other sexual orientations and gender identities feels about all of this so what can we do just to stop and reflect about biases and if i have to give you some takeaways uh from this experience of going through these uh ideas of bias and how are they transferred into artificial intelligence intelligence especially in my field i would say that first if you are going to create a persona or to give a body to an artificial intelligent power product without reflecting at all what you are going to do is to
replicate and what is worth to amplify the current social biases and the dynamics of gender and power so it's not that we are repeating is that we are making them very much larger with the help of artificial intelligence and the next step is just to think historically from a gender perspective uh what is the meaning that we are assigning to that uh service in combination with the gender are we aligned with them and are we aligned with them especially if as women in touch and so the last step would be to use inclusive design as a weapon to neutralize the biases basically by breaking the canon doing the opposite what they were expecting you to do if you have the uh enough arguments to proceed like that and also you can provide
enough variety for interaction styles but i think that at the current stage of the industry and this uh current state of alignment of such problems for women in touch i think that what we have to do is to stop and think first and about everything and vision and inclusive future so thank you very much thank you very much
Transcribed automatically; names checked against the speaker record. Select a time to jump there.
Frameworks and ideas mentioned
- Bourdieu's forms of capital
- Erotic capital (Catherine Hakim)
- Inclusive design
Cite this talk
WomenTech Network. "Dr. Carmen Martínez - "Your Chatbot is too sexy"": Stolen Female Capital in the Era of AI." WomenTech Network, 2021. https://www.womentech.net/video/dr-carmen-martinez-your-chatbot-too-sexy-stolen-female-capital-in-era-ai
Related talks
Browse the libraryResponsible Tech in an Era of Hostile Tech
Most hostile technology was never meant to be hostile
Voice AI: The Future is By Everyone and For Everyone
Your voice assistant may grade you a D plus, and it isn't your fault
Does Artificial Intelligence (AI) Support Diversity, Equity and Inclusion?
A friend aged 60 never saw the job ad, and that was by design
Overcoming Gender Bias in AI
AI learns our gender bias, and in cars and clinics that can be dangerous
Impact of AI on Society
Round-trip 'she's a doctor' through Hungarian and AI makes her the nurse
Does Artificial Intelligence (AI) Support Diversity, Equity and Inclusion?
A job ad hidden from anyone over 54 shows how AI quietly discriminates