Hello, everyone. Hello, everyone. I'm Jagdeep Kaur.I am actually a product strategy and operations manager at Google, where I work at the intersection of data analytics, business strategy, and privacy compliance. My focus has been on building, like, responsible, scalable, driven solutions, that actually work for our customers. And I solely focus on, like, advertisers these days. Before Google, I spent, like, several years as a management consultant at McKinsey, help multiple Fortune 500 companies, essentially focusing on, like, their thoniest data and best challenges. So I'm really passionate about making sure, like, how can it's not just about AI. AI is such a, you know, everybody is talking about AI, but even before that, like, the algorithms, how do we make sure they're not just smarter but also are safer and more inclusive?

So today, I'm very excited to dive into this ethical dimensions of AI enabled because as powerful as these technologies are, what we choose to build and how it, like, matters more than ever in this situation. With that in mind, today's session is gonna be all about, like, how do we unpack bias that sneaks into, like, AI? How can we kick it out? And, of course, how you, you know, how yes. You can help design more ethical tech. Right? Can we fully remove it? That's something we'll we will, talk about and we'll dive into. Now I also wanna call out, like, you know, AI is we think about this machine that humans build. So it's not something that, you know, it's built to, like, hate you, but it might ignore you, especially if you're not represented in the dataset or in the data that you are trained on. So imagine being denied a job because an algorithm couldn't interpret a gap in your resume for maternity leave. That's not futuristic. It's actually happening right now.

Bias isn't just about bad PR, but can it can also lead to, like, legal risk or lost trust and even, like, human harm, which, honestly, we even haven't, like, figured, like, what are the what what could be actually possible. So that's, like, what we we realized, like, this is something as more and more. Like, it's like we are living in a sci fi world where more and more features are coming in and new models come in every day. Every day is it's a new model that gets launched. How do we make sure that, with that pace, we also launch on the principles in place? Now, I also wanna call out here. If if we talk if we look at this definition, it seems very easy. Right? Like, okay. Like, you know, let's say we can blame, like, AI developers unintentionally introducing bias, or it can be intentionally introducing bias, on which they are trained on.

And there are two different types of biases that can happen. One is implicit bias. When I say implicit bias, as the name suggests, it's more essentially like unconscious preferences and stereotypes that can be embedded in algorithms. If you think about it, all of us have, like, our own internal biases that when we are working as a developer or, like, as a software engineer or even as, like, a go to market strategist, like, now as my current role, I might have some human stereotypes that I might embed as part of the strategy that we are defining or as part of the algorithm that we are defining.

So this can actually manifest in the form of, like, you know, of favoritism or discrimination against certain racial or ethnic groups, that can perpetrate, I would say, like, into historical inequalities. And the overall, is, like, systematic bias. Now systematic bias in AI, it can occur when, you know, let's say, structural inequalities. And we'll dive into that, what that means. Like, within the development process, that leads to biased outcomes. Now this is like a more formal definition. But if you, going back to the next one, how does AI bias occur? Right? Like, you and I've just I was searching this, and I realized it's not you would see this. Like, it's actually already happening right now. You will see, like, so many news articles that I've I've added here. It's like every day you might have seen it. Right?

And this is actually if you think about this, this is like an unprecedented situation where we don't have regulations. Or we do have, but it's not matching the speed at which the evaluations are coming out, the the way we are using these use cases, which are based on AI, making the decisions for you. How do we ensure that we go beyond this? So let's say okay. Let's go beyond theory. Now in health care, patient recognition, credit scoring, the consequences are already there. Like, if you think about it and the fun fact actually, it's not a fun fact. Not so fun. If you think about it, an AI model used in US hospitals was less likely to recommend black patients for advanced care because it assumed that the health care spending meant less sickness. Yikes. That I feel is, like, definitely a problem area. Right? And remember, like, every misclassified case here is a person.

Like, it impacts a real life, a real missed opportunity. And if I if I have to say about this, like, till the time it does not impact you, you don't realize it. But I feel like if you are, like, the person who's actually developing your solution or developing an algorithm, you wanna make sure you kind of think about all of these in inherent biases that we might embed as part of you know, we are developing a product for a company or a product for a customer or for a product for an advertiser.

Whatever the use case is, this it can have an impact on a person. And what that looks like can vary. Now I think going to that, I'm I also wanna pause to say, like, it can happen. These are, like, high level overview that I'm highlighting of, like, the types of bias that can happen. Now the bias can be based off the algorithm itself, or it could be based on the datasets. Right now, let me focus more on, like, the algorithm itself. Now you will see on the left, which I found it so funny. Like and I hope nobody is doing this. There was a time when I started as a computer scientist. I used to write code. Right?

We didn't have all these codes that all these platforms where we can actually audit and rate the code. Now think of what we are doing here. We are saying, okay. If it's like a a specific ethnicity, grant credit, our versus not. Like, you're manually inputting it. This is like an intentional bias you've added in the algorithm. Now the category that you see here is a complicated, decision tree. If you have worked in, like, if you have worked in developing, a data science model, now we are calling it as AI. Of course, because now we have, like, advanced LLMs, like, all of these algorithms. But if you think about it, before that, we used to have, like, decision trees or random forests.

In there, what you're doing is, like, there are hundreds of decision points. Each one might be biased. Like, a small section of your decision tree built with the machine learning, and a statistical model might suggest such a rule, like, less obvious but equally bad. So how do you so this is also one of the other way. Right? But do you think this is easy to fix? I mean, if you compare with the category, I would say, yeah. I think it's easier to fix. The category that you see here is actually welcome to artificial intelligence. Now one example I would say is a researcher, let's say, inputted phrases such as actually, that's like one of the examples that happened. One of the researchers inputted phrases such as black African doctors caring for white suffering children into an AI program, which was meant to create photorealistic images.

Now the aim was to challenge, you know, the white savior stereotype of helping African children. However, the AI consistently portrayed, like, these children as like black and in 22 out of more than three fifty images like the doctors appeared white. So you can see like how we are in when the when the algorithms become more and more complicated, it becomes more and more challenging to actually reduce those bias because it's like auto learning it and and the the, you know, all the algorithms are not just that, but like the the techniques that are being used.

And I will I will also wrap this slide in a way, like, do you think Gen AI is biased? Now Gen AI is all over. Right? Like, if you are, following the trends here, you have so many, you know, platforms. Like, you have ChargeGPD. You have Gemini. You have LAMA models. You have. Like, you have so many. You have. And the latest research shows that the data created by Gemini can be biased just like any other air models. I don't think that's a surprise. But, for example, like, I believe in 2023, there was an analysis that was done over, like don't quote me over that. I believe it was more than 5,000 images. It was created using the GenAI tool. And I won't name the tool, but it amplified both gender and racial stereotypes. So you can see, like, what it can do. And right now, we are just doing it at our own personal level.

Or, like, now companies are thinking of embedding it as part of their own systems of how they kind of work towards their clients. So you can see how drastic impact it can have on the customers. I think keeping that in mind, I want to call out, like, there are multiple categories of AI. See, like, this is what one way of categorizing it. The more important piece in this is not knowing, Okay, what type of biases it can be. Can there can be multiple that I'm not even covering here. But also being mindful of it. Right? Like, there could be cognitive bias. The reason we call it as cognitive bias because, honestly, as humans, there's so many things that we don't even know that's a bias.

So, like, if you think about unconscious errors and thinking that can affect individual judgments and decisions, and you can think even in our daily lives, it's it could be the impact of. It could be the impact of, where you grew up. Right? Could be the the news or the, you know, the resources you're exposed to. So, and and, honestly, under cognitive biases, there can be more than 180 human biases. That just blows my mind. So you can imagine that all of these human biases, they put steep into machine learning algorithms by either, like, designers unknowingly introducing them to the model, or it could be a training data set, which includes more biases. Now, the overall bucket that I want to call out is the algorithm bias. Now, if you think about, like, the ML softwares or any other technologies that we just talked about, it reinforce existing biases present in the training data or through the, you know, whatever the design we have in the algorithm.

This can happen due to, like, explicit biases in the programming or preexisting beliefs held by, like, the developers. So, again, going back to that, this is more focused on, at very tactical level of how we can, you know, introduce or reinforce ample stereotypes and discrimination against, like, marginalized groups. And category that, as you can see here is actually lack of complete data. If data is not complete, you can imagine you are just showing one side of the story. But if it if I I keep calling it as AI like any other person, but I'll just to, keep it simple. Because if you're training something on very incomplete data, it's not representative of the entire population. And that can also include bias.

Like, for example, like, most psychology research study include results from undergrad students, which are a specific group and do not represent the entire whole population. So how do you solve for that? So me being a nerd, I work with couple of people in my group, to kind of do some research on, like, what could be the, what are things that happen in health care. And the reason is because I was actually diagnosed with a health care problem, and I had to go through a a surgery. And that's when I when I came out and I realized, like, how some of the things are automated, which is great. There's so many new technologies coming in. That's amazing.

But what I realized is, like, this paper actually kind of it dives into the exciting yet complex part of, AI in health care where we explore, like, how these game changing, if you per se like technologies can revolutionize like the patient care from smarter diagnostics to personalized treatment.

But you know what it's also a very famous line that says great power comes with great responsibility. Here in this paper actually if you are interested I can definitely share the link where we spotlight like the critical ethical tightropes we must walk through right which includes like how do you navigate a privacy? As for, like, how do you align yourself with, like, the privacy regulations that you have? How do you tackle, like, the bias to ensure fairness for all? Demanding transparency from, like, black box algorithms. If you remember a few pages before, we the category that we saw, it's more think of this as a black box algorithm. Right? All the rags, all the LLMs, all of that comes under that bucket. So we kind of evaluated that. And our ultimate, of course, like anything, it kind of champions a path forward where we should embrace, like, ethical best practices and foster collaboration.

Now I'll talk more about the collaboration piece. It seems we're types, But, essentially, it is very essential, especially to work in a group where you can align on, to fully harness the full potential of this technology that is coming up. So I think we talked about the types of biases. Now it is also based on the bias based on training data set, which I've mentioned that I'll dive into that. Now this is too much, information. But if I have to summarize it, there are two things. One is, the bias can, based on sharing data set could be it could be a reflection of a historical and subtitled biases.

Now when I say that, for instance, if data on criminal justice outcomes shows a disproportionate number of arrests or for racial groups that you do historical you know discriminatory practices yeah if you are training on that you can imagine like it will associate everything with those racial groups with a high risk of, you know, divisions.

Then you have unrepresentative or incomplete data, which I touch. I won't go into details there. And then you also they are also the biased data generation and collection. It's a huge thing now because we need more and more data. There's also a process of you can generate and collect data. But when you do that, you need to be very careful. What are the decisions that you're making about what data to collect, how to collect it, how to generate it, from whom can we, you know, skew the data. So because this all of this can also introduce buyers in a way, like, based on the demographic groups, or it can generalize well to others as well. So, like, how do you justify that is also gonna be very important.

And the last piece I also wanna call out is, like, you know, there are also biased proxies that I use. Like, algorithms often rely on, like, proxy variables. Like, you will never have perfect data. Somebody who is working at data engineering roles, they know clearly. And, of course, especially for the leadership, and they want to make a decision for, like, what is the next product they want to release or what is the next thing that would benefit our users. We will always have the data. And sometimes it's based on proxy data that can be correlated with protected characteristics. So if these proxies are themselves influenced by historical biases, that can also introduce, like, discriminatory outcomes. I can keep talking about this, but what I'm saying is, like, there are so many different types of biases that you can introduce.

Now I I also wanna call out the impact it can have. Right? Like, there's no surprise. Like, it can have a business reputation that can damage or that can get damaged. Right? So, personally, no surprise. You have a negative media coverage, right, of a bias that can lead to reputational harm. Like, if you remember, like, in data robot actually reported about 42% of actually organizations using or producing, like, AI systems very too extremely concerned about the reputational damage because they want they didn't want to scale it yet without knowing if it passes the at least the basic framework.

it can also erode from numerous trust. Like, if you are not taking care of it and let's say if I'm just not naming any of those platforms, but let's say you are typing something and you're trying to create a report and that just represents one group of people or it misrepresents it, that can also indicate the loss of customer trust.

Right? Like, that's a main incident, and we don't wanna do that in this world where every there's so much competition. There are so many competitors. So keeping all of that in mind, I think we learned a lot about, like, what what are the different types of biases. What can we do about that? Right? Do you believe now will AI ever be completely unbiased? Technically, if you ask me, yes. An AI system can be as good as the quality of its input data. But in reality, AI is unlikely to ever be completely advised as it relies on, like, data created by humans who are inherently biased. Right? So you there's no going away from there. But what we can do about AI bias is to minimize it. Right? Like any other thing in the world, it's I think if I won't go into the philosophy of it, but, like, we wanna make sure we have principles.

And if you think about these principles, they're not different. Like, it's also, like, our values that we have, some values that we want to embed as part of humans we are. Right? So I would say, like, have those principles in mind. Now you might be wondering, like, okay, we talked all about different types of biases. What can we do? Can we completely do, get it away? No. But McKinsey released this report where what what are the few things that you can do? Like, it seems like a naive approach. You can't protect all the classes. Right? Yet this approach may not work. I can tell you that. But you at least you can make some approach to, like, AI development that can actually help minimize bias in AI systems. This is, like, the meat of it.

Like, how do you fix it? Right? Like, now we talked about the problem. We talked about, okay, how what are the challenges that can come with that. How do you fix it? So what I'm trying to do here is actually capture all the things that you need to know. Like, of all, do you fathom the algorithm and data which is being used? Right? So you wanna make sure you assess that, examine the clean clean data set, conduct, like, analysis of, like, subpopulation that is being used, monitor the model over time. These are, like, basic things. Right? Then you should also establish a debiasing strategy. I feel like with all the I'm being part of now AI strategy piece. I wanna make sure that we have a debiasing strategy.

When I say that, it kind of involves tools that help can help you identify potential sources of bias or, like, the operational strategies that we should be following so that we can improve our data collection process using, like, internal red teams and party auditors. It's always beneficial to do that. And then you can also think about, like, human driven processes. Right? Like, identify those and see what we can do to automate it or at least have party to review that. And then I think last piece I would wanna call out here is, like, decide on use cases where automated decision making should be preferred and when humans should not be involved. Right? Again, this is, like, a lot, but the most important piece that kind of wraps it up is, like, how do you diversify your organization? Now diversity is very critical, especially if you're launching solutions for, you know, the entire world or for a specific group.

How do you make sure you kind of have diverse opinions considered as part of your product development or as part of your, algorithms that you're building. So that, I would say, it would be the most critical piece. And I believe, like, most of you might have heard about some of these tools. Like, there are multiple tools that you can follow. You have, like, like, AI governance platforms like Credo. You also have, like, various, IBM Cloud Pak, data, Snowflake. They also have that MLOps. All of that is, like, great. You have a lot of tools these days that are coming up. Like I said, we are living in a sci fi world where we have a lot of things that are coming up, along with, like, how do we make sure that we capture these, the biases or the also it's not just about capturing biases, by the way, but also now more and more regulations are coming in the data privacy piece, which excites me a lot.

And so most critical piece to make sure if you are working on a product or a solution or an algorithm, how do we align with with regulations so that it doesn't come back to bite you? With that in mind, I won't go into the details of this as I I'm mindful of the time here, but this is a high level playbook that I created. If anybody is interested, feel free to reach out to me. But, essentially, one of the few things that I've not designed anything from scratch here, there's so many privacy frameworks that you have that you can leverage. Like, if you are in health care or if you are in, like, overall, like, that same marketing advertisers. So GDPR compliance, HIPAA, all of these things are a great way to kind of think about how do you incorporate that as part of your system.

I think I would like to wrap it up is, like, you don't need to be, like, chief ethics officer to advocate for fairness. Honestly, I would say, like, your thirty second power move should be you should wanna highlight the risk. This could land us in hot water legally and reputationally. Name a framework. Right? I've talked about some of these, but some of the frameworks that I really like is, like, NIST AI risk, management framework or the OECD principle. I think it's a great starting point for you to know what are the things you should consider while creating or developing these solutions. And then propose a solution. Right? Like, any other leader, if you come go to them with the problem, it might not solve that. Right? Like, how do you propose a pilot where you can test one of the use cases with the bio instruction tools, but also, like, highlight, like, what is the learning that you had and how we can scale it across our organization.

So one, last thing that I wanna call out here, I mean, this ethical AI is not like a side hustle. It's the main story. Right? So let's not be the generation actually that just build, like, flying cars without breaks. So and I won't be surprised in next ten years we also have flying cars. So with that in mind, I just wanna call out. Thanks for letting me share my bias thing to a box with you today. And let's keep the conversation going in, you know, either I know we are out of time, but feel free to, leave your questions for me or just reach out to me on LinkedIn. I'm happy to talk about this topic. And I can talk about this day day in, day out. So thank you so much for giving me this up.