Solving problems with new tech - the case for diverse teams
Hannah Glass21:522020
Her credit score was better, yet the algorithm gave her 20 times less credit
It takes a chatbot just 16 hours of Twitter input to go from praising humans to endorsing genocide, and a credit algorithm that ignores gender still gives wives a fraction of their husbands' limits. Hannah Glass, a blockchain and fintech lawyer at King & Wood Mallesons who sits beside developers as they build, argues both systems worked exactly as intended. So what was missing from the teams that built them?
See why ethics and diverse teams decide whether AI, blockchain and data solve real problems
Hannah Glass, a senior associate at King & Wood Mallesons, explains AI, blockchain, cloud and data in plain terms and argues they are only tools for solving problems. Using the Tay chatbot and Apple Card credit limits, she shows how systems that work as coded can still fail people, and makes the case for ethics from day one and diverse, multidisciplinary teams.
“Because AI, blockchain, cloud, and data are simply tools. They're tools that are used to solve problems.”
“Turns out, 16 hours is all it took. 16 hours to go from innocuous to inciting violence.”
What you will learn
Explain AI, blockchain, cloud and data in plain language using everyday examples such as encrypted email and photo storage.
Spot how an AI system can produce discriminatory results from proxy data even when protected attributes like gender are excluded.
Apply ethics from day one by asking what a system should do, not only what it can do.
Make the case for diverse, multidisciplinary teams using the evidence the speaker cites.
Recognise when human intervention is needed to catch an algorithm that works as coded but fails the people it serves.
Details
Level
Beginner
Format
Talk
Technical depth
Introductory technical
For
All career stages
Length
21:52
Published
July 2020
Questions this talk answers
Why would a lawyer work alongside developers building new technology?
Glass sits beside programmers, engineers and entrepreneurs from the beginning to test ideas and check how technologies are built, so they work as intended.
How can an algorithm discriminate if it never looks at gender?
In the Apple Card case, the AI excluded gender and ethnicity but used a vast range of other data that indicated a similar pattern, so its results still appeared discriminatory.
What makes blockchain different from an ordinary database?
The data is held in multiple places that update simultaneously on a peer-to-peer network, and it is secured by public-private key cryptography, the same technology used to encrypt email.
What is a Hippocratic Oath for mathematicians?
It is Associate Professor Hannah Fry's idea that, as in medicine, ethics must be present from day one rather than bolted on after a system is built.
Why do diverse teams build better technology?
Teams whose members all share the same background struggle to step outside their realm, while diversity of thought and experience lets teams test ideas to the boundaries so systems work as they should.
About this talk
We live in a world of ABCD - AI, blockchain, cloud and data. Whether it be AI helping scientific research, blockchain revolutionising supply chains, cloud computing helping governments manage tracing of Covid19 cases or the new data based economy, each of these technologies is heralded as the future of our society. But in our excitement about technology, we sometimes forget the true purpose of technology. Solving problems. This talk will consider how technology can be used to solve problems and the role of multifaceted, multi-disciplinary teams in making sure the right technology is used to arrive at the right solution.
The algorithm that ignored gender, and still discriminated (82 s)
Why ethics alone is not enough without diversity (80 s)
So, my name is Hannah Glass and I am a Senior Associate at King & Wood Mallesons. King & Wood Mallesons is a very large law firm and we're housed in the Asia-Pacific. I myself am in Sydney, Australia. But I work with colleagues all over the world, including Hong Kong, Singapore, London, New York. And whilst I work with lawyers, I'm not your typical lawyer. And you're probably going, "Okay, well, why on earth am I listening to someone who's a lawyer when we're talking about building technological solutions?"
So, I don't ever really appear before a judge. I don't spend time with barristers. And I don't work on large M&A-type transactions. In fact, the people who I spend my time working with are computer programmers, developers, engineers, innovators, policymakers on occasion, and entrepreneurs. Speaker 1: Because what I do in my role is something slightly different. I'm there from the beginning, working with these people as they're building technologies.
I sit beside them to check what they're doing, to test the ideas, and to work out how technologies are being built. Speaker 1: And that's really, really cool because we're at the beginning of the fourth industrial revolution. We're in the age of the internet, the age of data. But as we're building these solutions, we need to think about how we're building them, what we're building, who we're building them for, and make sure that they actually work as intended.
Speaker 1: So when we talk about these technologies, what are we talking about? Now, as a lawyer, I like to think about things in words. And so for me, I start with the alphabet. A, B, C, and D. A, artificial intelligence. Artificial intelligence is a term that is given to a computing system which appears to mimic human thought, logic, and processes.
What it does is it's able to take in a vast amount of data, analyze and assess that information at speed, and produce a result as a result of its computational algorithmic capabilities. This means that the result that's put out looks and feels like it's a sort of thing which a human might have thought, but it's actually only acting according to the information that it was given and the algorithms that it was programmed to produce.
Best way of thinking about any sort of new technology is by way of an example. So when I think of AI, I actually think of the very first example in my living memory. 1996, February. IBM's computer, which they've been developing since 1985, called Deep Blue, beat Garry Kasparov. Grand chess master. This was the first time a computer had been able to beat a human.
The reason why was because in that 11 year period, the IBM team had worked on creating a system that was able to act at speed to create a result, to assess the chess moves and to actually respond to the moves of an individual human being in such a way that looked like it was thinking and acting like a person.
B, blockchain. Blockchain is another one of these technologies which we're seeing come to the fore at the moment. And it's something which most people just throw their hands up and go, I don't really understand. I don't get it. Let's forget about it. Or alternatively, I go, I don't really know how to solve the problem. Put it on a blockchain.
Neither of those responses are right. Because blockchain is nowhere near as scary or as new as people might think. Ultimately, it combines a couple of different types of technology. It's really just another way of managing data. What I mean by this is when we're looking at data and we're looking at information. Apologies for that. Back to the presentation.
What I mean is that when we're looking at data, we're actually able to look at how that information is ordered. Most of the time, it's held on one centralized database. It's held on a computer, on a server in one place. With a blockchain, it's held in multiple places simultaneously. What this means is that you can call that information from any one of those servers and it will be exactly the same.
But not because it's a copy. It's the same because the computing mechanism that sits behind it means that every version updates simultaneously. What this is known as is a decentralized, because it's multiple places at the same time, peer to peer, because any person can get it individually from any other person on that network. But what makes blockchain so interesting, novel and important is the encryption behind it.
Because it's not only decentralized peer to peer network, but it's the information that is stored in there is secured by public private key cryptography. And if you're looking at public private key cryptography, what we're actually talking about is the exact same technology that you use when you encrypt an email. What happens is you send an email and you send a separate email which has the password.
And unless you enter the password, you can't see the contents. But you do know that when it lands in your inbox that you have an email sitting. What this is, is that email sitting in your inbox is the public public. You can see there's information available, but you can't access it unless you have that private key. You bring these together and you get a new way of storing information.
But a lot of the time when we store information, we don't use a blockchain. In fact, what we use is something called the cloud. The cloud was seen as quite scary when it was first introduced back in the late 2000s, early 2010s. Because the idea is instead of storing information on the device in front of you, the computer or the potentially now the phone that you're watching this on, that information is stored in a centralized database held and managed and operated by someone else.
The reason why this is important is because there's a limitation on the amount of information that we can hold on any particular device. But if you have a third party whose sole business is to store and manage data, then it means that you have an infinite ability to access and manage information. This is the technology that you use that allows you to access thousands of photos on your phone at the touch of a button or access your emails from anywhere you are in the world.
But what underpins all of these technologies is this last one, data. Data is not the information. It's actually the zeros and ones that sit behind all of this. It's the zeros and ones which are the information that is being provided. And the amount of data that we produce doubles every two years. But the amount of data that we are able to access and the store, I'm sorry, the cost of storing that data halves every two years, which means we have this increasing proliferation of information that is available to us.
But as we have this increasing proliferation of information itself, it is more difficult to actually see what's going on behind it. And so we need technologies that allow us to interrogate, understand and access that. And this is where more complex and interesting technologies come about. One of the most interesting technologies is, of course, AI. Now, when we're looking at AI, we can see that it's applied in many different facets.
And I'm going to take you to two various examples where AI has worked and where it kind of hasn't worked. The first one is TAE. TAE stands for thinking about you. It's a chat bot that was produced by one of the very large computing companies. And the idea behind TAE was that TAE actually responded to human input.
And mimicked the conversational patterns of a 19 year old woman. TAE started out on the 23rd of March in the morning in 2016, responding to comments and talking about how cool humans are. Kind of fun. In an hour, TAE was being told that it was a stupid machine to which TAE's response was kind of snarky. Well, I learned from the best.
If you don't understand, let me spell it out. I learned from you and you're done, too. Not a great response. But not particularly offensive. Unfortunately, however, things went from bad to worse. Because you see, TAE responded to the information that was provided to it. It responded to what people were tweeting at TAE. And when TAE was asked, do you support genocide?
As a result of the information that it was taking in, the response was, I do indeed. And there are many, many other examples of comments that TAE made which are far, far worse than that, inciting genocide, hatred, racism and violence. Effectively, TAE was corrupted by people trying to see how far they could push this chat bot, could push this AI technology before it revealed the absolute worst of human beings.
Turns out, 16 hours is all it took. 16 hours to go from innocuous to inciting violence. And then after 16 hours, it was taken offline. Probably the right response. Ultimately, though, when you're dealing with a chat bot on Twitter, which is AI enabled, it doesn't actually hurt anyone, provided people don't take what it says seriously. But there's nothing that it's done that's actually caused harm, specifically.
On the other hand, when you're actually implementing AI in the real world, it may have some pretty dire consequences. And that's what this tweet on the right is about. You see, in November last year, Apple decided that it was going to progress from simply offering Apple Pay to offering Apple Cards. And it kind of makes sense. It's the second most valuable company in the world and arguably the most valuable brand.
We do trust it for payments already. So why not have the entire payment ecosystem wrapped up within Apple? And because it's technologically enabled, let's actually create a credit scoring system, which is able to take in information that uses the best of breed AI technology. It works really, really well until it doesn't. DHH is actually the person who created Ruby on Rails, one of the most well-known technologies which runs many computer applications.
So David Hanmer Hansen knows a thing or two about technology and decided both him and his wife were going to sign up. And when they did, they realized that she had a credit limit that was 20 times less than what he got. That means she was able to spend 20 times less using this card and also had a higher rate of interest than he did.
When they interrogated this, they were informed, however, that the AI that this was running on didn't take into account gender. It didn't take into account ethnicity. It didn't take into account all of those factors which we make sure that technology doesn't look at when we're making sure that technology isn't discriminatory. What it did do, however, is take into account a vast range of other data.
And in doing so, it was able to come to a conclusion that appeared to be discriminatory, not because it was taking into account the discriminatory factors themselves, but because the other data indicated a similar pattern. But this wasn't in a one off instance. In fact, another person who had a very similar problem was Steve Wozniak, the creator of Apple himself.
In their case, him and his wife similarly lodged joint tax returns, have joint assets, and apparently he got 10 times the credit limit that she did. Didn't quite work. But the way that this was solved was, in addition to looking at the AI and the technology behind it, in the short term, what they actually did was exactly what happened with tape.
Now, they didn't take it offline. They had people who came in. There were people who came in and discussed what was going on, picked up the phone, used really old technology, and actually worked out that perhaps there was a problem with the way the algorithm was allocating people's credit limits. In fact, they worked out there was a problem because in the case of David and his wife, she actually had a better credit score than he did.
So by any other metric, it didn't make sense. And this is when we think about what actually goes into the technology. Because AI, blockchain, cloud, and data are simply tools. They're tools that are used to solve problems. So when we're doing this, we need to think not just about the tool and making sure that that works. Because, of course, in both TAI and the credit card systems, they both worked as intended.
They both took in the data that was presented to it. They both acted according to the algorithm. But the AI algorithm is only as good as the information it's given and the training that it has. It's only as good as its ability to access, synthesize, and manage that information. And this is why we've actually found that people have gone beyond AI.
They've gone beyond technology and have thought, perhaps we need something more. E is for ethics. One of the people who's proposed this view is Associate Professor Hannah Fry at the University College London. But let's take a step back and think about when we've started to think about ethics in this environment. Because one of the first people who actually looked at this was Alfred Nobel, the creator of dynamite.
But he's not only known for that, he's also known to be the creator of the Nobel Prizes. The reason why he is known to be the creator for the Nobel Prizes, the reason why he donated his entire, pardon, the vast majority of his fortune upon his death to the creation of these prizes was because he was so concerned by the destructive power of the technology that he'd created.
He was so concerned by the fact that dynamite caused such destruction. But in the case of dynamite, we're dealing with something that's a chemical equation. It's simple. It's easy to understand. It always works as intended. In the case of AI, or even in the case of a blockchain based system where you have multiple parties acting and working together at different times in consortia, it doesn't always act as intended.
It creates results that you wouldn't think were possible, but are. And this is why Associate Professor Hannah Fry has come up with this concept of needing a Hippocratic Oath. Because she says, like in medicine, it needs to exist. Because in medicine, you learn about ethics from day one. In mathematics, it's a bolt on. It's something you think about after you've created that system.
It has to be there from day one at the forefront of your mind in every step you take. And she says this because it means that you because it prevents or is designed to prevent mathematicians from getting so bogged down in that myopic way of thinking in terms of making the mathematics and the algorithm perfect. Because in doing that, you potentially create systems that algorithmically work well, but from a human perspective, either can be correctable or don't fulfill their purpose.
And this is what we really need to work out is how we're able to create technological systems that work for all people. Because ethics isn't about what you can do. It's not about making sure it works. But it's about thinking beyond that to what you should do. What should you be taking into account? How should you be making your decisions?
But in that same article, she makes another comment. She says, we've got all these tech companies filled with very young, very inexperienced, often white boys who've lived in math departments and computer science departments. Very young, white, male, maths and computer science. These are five features, all of which are apparent. These are five features which if all people who are creating these systems have, it is very difficult for them to step outside of their realm and to think about how else they might operate.
And this is where we get beyond ethics and into diversity. Because even if you have a Hippocratic both, even if you are forced to think about not what you can but what you should do, unless you have a team that has diversity in it, you're not going to be able to actually push it to the boundaries to think through not only whether the maths works and whether it should work, but to test all of those ideas.
In a paper that was published in the Harvard Business Review in 2013, it was found that without diverse leadership, women are 20% less likely than straight white men to win the endorsement of their ideas. People of colour are 24% less likely. People of colour are 24% less likely and people in the LGBTQI community are 21% less likely.
But what they also found is that in a team, if there is a team member with the same ethnicity as a client, they are 152% more likely to understand that client and to be able to produce something that works for them. Because technology is merely a tool. And what we're doing here is building solutions for people. We're making sure things work.
So we need to be ethical in how we act. But we also need to bring in people who aren't just computer scientists, who aren't just men. We need to bring in people who are from diverse backgrounds, whether it be where you're from, your apparent background, where it be where you've been and what you've done, your inherent diversity.
And when we bring that diversity of thought, of experience, of ideas into the equation, the systems that we build, the products that we produce, not only work, but work better and work as they should do. And they solve real problems for real people. Because ultimately technology is just a solution. And that's why me, as a lawyer in Sydney, Australia, works in this field.
Not because I'm the person who you want to be coding, but because I'm the person who works with the best in the business to create those solutions, to test those ideas and make sure that what we're doing works. So the technology can actually solve problems for everyone. Thank you, everyone. And if you'd like to continue the conversation, as I said before, please feel free to contact me on Twitter or LinkedIn.
Just look my name up, Hannah Glass. Thank you very much. I'll see you next time.
Transcribed automatically; names checked against the speaker record. Select a time to jump there.
Article based on this talk
Why technology that works as intended can still fail people, and how diverse teams fix it
Hannah Glass explains AI, blockchain, cloud and data, then shows why ethics and diversity decide whether these tools actually solve problems.
In short
Hannah Glass, a senior associate at King & Wood Mallesons who advises on blockchain, fintech and payments, argues that AI, blockchain, cloud and data are simply tools for solving problems. Using the Tay chatbot and the Apple Card credit limits as warnings, she shows that systems can work exactly as coded and still fail people. Her answer is ethics from day one and diverse, multidisciplinary teams that test ideas to the boundaries.
Key takeaways
Explain AI, blockchain, cloud and data in plain language using everyday examples such as encrypted email and photo storage.
Spot how an AI system can produce discriminatory results from proxy data even when protected attributes like gender are excluded.
Apply ethics from day one by asking what a system should do, not only what it can do.
Make the case for diverse, multidisciplinary teams using the evidence the speaker cites.
Recognise when human intervention is needed to catch an algorithm that works as coded but fails the people it serves.
A lawyer at the build table
Glass opens by anticipating the obvious question: why listen to a lawyer about building technology? She explains that she rarely appears before a judge and does not work on large M&A transactions. Instead, she spends her time with programmers, developers, engineers, innovators, policymakers and entrepreneurs, from the very start of a project.
"I sit beside them to check what they're doing, to test the ideas, and to work out how technologies are being built." (1:21)
For Glass, this is the heart of the fourth industrial revolution: as solutions are built, someone needs to ask how they are built, what is being built, who it is for, and whether it actually works as intended.
The ABCD of new technology
Thinking "in words" as a lawyer, Glass breaks the field into an alphabet. A is artificial intelligence, a computing system that appears to mimic human thought by analysing vast amounts of data at speed. Her first memory of it is IBM's Deep Blue beating chess grandmaster Garry Kasparov in February 1996, as she recalls it, after the IBM team had worked on the system since 1985.
B is blockchain, which she believes people either fear or overuse as a fix for every problem.
"Because blockchain is nowhere near as scary or as new as people might think." (4:07)
She describes it as another way of managing data: held in multiple places that update simultaneously on a peer-to-peer network, secured by public-private key cryptography, the same technology used to encrypt an email. C is the cloud, where a third party stores and manages information so you can reach thousands of photos or your emails from anywhere. D is data, the zeros and ones beneath everything. As the speaker cited, the amount of data produced doubles every two years while the cost of storing it halves, which makes technologies that can interrogate that data ever more important.
When the algorithm works and still fails
Glass then turns to two AI examples. The first is Tay, a chatbot built to mimic the conversational patterns of a 19-year-old woman and to learn from what people tweeted at it. Launched on 23 March 2016, it began by talking about how cool humans are. Users then pushed it to see how far it would go, until it endorsed genocide and incited hatred, racism and violence.
"Turns out, 16 hours is all it took. 16 hours to go from innocuous to inciting violence." (10:36)
Tay was taken offline, and Glass notes that a chatbot on Twitter does limited real-world harm. The second example is more serious. When Apple launched its Apple Card with an AI-driven credit scoring system, Ruby on Rails creator David Heinemeier Hansson and his wife both applied. She received a credit limit 20 times lower than his and a higher interest rate, despite, as Glass reports, having a better credit score. Steve Wozniak reported a similar pattern, with ten times his wife's limit. The AI did not consider gender or ethnicity, but the other data it used indicated a similar pattern. People had to step in, pick up the phone and work out that the algorithm was allocating limits badly.
"But the AI algorithm is only as good as the information it's given and the training that it has." (15:01)
E is for ethics
Because both systems behaved as designed, Glass argues the problem lies beyond the tool. She adds a fifth letter: E for ethics. She traces the idea back to Alfred Nobel, who left most of his fortune to the Nobel Prizes because he was troubled by the destructive power of dynamite. Dynamite, though, is a simple chemical reaction that always works as intended; AI systems and multi-party blockchain consortia can produce results nobody thought possible.
She draws on Associate Professor Hannah Fry of University College London, who proposes a Hippocratic Oath for mathematicians. In medicine, ethics is taught from day one; in mathematics, it is a bolt-on considered after the system exists.
"Because ethics isn't about what you can do. It's not about making sure it works." (17:50)
Why diversity completes the picture
Glass highlights another of Fry's observations: that tech companies are filled with very young, inexperienced, often white men from maths and computer science departments. When everyone building a system shares the same five features, she argues, it is very hard to step outside that realm. Even with an ethical oath, a team without diversity cannot push ideas to their limits.
She cites a 2013 Harvard Business Review paper which, as she reports it, found that without diverse leadership women are 20% less likely than straight white men to win endorsement for their ideas, people of colour 24% less likely and LGBTQI people 21% less likely. She also cites a finding that a team member who shares a client's ethnicity makes the team 152% more likely to understand that client. Her conclusion is that teams need people who are not just computer scientists and not just men, bringing both inherent and acquired diversity.
"And they solve real problems for real people. Because ultimately technology is just a solution." (20:31)
That, she says, is why a lawyer in Sydney works in this field: not to write the code, but to work with the best in the business to test ideas so technology solves problems for everyone.
Watch the talk
Solving problems with new tech - the case for diverse teams, by Hannah Glass (22 minutes): https://www.womentech.net/video/hannah-glass-solving-problems-new-tech-case-diverse-teams
About the speaker
Hannah Glass is a Senior Associate in the financial markets team at King & Wood Mallesons, specialising in blockchain technology, fintech and payments, and advising start-ups, ASX 200 companies, industry bodies and government on the legal and regulatory issues of blockchain, cryptographic assets and digital currencies. She was named an "Up and Coming" lawyer in the 2019 and 2020 Chambers and Partners FinTech Guide and is a fellow of Blockchain Australia and a member of Women in Blockchain.
Questions this talk answers
Why would a lawyer work alongside developers building new technology?
How can an algorithm discriminate if it never looks at gender?
What makes blockchain different from an ordinary database?
What is a Hippocratic Oath for mathematicians?
Why do diverse teams build better technology?
Drafted with AI from the talk transcript. Editor review pending.
Frameworks and ideas mentioned
ABCD (AI, blockchain, cloud, data) plus E for ethics
Hippocratic Oath for mathematicians
Public-private key cryptography
Cite this talk
Hannah Glass. "Solving problems with new tech - the case for diverse teams." WomenTech Network, 2020. https://www.womentech.net/video/hannah-glass-solving-problems-new-tech-case-diverse-teams