0:0016:50

Responsible Tech in an Era of Hostile Tech

Rebecca Parsons 16:50 2022

Most hostile technology was never meant to be hostile

A teaching hospital's ICU model said nothing about asthma patients, because policy had always sent every one of them straight to intensive care: no malice, yet potentially deadly. Thoughtworks CTO Rebecca Parsons argues that as technology reaches credit, criminal justice and medicine, unintended harm is the real risk. So how can teams spot the people their products quietly fail?

Learn how to surface unintended consequences and look beyond your target audience

Watch on YouTube

In short

Rebecca Parsons, CTO of Thoughtworks, argues that most technology that ends up hostile to people was never intended to be, and that technologists have a responsibility to anticipate unintended consequences. Using examples from a biased hospital ICU model, facial recognition, credit scoring and autonomous driving, she shows why teams must look beyond their target audience and introduces the free Responsible Tech Playbook of facilitation techniques.

“But the vast majority of technology that ends up being hostile often isn't intended to be.”
“We need to lift our head up and instead of just focusing on our target audience, look around.”

What you will learn

  • Distinguish intended consequences of a technology from unintended and second order effects.
  • Recognise how hidden bias in training data can make a well-meant model harmful, as in the asthma ICU example.
  • Question what counts as fair in your own context, for example when using proxies in credit scoring.
  • Look beyond your target audience to stakeholders who are invisible during design.
  • Apply facilitation techniques such as the tarot cards of tech and consequence scanning from the Responsible Tech Playbook.

Key takeaways from the speaker

  • Technologists need to know how to identify unintended consequences of their tech choices and how to mitigate the risks or amplify the opportunities those consequences present.
  • We have a playbook outlining different approaches to identifying these consequences.
  • Team diversity helps a lot in simplifying the process, but it's insufficient.

Details

Level
Intermediate
Format
Talk
Technical depth
Introductory technical
For
Mid-career, Senior individual contributor, People manager
Length
16:50
Published
June 2022

Questions this talk answers

What is responsible technology?

Parsons treats it as an umbrella term covering ethics and equity, and the responsibility of creators and deployers to consider the broad consequences of their technology, intended and unintended.

Is hostile technology always malicious?

No. Beyond hackers and ransomware, much technology becomes hostile unintentionally, as when a hospital's ICU model learned nothing about asthma patients because of a hidden bias in its data.

Why is responsible tech more urgent now than in the past?

Early systems handled ledgers and payroll with few grey areas, but technology now shapes criminal justice, medical decisions, credit scoring and physical systems, so the footprint of what can go wrong is enormous.

What can teams do if they cannot build a fully diverse team?

They can use facilitation techniques that prompt them to look at problems from different perspectives, collected in Thoughtworks' free Responsible Tech Playbook.

About this talk

The reach of technology is extending into more and more sensitive and complex arenas, from credit decisions and medical diagnoses to sentencing — to name just a few. In addition, there continues to be little discussion around what boundaries should exist to protect individuals and organizations, although legislation and regulation is increasing coming into play. While technologists don’t have an analogue to the Hippocratic oath, we still need to consider what responsibilities we have to those who use the technology that we build, as well as those who are invisible during the design process. We will discuss approaches to help technologies examine their product and technology choices to be more inclusive, aware of bias, transparent and to mitigate negative inadvertent consequences.

Topics

Related talks

Browse the library