0:0020:38

Are We Building Products that are Ruining the World?

Carey Jenkins 20:38 2021

A startup's $1 million prescribing alert fired 230 million times

A free health records platform let a pharmaceutical company help write the unbiased-looking alerts doctors saw while prescribing pain drugs, and the product did exactly what it was built to do. Carey Jenkins, CEO of the research, innovation and build studio Substantial, argues this was no unintended consequence but the result of how tech teams are trained and rewarded. So what would it take for the people building products, including you, to choose differently?

Learn five practices that help teams build products without trading users' wellbeing for revenue

Watch on YouTube

In short

Carey Jenkins, CEO of Substantial, argues that the tech industry is building products that harm the world because it trains people to chase growth and revenue at any cost. Through a case study of a health records startup that let a pharmaceutical company shape prescribing alerts, she shows how ethical lapses happen in small steps, and offers practices for more ethical innovation: change the operating system, lead from why, reduce blind spots, think in systems and create with foresight.

“Technology is capable of making the world a better place. But the technologists have to choose that.”
“This wasn't an unintended consequence. This was very intended and very effective.”

What you will learn

  • Recognize how growth targets, equity-heavy pay and revenue-only bonuses push teams toward ethical shortcuts.
  • Distinguish legal compliance, a minimum bar for risk mitigation, from ethics as a daily practice.
  • Ask who will be most affected by a product, not only who pays for it or uses it.
  • Design research free of a customer agenda and bring in independent experts to reduce blind spots.
  • Map the system a product lives in, identify its key actor and anticipate how it will evolve.

Key takeaways from the speaker

  • The enormous power technology wields requires us to hold ourselves accountable for the role technologists play every day in blurring ethical boundaries.
  • Ethical practices cannot be siloed into a process or team - it must be embedded and empowered throughout your culture.
  • The path to revenue does not have to be paved with ethical trade-offs - how we build is as important as what we build.

Details

Level
All levels
Format
Talk
Technical depth
Non-technical
For
All career stages
Length
20:38
Published
June 2021

Questions this talk answers

Are tech companies building products that are ruining the world?

Jenkins says yes, because the industry has trained creators to value growth and engagement at any cost, but she insists they do not have to.

Why isn't a strong design process enough to make a product ethical?

Design is a skill that can serve many purposes and usually reports in to the business, so ethics must be embedded in every department and every step of creation.

Is regulatory compliance the same as acting ethically?

No. Compliance is a minimum bar mostly about risk mitigation, while ethics is about how you contribute, influence and lead, and it takes practice and courage.

What is the most important question to ask about a product?

Who will be the most affected by it, not who will pay for it or necessarily who will use it.

How can a company do thorough research and still build an unethical product?

When research is driven by the customer's point of view or an existing agenda, teams miss important information while conducting or synthesizing it.

About this talk

Innovators and technologists are in an extraordinary position to create experiences that impact how society interacts with nearly every facet of our world. In this talk, Carey Jenkins - CEO of Substantial - shares her perspective on how the intersection of technology and ethics leads to the development of more compelling products and more conscious innovation cultures. She unpacks how technologists have not just a responsibility, but also a mission to create more intentionally with creativity, diversity, and deeper purpose. Intertwining insights from over 20 years of professional experience, Carey overviews her five key frameworks for ensuring a more ethical approach to innovation: Lead from Why, Change the OS, Reduce Blind Spots, Think in Systems, and Create with Foresight.

Topics

Related talks

Browse the library

The Purposeful Innovator

Carnellia Ajasin CEO, Mind Katalyst

A tobacco client and her aunt's cancer changed which products she would build

Beginner Society, sustainability and impact 2021