0:0018:17

Hybrid Decision making: Black box ML ft. explainable AI

Laisha Wadhwa 18:17 2022

An image model told wolves from dogs by looking only at the background

An image classifier turns out to tell wolves from dogs by the background alone, and a digit model is 50% sure an eight is a three. Laisha Wadhwa, a software engineer on the Goldman Sachs data team, argues that when models decide loans or read cancer scans, accuracy without explanation is a risk. So how can anyone, expert or not, see what a black-box model actually learned?

See how LIME explains a single prediction in terms a non-expert can follow

Watch on YouTube

In short

Laisha Wadhwa, a software engineer on the Goldman Sachs data team, argues that a model's accuracy says nothing about whether it learned what you intended, and that explainable AI should sit alongside every ML pipeline. She explains how the LIME library builds simple local explanations for any model, and uses image, digit and text examples to show how these explanations expose bias, guide debugging and help non-experts trust a model.

“But what good is a churn prediction model if it cannot tell you why so many people are leaving?”
“So essentially, how my model is differentiating between wolves and dogs is only the background.”

What you will learn

  • Explain why a high-accuracy model can still be learning the wrong thing
  • Describe how LIME fits a simple local model to explain one prediction of any black-box model
  • Use explanations to decide whether to fix training data or tune hyperparameters
  • Present model decisions to business stakeholders without walking through the mathematics
  • Weigh LIME's limitations and recognise when SHAP suits tabular data better

Key takeaways from the speaker

  • Get a chance to learn a hybrid way of decision making, while building ethical models on the way!
  • At the end of the talk you’ll know how to explain your model to possible anyone! This makes it easier for you to sell your Business idea.
  • You’ll be equipped in creating a model agnostic easy to understand explainations for possible any ML model
  • There will be hands on for multiple datasets to build a profound understanding of the concepts discussed and equip you to start building explainable AI!

Details

Level
Beginner
Format
Talk
Technical depth
Introductory technical
For
Early career, Mid-career
Length
18:17
Published
July 2022

Questions this talk answers

Why isn't a high-performing model enough?

A model that works well is not necessarily working as expected. In critical cases such as declined loans or cancer scans, you need to know why it made a decision.

What is LIME and how does it work?

LIME stands for Local Interpretable Model-agnostic Explanations. It learns a simple model on top of your existing model by sampling around one instance and minimizing a locality-aware loss that measures unfaithfulness.

How do explanations help debug a model?

They show which pixels or words drove a prediction. When an eight was read as a three, the explanation suggested augmenting data rather than spending time on hyperparameter optimization.

What are LIME's current limitations?

You need to know the complexity of your model beforehand, and sampling from a Gaussian distribution introduces bias that you tune by choosing how many samples the local model uses.

When should you consider SHAP instead of LIME?

The speaker says SHAP is better suited to tabular data, where explanations are more involved.

About this talk

With wide no. of libraries & frameworks available for building ML models ML has become a black-box these days Thus model interpretability is vital. But its hard to define a model’s decision boundary in simple terms. With LIME(an open source library) its easy to produce faithful explanations and decrypt any ML model.

In today’s business centric world, there’s a renewed focus on model interpretability. With ML finding multiple use cases for elevating businesses, it has become vital to Interpret the model, build trust in the model (because there’s money at stake!) and understand how it works for any given data. The most common all ML enthusiasts have is: why was this prediction made or which variables caused the prediction?

Model interpretability tricks like CV and grid search only answer the question from the perspective of the entire data set. It is hard to diagnose specific model predictions.

How many times have we all been stuck when our model performs well for some labels while fails for spurious data. We are left thinking what went wrong during training? That’s where the magical libraray LIME comes into the picture. LIME is a python library which tries to solve for model interpretability by producing locally faithful explanations thus explaining the decision boundaries of our model in human understandable form. If you are a novice or an expert in ML, LIME is just the tool for everyone. It explains a prediction so that even the nonexperts could compare and improve on an untrustworthy model through feature engineering. What will you learn? - The talk will primarily highlight the necessity of understanding the outputs of ML models to build a profound understanding of how ML works. - You'll learn a new way of hybrid decision-making. - At the end of the talk you’ll know how to explain your model to possible anyone! This makes it easier for you to sell your business idea and build transparent and responsible models.

In order to create trust in our model, we need to explain the model not only to ML experts but also to domain experts which require a human understandable explanation. You’ll be equipped in creating a model agnostic locally faithful explanation set which helps even the non experts in understanding how the original model is making it’s decision. All this sounds exciting, to add to this, a demonstration on the classic TITANIC data-set will be given to build a profound understanding of the concepts discussed.

Topics

Related talks

Browse the library