If two human labelers disagree, don't expect your model to beat them
Give the same tire image to two human labelers and they will not always agree on whether it is damaged. That gap, Continental data scientist Filipa Castro argues, sets the ceiling for any model trained on their labels, and it is one of many ways machine learning products break the planning habits of traditional software. So how do teams set realistic goals and design around errors they cannot eliminate?
Learn to set ML expectations from your data and design apps that make model errors forgivable
Filipa Castro, a data scientist at Continental, explains why machine learning products (software 2.0) are harder to plan than traditional software, because they learn from data and behave probabilistically. Through a tire inspection case study and an activity-detection app, she shows how deep data exploration, human-label baselines and error-tolerant design help teams set realistic expectations and protect the user experience.
“So while in software 1.0, it's a deterministic behavior of the product here, it's a probabilistic behavior because it's a model that is answering or outputting something.”
“So to sum up, I would say that it's very important to dive deep into the data understanding and show what is feasible, what is not according to the data we have.”
What you will learn
Explain the difference between deterministic software 1.0 and probabilistic, data-trained software 2.0 to stakeholders.
Plan for an ML lifecycle that loops back between business understanding, data preparation, modeling and deployment.
Use data distributions to show stakeholders which cases a model can and cannot be expected to handle.
Set realistic performance targets by measuring agreement between human labelers.
Design ML features so errors are reversible and confirmed by users, feeding corrections back into the model.
Key takeaways from the speaker
Learn to identify and recognize the main challenges of building ML software, in contrast to traditional software
Learn how to deal with the uncertainty of building ML software, especially get some tips on how to manage expectations and set realitistic goals for the project
Find which type of approaches big companies can follow to manage ML projects, from consulting to prototyping and deployment of real solutions
Learn about the main differences on designing a ML product vs a traditional software product, and understand the impact in the user experience
Get introduced to data protection and model bias issues, which need to be intensively discussed and taken into account when building ML software
Details
Level
Intermediate
Format
Talk
Technical depth
Introductory technical
For
Mid-career, Senior individual contributor, People manager
Length
19:33
Published
July 2022
Questions this talk answers
Why is machine learning software harder to plan than traditional software?
Traditional software follows explicit instructions and behaves deterministically, while ML software learns from examples and produces probabilistic outputs. Its development cycle also involves many steps that often loop back to earlier ones.
How can a team set realistic expectations for an ML model?
Explore the data distribution to show which defects or cases are missing, break the broad problem into smaller priorities, and use agreement between human labelers as a performance baseline.
How should an app be designed when its model will sometimes be wrong?
Prefer designs where errors are transparent and reversible, such as detecting events after recording and asking the user to confirm them, rather than acting irreversibly in real time. User confirmations can later improve the model.
How does a corporate data science team engage with business units on AI projects?
At Continental the team acts as a pool of data experts: it can consult, explore data for potential, build an MVP prototype using methods such as Scrum, or deliver an end-to-end product into production.
How much data does an image or activity recognition model need?
Typically in the region of thousands of examples, around 1,000 to 3,000; about 1,000 is a good reference for a binary problem, and multi-class problems need more.
About this talk
Smart digital products, such as Alexa and autonomous cars, are taking over our lives. The development of such products, which are based on machine learning (ML) and artificial intelligence (AI) features, brings new challenges to the table.
In contrast to traditional software, the results of a ML model are not deterministic. Thus, managing expectations and setting clear requirements is key for the project kick-off. The expected performance of the ML models must be realistic and well defined. State of the art methods and human performance should be used as baselines.
Just as humans do, smart products might fail sometimes. It’s then essential to carefully design these applications, so that errors are transparent, well understood and tolerated by the user.
More than ever, ethics is a real concern. Topics such as data protection need to be intensively discussed and taken into account. Model bias needs to be measured and mitigated.
In this talk, we’ll go through real use cases that prove how challenging it is to build ML products. This will include tips and best practices for dealing with such challenges.
Software 1.0 vs 2.0, explained with a child (58 s)
Human disagreement caps model performance (56 s)
Design ML apps so errors are reversible (70 s)
Good morning or good afternoon. I'm not sure everyone. My name is Filipa. I work as a data scientist at Continental. I currently sit in Portugal, more specifically in Porto. And basically, I think most of you may already know Continental. It is mainly known by its production of tires, but the business is much broader than that. So specifically in the field of building smarts and new digital products, we are building a lot of expertise there.
We build products for different fields, such as agriculture, sports, of course, automotive also. So today I would like to share my view, my experience in building this kind of products. I will mainly focus on the big challenges that arise when we are building machine learning based software instead of traditional software. And I also want to come with some solutions or at least present the solutions that we are pushing forward at Continental.
So let me start by explaining what is the main difference between building traditional mobile apps or web apps that we all know in contrast to building smart products that are based on what we call artificial intelligence. So if you see here in this comic that I really like, if we have a common or normal traditional software mobile app, if you give the designer and the developers, the designer gives the developers a wireframe explaining what is the expected behavior of a certain button of a certain layer, then it's quite easy outside to understand what are the steps and to plan for it.
So, while for an AI product, this can be really different. So let's say you want to build a self-driving car, you know what the self-driving car is supposed to do, but it's difficult to plan right and to see to foresee the steps that are needed to build this kind of features. Why is that? Basically, because in traditional software, the developers use what we call the software 1.0.
This means that the developer will give explicit instructions to the computer. So if you hit this button, this is the behavior. If you take a picture, this is the behavior. While for building machine learning based products, we are in the era of software 2.0. This means that normally the developer, which can be like a data scientist like me, we won't give explicit instructions to this computer or to this mobile app.
Instead, we will give a lot of data, a lot of examples and the computer will learn with this example. To make it simpler, if you are trying to teach a kid the difference, for instance, between a dog and a cat, you will show this kid a lot of examples. You can show pictures, you can go to the street and show how a dog looks like, how a cat looks like, and then the brain of the kid starts to learn.
And for software 2.0, it's exactly the same. You give a lot of data examples, you build a model that is based on mathematics, and because of this, it's a probabilistic model. So while in software 1.0, it's a deterministic behavior of the product here, it's a probabilistic behavior because it's a model that is answering or outputting something. And how does this affect the development cycle or when you are an owner or a manager of the product, why is it so challenging to base the products on this software 2.0?
So if we look at the traditional software cycle, I would say that if you follow the good practices that have been studied for years now, if you plan well, you do a good analysis of requirements, a good design, then this cycle runs quite smoothly. So for machine learning software, it is also a cycle, but with a lot of forwards and backs.
So we start by understanding the business like we do here, but then we also need to understand the data to prepare the data to create the model. So there are a lot of additional steps, and at each one of these steps, you might need to go back to the previous ones. So regarding our experience at Continental, what we see is when we are in this region, so this gray area of understanding the business and the data, we find a lot of different scenarios within the same company.
So sometimes we have some teams, they don't have any kind of data, they don't have the knowledge, they don't know what they want to do, but they know they want to do something that is intelligent. Sometimes they already have the data, but they don't have the knowledge, so we need to help them do this. And sometimes they have the data, they know exactly what they want, but they don't have the know how yet.
So this is quite difficult because you have such different scenarios. And also when you go to each of these steps, you can also find different obstacles. So for instance, if you go to the data preparation section, sometimes you need to collect data. So you need to set up a new project, which is data collection. You need to give recommendations to give them the best practices for collecting this data.
Sometimes they have the data, but there's no quality in this data. So you need to set up maybe a data engineering project before the actual data science project. And sometimes it's really difficult to handle the data if it's like stored in a very weird unknown database, for instance. So when we move to this part of the modeling also, it will depend a lot if it's a research based project.
If it's something that needs to go live and then you have speed requirements, for instance, sometimes you need to come up with new metrics because it's not all about accuracy and reliability. It's also about efficiency of these models, how fast they can run. And when you go to the deployment, you can also find different scenarios. So sometimes they want to build a web app or mobile app.
Sometimes it's only a dashboard. Sometimes it's only an end point. So an end point is something that you can call a model and ask for a prediction. So every day I want to predict how many tires the company will sell. This is what we call an end point. And with such uncertainty, such diversity of scenarios, how are we dealing with this in our company?
Basically, the team of data science or the data scientists, we don't see ourselves as a team that is only focused on delivering AI products. We see us as a pool of data experts that want to help the company foster data science, to improve the data literacy, to give support to any team that wants to develop AI based solutions.
So sometimes we might be acting as consultants. If some team just wants to evaluate potential use cases, we can advise them to follow our standards, our processes, our good practices. So they are the owners of the project. We just go there to give some support. Sometimes we can move forward and it makes sense to move forward and explore some of the data to see the actual potential.
When there is a clear goal, a clear business problem, we want to check if AI can solve this. If yes, if there's value, we can move to a prototyping phase where we build the MVP and we can start using more resources, maybe some methodologies such as Scrum. And if finally there's like the interest to invest more in this kind of solution, we can implement an end to end product and be the responsible ones to bring it into production.
So this is our main approach, I think the strategy, let's say. And now I would like to give you a concrete example of a project where we faced some challenges, how we solved them and how we dealt with this kind of problem. So let's say someone asks, of course, that we design a tire inspection system. This is something like a self-driving car for inspecting tires and we are not very sure about how to start.
So in this case, it was a real use case where the team already had images of tires. They already had the labels, so the labels mean that they already know this tire as a problem. This tire doesn't have a problem. They add some knowledge. They were using traditional methods, not deep learning or machine learning yet. And they had a very target solution already defined.
So for each image of a tire, a picture, they wanted to detect if it was in a good state or if it was damaged somehow. If there was some kind of anomaly, the metric, they also knew they wanted to reduce the false positive rate. So with these traditional methods, they were detecting a lot of false anomalies and they wanted to reduce this.
So after understanding the business, we went forward to understand the data and you will see how important this is. So the classes we already know. We have tires that are okay, tires that are not okay. So this is a binary problem. Then it's also important to dive in the data and understand that we have several tires, but for each tire, we might have several images.
So then how can we decide if for one image we have okay and for three image we have not okay. And also for each image we have, we had in this case several patches. So several regions of the image that we wanted to focus our attention and detect if there's an anomaly or not. So in terms of categories, this is still, I would say data, but also business understanding.
So for among all the images, we could have different tires, small tires, big tires. We could have different mold types. This means that the way the process, the tire was created was different. And then we can also have different types of defects. It's only not only one type of defect. There are many, as you can imagine. And also they might have different types of severities.
So why is this important? One of the first steps we did was to visualize this distribution. So let's look at the data and let's look at, for instance, how many defects of each type we have there. Why this is important? Mainly, I would say to set expectations to the stakeholder, to this team or clients. Here you can see that there are some defects that are really common.
But then you also have some defects that are not even represented in the data set. So you don't get to see any kind of example of this defect. And this is important because then you can go back to the stakeholder and explain them. Okay, we are building this model, but once you are starting to test it, if you test it in a new defect that we didn't have access before to any example, then we cannot expect the model to accurately detect these kind of defects that were not represented in the training set.
So as I said with the kid, if the kid didn't saw any examples of these defects, then it doesn't know what it is. The same applies to the article distribution. If there's some article that is not represented, then you cannot accurately detect defects on these articles. And for the severity, I think what is more important and more interesting here is you can try to convince the stakeholder to break the broad problem, which is very complex, into some problems.
So here you could say, let's focus on level one severities because they are the most common ones. And if you solve this problem, it's already a very good progress. Or maybe let's focus on the level four because they are less cases, but they are the most critical ones. So if you solve these ones, if you find these ones with a high reliability, then it's already a good, good progress.
Apart from that, if you have human labels, so in this case, we have humans labeling images and these humans are the ones who decide this is a not okay tire. This is a tire that is okay. And what we saw is for very similar images that there's no, if there's not no agreement between two labelers. So if you give the same image to two humans, they will disagree.
And this is important because if you are able to show that the labels are not always consistent, then you can set the expectations and and tell. Okay, if the humans cannot agree in a in a label for more than 80% of the times, then we shouldn't expect our model to perform more than 80%. So to sum up, I would say that it's very important to dive deep into the data understanding and show what is feasible, what is not according to the data we have.
Try to break the problems and choose the most relevant and critical ones. And if you have human labels, try to use this as a guideline for for like a baseline for performance. Lastly, and very quickly, I would like to give another small example. So let's say we want to build an app that is able to detect activities of humans and we have a video and we want, for instance, our user is jumping and we want to detect this jump.
So let's say our model detected two jumps. This is a example I want to give regarding how to design machine learning apps. So in this case, let's say this jump is a false positive. So the model detected a wrong jump. It's not a jump, but the model says it is a jump and here it detects a real jump.
So this is an error. And how can we design the app better so that this error is more transparent and doesn't give a bad user experience? We have one option, which is the streaming mode. So let's say we are recording and on the fly we want to detect the jump. If it's a jump, we crop the video.
The problem with this is if we have a false positive like this one here, then we will crop the video. The user cannot understand why the video was stopped and cropped. And these kind of mistakes from the model are not reversible. So this is a dangerous, I would say, mistake. It gives a bad user experience. But if somehow we manage to deliver a product where we do this in a post-processing mode.
So let's say the user records an entire video. And after this video is stopped, when the user wants, we will detect the jumps within this video. We ask for the confirmation of the user. So if it says, no, this is not a jump. It just clicks on delete or not confirm. And then we can kind of revert the decision of the model.
And we can even use later this feedback of the user to improve the model. So to sum up in terms of design, I would say it's really important to plan for the real scenario to choose. In this case, streaming mode, post-processing mode to make the errors transparent. So if the model fails, give it to the user and the user is required to verify it.
For instance, this way, I think you can try to design the app so that the impact of the errors, they are still there, the errors, but they don't have such an impact in the user experience. So now I would like to have some time for questions. If you have questions are also, if you deal with this kind of challenges in your daily jobs, I would like to hear about this.
If we don't have time here today, you can reach me in LinkedIn. And I would like to, of course, thanks to Human Tech Global Conference for the opportunity to be here. And thank you for your attention during this session. If there are no questions, maybe I would ask, for instance, and if people are still here, if you have some kind of background in machine learning or if you also deal with this kind of problems, or if you just came out of curiosity, if you are interested in this kind of session and topic.
If you have any questions, or if you are interested in here, if you are interested in this kind of video, or if you are interested in the chat, if you are interested in the chat, if you are interested in the chat. If you are interested in the chat, if you are interested in the chat, if you are interested in the chat.
If you are interested in the chat, if you are interested in the chat. most common number. For activity recognition, for instance, one of the projects I've been part of, it's more around 2,000 to 3,000 images, but also if you have a video and if you use all the frames from these videos, then for each video, you already have a lot of examples. But I would say it's definitely in the region of thousands, 1,000 to 3,000 normally. Of course, if it's a very complex project, for instance, the tires, if you have 100 different effects, then you need maybe 200 examples of each defect, then you need to multiply. It depends. Here we saw a binary classification problem.
If you have multi-class problems, so instead of just detecting okay or not okay, you want to detect what kind of defect is there, then you need more data. But if you want a reference, I would say 1,000 for this kind of problems is a good reference. Okay. Thanks a lot. And I wish everyone a good conference after all. See you.
We'll see you next time.
Transcribed automatically; names checked against the speaker record. Select a time to jump there.
Article based on this talk
Why machine learning products can't be planned like ordinary software
Continental data scientist Filipa Castro shows how data exploration, human-label baselines and careful design make uncertain ML products manageable.
In short
Filipa Castro, a data scientist at Continental, argues that products built on machine learning cannot be planned like ordinary web or mobile apps, because their behavior is learned from data and is probabilistic rather than deterministic. Drawing on a real tire inspection project and an activity-detection app, she shows that the answer is to let the data set expectations, use human performance as a baseline and design products so that inevitable model errors do not ruin the user experience.
Key takeaways
Learn to identify and recognize the main challenges of building ML software, in contrast to traditional software.
Learn how to deal with the uncertainty of building ML software, especially how to manage expectations and set realistic goals for the project.
Find which approaches big companies can follow to manage ML projects, from consulting to prototyping and deployment of real solutions.
Learn about the main differences between designing an ML product and a traditional software product, and understand the impact on the user experience.
Get introduced to data protection and model bias issues, which need to be discussed intensively and taken into account when building ML software.
From explicit instructions to learned behavior
Castro opens with a contrast familiar to anyone who has shipped a traditional app. When a designer hands developers a wireframe describing what a button should do, the steps are easy to understand and plan. With an AI product such as a self-driving car, everyone knows what the end result should do, but foreseeing the steps to get there is much harder.
She explains the difference using the idea of software 1.0 and software 2.0. In software 1.0, developers give the computer explicit instructions. In software 2.0, data scientists instead provide large numbers of examples and let a model learn, much as a child learns to tell a dog from a cat by seeing many of each. The consequence, she says, is fundamental: "So while in software 1.0, it's a deterministic behavior of the product here, it's a probabilistic behavior because it's a model that is answering or outputting something." (3:28)
Why the ML lifecycle keeps looping back
A well-run traditional software cycle, Castro notes, tends to flow smoothly when requirements and design are done well. The machine learning cycle is different: business understanding, data understanding, data preparation, modeling and deployment, with frequent returns to earlier steps.
At Continental she sees very different starting points even within one company. Some teams have no data and no clear goal, only a wish to build something intelligent. Others have data but lack knowledge, or have both data and a clear goal but not the know-how. Each stage can bring its own obstacles: data may need to be collected first, poor data quality may require a data engineering project before any data science, and live deployments may add speed requirements so that efficiency matters alongside accuracy. Deliverables vary too, from mobile apps and dashboards to a simple prediction endpoint.
A pool of data experts, not just a delivery team
To handle this diversity, Castro describes how her team positions itself inside the company: "We see us as a pool of data experts that want to help the company foster data science, to improve the data literacy, to give support to any team that wants to develop AI based solutions." (7:08)
In practice this means a staged engagement. The team may act as consultants while another team owns the project, then explore data to test potential when there is a clear business problem. If AI shows value, they move to a prototype or MVP, possibly using Scrum, and if the business wants to invest further, they build an end-to-end product and take responsibility for bringing it into production.
The tire inspection case: let the data set expectations
The core of the talk is a real project: an automated system for inspecting tire images. The requesting team already had labeled images, a defined target (classify each image as okay or damaged) and a clear metric: reduce the false positive rate of their traditional methods.
Castro's team began by visualizing the data. Some defect types were common, while others were not represented at all. That finding became a tool for managing stakeholders: a model cannot be expected to detect defects it has never seen, just as a child cannot recognize an animal it has never been shown. The same applied to tire articles missing from the data. Severity levels offered a way to break a broad, complex problem into smaller ones, for example focusing first on the most common severity or on the rarest but most critical one.
Human labels gave another lever. The team found that two human labelers would disagree on very similar images, which provides a natural ceiling for model performance. "So to sum up, I would say that it's very important to dive deep into the data understanding and show what is feasible, what is not according to the data we have." (13:40)
Designing apps that forgive model errors
Castro closes with a design example: an app that detects a user's jumps in a video. If the model flags a false jump in a live streaming mode and immediately crops the video, the user is left confused. "And these kind of mistakes from the model are not reversible. So this is a dangerous, I would say, mistake." (15:26)
A post-processing design changes the experience. The user records the whole video, the app then proposes detected jumps and asks for confirmation, and the user can reject a wrong one. That feedback can later improve the model. Her principle is to plan for the real scenario and make errors transparent: "For instance, this way, I think you can try to design the app so that the impact of the errors, they are still there, the errors, but they don't have such an impact in the user experience." (16:40)
In a brief answer to an audience question, she adds that projects like these typically need thousands of examples, with around 1,000 a good reference for a binary problem and more for multi-class problems.
Watch the talk
Traditional Software vs Machine Learning Software, Filipa Castro, about 20 minutes: https://www.womentech.net/video/traditional-software-vs-machine-learning-software-filipa-castro
About the speaker
Filipa Castro is a Data Scientist at Continental in Porto, Portugal, who has worked as a data scientist since the beginning of her career, starting with autonomous solutions for the subsea environment before moving to the automotive industry. She is also part of the Lead Team of Data Science for Social Good Portugal (DSSG PT).
Questions this talk answers
Why is machine learning software harder to plan than traditional software?
How can a team set realistic expectations for an ML model?
How should an app be designed when its model will sometimes be wrong?
How does a corporate data science team engage with business units on AI projects?
How much data does an image or activity recognition model need?
Drafted with AI from the talk transcript. Editor review pending.