In short
Natalie Rouse argues that most organisations already agree on what ethical AI should look like, but few have a method for building it into the way models are actually scoped, trained and run. Her answer is to attach specific ethics activities to each phase of the CRISP-ML process model, keep the added overhead light, and treat the whole effort as an iterative practice rather than a one-off compliance exercise.
Key takeaways
- Extend a privacy impact assessment into an ethical impact assessment that lets low-risk projects drop out quickly.
- Define subpopulations in your data so you can measure representation and model performance for each group.
- Set performance tolerances by asking how accuracy gaps across groups translate into real world outcomes.
- Document consent, contestability and explainability decisions in a living report before deployment.
- Embed ethics KPIs and alert thresholds in your existing MLOps monitoring and retraining processes.
From principles to the development lifecycle
Rouse opens with the familiar headlines: discriminatory hiring models, facial recognition that performs poorly for some groups, racial profiling and deepfakes. These failures, she says, pushed individuals and organisations to draft ethical principles, which have largely converged on fairness, transparency, accountability, contestability, privacy and security, and human-centred values such as autonomy and consent.
The problem is what happens next. "However, agreeing and establishing these principles does not automatically bridge the gap from organizational principles to the development lifecycle." (2:01) Bridging that gap takes careful thought, she explains, so that ethics is embedded without adding prohibitive administrative overhead, and it requires representation and diversity so that models stay robust for underrepresented groups.
She is equally clear that principles must be owned, not borrowed. "Please don't just copy and paste some ethical principles from the internet." (3:36) Principles developed in line with an organisation's own values, she says, get real buy-in from the team and pull in the same direction as the rest of the business. Agreeing and recording an acceptable level of risk is another foundation, both for development and for managing models in production.
Scoping with an ethical impact assessment
Rouse frames AI work through the CRISP-ML process model, which evolved from CRISP-DM and describes three phases: understanding the business problem and data, developing and evaluating a model, and deploying and stewarding that model throughout its life. Each phase offers a natural point for ethics.
At scoping, she recommends building on a practice many organisations already have. "But going a step further and extending a PIA to be an EIA or ethical impact assessment is an important step up front." (8:28) The assessment should be structured so that low-risk projects with little human or environmental impact proceed quickly, while riskier ones get proper due diligence. Its findings can shape the design itself: the granularity of the data, the explainability required of the model, and what data the team actually has the right to use. If one approach carries more risk than another, that can decide which one goes ahead.
"The risk register that you should produce at this stage will guide the rest of the project." (9:31)
Defining subpopulations and measuring fairness
During development, Rouse treats exploratory data analysis as the key quantitative element of ethical AI, since bias comes from unbalanced datasets and underrepresentation. To measure representation, though, a team first has to define the groups it cares about, and she admits this is far from solved. "It turns out this is a hard question to answer." (10:09) Demographic fields may be available, but attributes such as ethnicity and even gender can be subjective, inadequate or only a proxy. Her team has identified some approaches for classifying image data, and she invites discussion on how to do this objectively and repeatably.
Once subpopulations are defined, teams choose performance metrics suited to the use case. In a facial recognition system, for example, the metric might be pure face detection, recognising a known person or estimating age. After training, they review performance across each subpopulation, which Rouse describes as an enhancement of the slicing and edge-case testing good practitioners already do.
Setting tolerances that reflect real outcomes
The harder judgement is deciding how much difference between groups is acceptable. Rouse poses a series of questions: if every group is above a high accuracy level, do small gaps change outcomes? What happens to decisions for a group whose accuracy drops well below the others? Is there a tipping point where a few percentage points matter a lot?
"Asking yourself these questions and being really curious about the link between model performance and real world outcomes helps you set realistic tolerances that can be used to monitor the performance of your model in production." (12:38)
Ethics after deployment
Before release, Rouse asks teams to confirm that the risks identified at scoping have been mitigated: is informed consent built in, is there a clear process for contesting outcomes, and is explainability fit for purpose? These decisions belong in a living report for downstream consumers and the team maintaining the system, along with recommendations such as the level of aggregation below which outputs become less accurate.
Production is not the finish line. "Embedding your ethics KPIs within your existing MLOps framework alongside other performance metrics is the ideal way to do this." (13:58) Tolerances become triggers for alerts or retraining, and stakeholders get visibility of how the model performs across subpopulations over time. Her closing advice is to begin now, start with a basic framework, and improve it with each iteration: "Doing something is infinitely better than doing nothing." (14:30)
Watch the talk
Operationalising AI Ethics by Natalie Rouse (about 15 minutes): https://www.womentech.net/video/operationalising-ai-ethics-natalie-rouse
About the speaker
Natalie Rouse introduces herself in the talk as general manager of a data science, analytics and strategy consulting company based in Australia and Aotearoa New Zealand. Her consultancy works across many industries and use cases, which she says positions it to help clients and partners map a practical path to ethical AI.
Questions this talk answers
- Why is adopting AI ethics principles not enough?
- Where should ethics checks sit in a machine learning project?
- How do you check whether a model is fair across groups?
- How much accuracy difference between groups is acceptable?
- What happens to ethics work once a model is in production?
Drafted with AI from the talk transcript. Editor review pending.