Liji
Thomas
Manager, Generative AI at H&R Block
"Evaluation-Driven Development: Turning AI Demos into Real Products"
"Evaluation-Driven Development: Turning AI Demos into Real Products"
Don’t miss out and join visionaries, innovators, and thought leaders from all over the world at the Women in Tech Global Conference.
Vote by Sharing
Unite 100 000 Women in Tech to Drive Change with Purpose and Impact.
Do you want to see this session? Help increase the sharing count and the session visibility. Sessions with +10 votes will be available to career ticket holders.
Please note that it might take some time until your share & vote is reflected.
Session: Evaluation-Driven Development: Turning AI Demos into Real Products
If you want to move POCs into production, they have to do more than impress. They have to work.Generative AI demos can feel powerful- fast, fluent, and full of potential. But capability alone doesn’t scale. Without measurement, prototypes stall, trust erodes, and systems never make it to production. The gap between a compelling demo and a reliable product is rarely the model. It’s the absence of evaluation.
To build enterprise-grade AI, you have to measure what you build.
This session introduces the Microsoft.Extensions.AI.Evaluation libraries, designed to make evaluation a first-class part of Gen AI applications. These libraries provide a practical foundation for assessing what matters in real systems: relevance, truthfulness, coherence, completeness, and safety. They include built-in quality, NLP, and safety evaluators, with the flexibility to extend or tailor them to your domain.
And as agentic AI takes hold — systems that plan, reason, and take multi-step actions — evaluation becomes even more critical. We’ll explore how evaluation extends beyond static responses to cover agent workflows, action orchestration, and decision chains. When AI can act, understanding why it acted is as important as the outcome.
By the end, one principle should be clear:
You can’t scale AI on intuition. You scale it by measuring it.
Key Takeaways
- Why evaluation is the foundation of LLM Ops, not an afterthought
- How to useMicrosoft's evaluation libraries to measure response quality
- How to evaluate agentic AI — from workflows to reasoning steps
Bio
Liji Thomas, a Microsoft MVP in AI, is a seasoned technologist with over 15 years of experience in creating transformative digital experiences. As Gen AI Manager at H&R Block, she drives innovation by leveraging generative AI to enhance customer experiences and deliver business value.Liji is passionate about harnessing AI's transformative potential to revolutionize business and human interaction with technology. A thought leader in the field, she actively shares her insights through speaking engagements, publications, and community involvement. Committed to diversity and inclusion, she mentors aspiring technologists and advocates for the next generation of AI leaders.
