Hello, everyone. Hi, I'm Asma. Sa. I'm a A I product manager uh at a company called Alberta Motor Association.I'm from Canada and I'm here to speak about some product management practices uh and how they kind of like uh are relevant with A I because A I is different and it's different from traditional project product management. So I'm here to speak about how we need to do things a bit differently. When we are building A I products. I have got about 15 years of experience in driving innovation and transformation uh in different environments. Uh And it's all coming along with uh creating A I models ensemble with my product management experience, along with my design, learn thinking and le uh management capabilities. So I'm really excited to talk about some A I product management stuff. So before we do that, I know the world is undergo, I just want to give a brief introduction on technology as, as itself, the world is undergoing a huge revolution once again. And technology, particularly A I is the key driver of advancement in all the industry areas and we have a whole new way of doing things now.

Like words, phrases like digitization transformation, start ups, artificial intelligence, machine learning API S have become the common words on every content and it seems pretty overwhelming too, right? Like along with new skills that we need to kind of uh acquire like Scrum agile product management skills which we have never need heard before, have become a needed area of expertise to reimagine how we build on ideas and to solve real pro problems. Uh looking through the lens of the customer that's really very important. And it's what about it's all what the product management is about. So what exactly is product management? Um Let me share my screen. I think I'm sharing my screen. So yeah. OK. So everybody must be familiar with this diagram building the right product management. This one diagram wherein it involves skills around UX tech and business. But what does it mean to somebody who's really new to product management and has never heard about product management?

So product management is all about building the right thing for the right user at the right time. But we as P MS, who do we need to uh uh who, who are we allowing it for? So I think you have kind you are familiar with this type of diagram which is really very common on the neck when we talk about what customers really want and what has been envisioned and what people think about. So A PM is really an ally of customers who advocates for being the voice of the customer, right? Like it's really important for someone to be in the shoes of the customer have empathy for users. Uh Just because because I think business was sponsoring and funding the product is always thinking about benefit realization, Ry or how soon can we get returns? And engineers who want to build the features, want to build it in an in a build, the ones that are really easier and simple to deliver and also don't require a lot of maintenance, right? So, so in this kind of a setup, we need to think about the need of the customer, which is often overlooked. And I think this image really explains well about what a customer wanted and what was understood and delivered, right?

Like so the customer described it as something else, the business understood as the designers de designed it as really something different, understood different and the engineers built it as a totally different thing, right? Like so I think product management is, is the one that bridges the gap between the customers who are users of the product, the business and the tech, they strategize the problems into ideas and partner with internal and external stakeholders to actually bring them to life. OK. Moving on.

So A PM needs to understand their customers and their pin points, right? Like so as a product manager, so what are their pin points? A product manager needs to work with the UX team who I think they really start with when they are discovering how do they understand their customers and their pain points. So they have to partner starting off with. I, I, I'm just calling it a different hacks of a product manager that a product manager needs to wear. So they have to act like a psychologist. It's just not enough to know the customers. So P MS really care deeply about their customers and emphasize with their struggles and desires to get a full view of who they are, right? So they need to be like a detective uh learning to leave no stone unturned. So PM needs to push to discover the hidden insights so that they have a complete and accurate view of the problems uh further on. They need to do, do a deep dive like a scientist. P MS need to regularly test the hypothesis uh using a mixture of structured frameworks and creativity to make connections uh invent new ideas and solutions to customer problems.

So I think these are the critical steps while P MS need to be aware of when they are kind of like doing user research or just doing ideas around discovery. So after the completion of the discovery phase, next is to strategize talking to the business. How do you get your ideas? Um how do you influence people to kind of like build your ideas, to subscribe to your ideas? That's when I think P MS need to put the hat of a director uh to leverage creativity and talent uh to create and tell that story, right? Like so to tell that product story, uh they know when to cut great ideas to keep the experience focused and memorable, that's really important. Uh Also to keep up the momentum. P MS need to actually behave like a diplomat. They need to drive the product strategy across all the organizations, right, to uh to get all the teams involved uh to subscribe to the vision uh creating elevator pitches, uh influencing stakeholders for funding um nailing down Okrs and KPIS to show benefit realization white labeling or creating even the blue sky solution.

You just don't want to create a solution which is just fit for one business use case. You want to repurpose your solution. That's where you would your businesses would be interested, right? Uh to actually invest more when they see that scalability and the benefit realization years after years, years after years. So next is exciting phase. I call it the architect hat. Um That's when idea setting gets formalized into creative designs and blueprints. So it's the connecting the dot space where P MS need to put on the architect hat, they design and plan for an elegant solution um that will delight customers also making necessary arrangements and tradeoffs. Uh while partnering with departments to orchestrate that pipeline based on the reality of getting it built and delivered So once they have finished their strategized, strategizing phase, uh what do they do next? Right. Like you have all your things ready, you all, you have your teams uh in play. So next is I think the exciting phase where ems need to demonstrate servant leadership. You have to really rally around the team by listening uh being for their entering, keeping the momentum high by influencing and motivating um them like a coach. these are the people who need the food and water to be brought to them.

So that yeah, you, you can, they can build the solution without any having roadblocks. Uh So that's where they need to act like an er doctor, right? Like not everything goes as planned. So PM needs to quickly diagnose and steer the team uh from a very stressful situation to a positive outcome. Um Also it's time to celebrate, right? Like so like a gardener uh where PM uh needs to iterate and nurture and cultivate a delightful experience to see their product uh grow and scale beyond MVP year after year by monitoring and paying really close attention to the product metrics, uh the market changes. So I think, yeah, different hats. I know it's overwhelming because yeah, it's different hats like a product manager needs to wear when they are kind of like moving around building things. But what is actually a product manager do? As I said, they now that we know that uh the different roles of a product manager. We need to see how they need to leverage all these skills, tie them together and perform. So what are the key responsibilities? I know they need to start, we, we've learned they need to develop uh the product, they need to ship it. But how do they do it within the, within the realms of their job description or role? So you need to engage with customers on a regular basis.

There are multiple channels uh to understand what the problem they might be. Maybe like doing conductive uh deductive research, uh understand what the pro the problem that uh the product may solve. So that's very important uh specifying market uh and prioritizing market and product requirements.

Uh Feature sets along with key positioning and messaging of elements, uh collaborate with designers and engineers to solve problems, uh curate, communicate and manage long term product relationship or the road map, uh analyze external and internal key holders. They might be sales people or legal or compliance or design people that you might have to partner and work on on a regular basis. Conduct re relevant research and discovery on an ongoing basis. Your discovery doesn't, doesn't always need to happen at the starting of the product.

You need to have that discovery session going on and on to identify where your but what is that aha moment that you need to create for your customers uh support the teams in uh communication. Uh making sure all the teams uh are shipped to align your product so that it reaches the direct set of users, right? Like measuring metrics, drawing insights, uh creating those feedback loops to help the product grow and mature, right? Like and also don't forget to or uh to define your objective and KPIS and set targets so that your team can actually move towards them for success without having these. It's very difficult to actually measure what your success looks like. So it, it might be very ambiguous and cloudy when you have not said them. So make sure you do that. Uh So now that we have understand, I'll give you a moment to see if anyone has any questions. OK. So moving on. OK. So these are the product life cycle stages and I've measured them against time and market uh market research acquisition adoption um and retention. So you start with your discovery ideation and planning. That's where you strategize your, who are your users of your product? How, what type of research you need to do?

Should it be an inductive research wherein you are creating focus groups, groups going out, um interviewing the customers, asking them putting themselves in the shoe, in your shoes, looking everyone from a customer per customer lens or do you do a deductive research if you your co company or if you don't have resources to do that kind of uh uh UX research?

How do you come up with a hypothesis, uh create a lightweight prototype, test it, decide what your product is going to be called the problem it solves the business value. So you have to figure out all those steps before you actually decide on uh what, what, what your product would be or what your MVP would be. So planning also includes zooming out having that long term vision for your product. So just ask yourself those questions, how will my product look like one year from now? And also zoom in uh to see that short term vision, which is 3 to 6 months away. So what should be our first release, our MVP or a minimum viable product as it is called? And now we have another term called as minimum marketable product, uh which is like a set of features that our product should contain um without which the users will not be able to use your product. So we don't want to build and keep on building until we have defined all the features.

But we want to build something that is really quick and which can provide value to the users and the users can start using your product right after so that you can get that feedback and iterate on it. Um Then you need to figure out what your road map should look like. What are your features that you need to build? And when do you need to build them? How do you want to prioritize and distribute them over your road map, right? Like, so what should be your milestones we need to target and then uh how do we measure success? How do we kind of like, yeah, move on. Uh what should be our de delivery model? Uh Who should be our team's resources? Uh What kind of expertise we need? Um Which type, which type of framework should we be following? Would it be the agile? Uh the Scrum Kanban? Do we need to do api planning and plan for every 10 weeks? How do you go about working around that? Uh We need to identify when and how we need to work and what framework we, we would be following so that we can have this alignment with um our ex and teams are working on the business goal. Um So yeah, next comes uh the building and the testing phase. Yeah, a really critical phase.

Uh uh because as product managers, you'll be really very busy creating your product backlog, your sprint backlogs, uh attending sprint ceremonies, uh demos with business, uh getting that feedback, incorporating it, reprioritizing your backlog, it rating until your MVP is really ready and um good for release.

And that and next would be your uh introduce and test phase. So once you are, you're ready, like, you know, work with your peers to kind of like, you know, position your product into the market, uh you have to identify on who your buyer persona would be uh plan your launches, uh who would be really willing to pay for your product. So identify your users first and then make them your users for your beta release. Um Once your product is live, you need to have that feedback loop uh through various channels like surveys uh calls or like, you know, kind of like uh uh uh any kind of like feedback loops which can measure the success of your product. And then also you need to actually plan for, where would you be seeing this? You can't keep on listening to phone calls or listening to each uh individual surveys and uh saying like, oh what happened? Why did my users drop off or what stopped them from reaching that aha moment. You need to tie them all into a dashboard.

So it's very critical to work ahead with your data analytics team uh to make sure that you have, you have plans for a dashboard or you have a dashboard in your MVP release, which would help you uh it rate and grow your product. Um So once you have done all that, I think the next one is you have to start thinking as a product manager is how do I improve on my MVP? How do I scale it? How do I get more benefits or how do I get the RO I most ro I out of my product now, is that phase that you have delivered your MVP? How do you scale and glow grow? So, scalability should be the top goal. When you start envisioning your idea, you don't want your problem to be limited or pro your product. Sorry to be limited to a subset of your users. You need to see how you can reach horizons uh on trying various different models like, you know, in A B to C or A B to B environment uh wherein you can partner with either internal businesses within your or or external business, businesses. And to see how you can grow your product, you don't need to re invent the wheel again and again, you have to focus on how we can repurpose or reuse our work and drive ro I.

Uh so once you've done all that, your product has reach uh reached the maturity level, um you have gained your ro I. So now it's the time to think about next steps. So because technology trend and frameworks, uh existing ones uh start becoming a legacy over time, they start losing their shine and wearing out. Uh Now you really have to stay uh start to think about how do I kill my 1st 1st version because you already need to start planning on your second one. So that's the time when you need to see how do you decommission your first version because maintenance of legacy systems becomes harder and we lose out on support. Uh Because even bureaucracy of new tech kicks in right. Uh we are forced to migrate to new platforms if we need to stay in competition, uh sustenance uh is expensive, investment is easier. So plan on your next generation of product and uh retire the current model. So I know it's a hard decisions. But I think, yeah, that's very so understanding the product life cycle and then planning in tandem to get your road map, it's very critical because you need to have that clear picture of how your product life cycle would be year after year.

But also keep a a sharp focus when you're zooming in and seeing what should be the next set of features that would help my product grow from uh the version one to a version two or to get to that aha moment. So that's really very a critical skill. A product manager needs to have any questions so far for me. Cool. OK. Moving on. Oh sorry. Yeah, I lost my screen. OK. It's going to the next screen. So, so now that we know what product management looks like uh from a traditional perspective, what does it look like from um an A I perspect perspective, uh product management, we need to can product management be a good framework to build A I products with a challenge that they are not heavily uh UX focused.

What specific approach and skills to product managers need to get their A I ideas to work because everyone with a tech mandate is feeling pressured to use A I. So this diagram actually creates that distinguish uh distinction between what a product, regular product manager does and what A A I product manager does. So of A I product manager. And since A I products are heavily data driven, um they need to be really be willing to get their hands dirty in data and they need a lot of A I fluency. Um You know, if you know, you know, a use case familiarity A I doesn't work for every and any, any use case. So you have to understand and assess that capability of product capability against which uh use case. Um Your, yeah, your pro your model would work against uh buy in an align an alignment. It's really very critical for A I product managers to work with cross functional teams and that comes with handling a lot of challenges. Uh So A I product managers need to be really skilled at that. That's OK. So A I product management. So we are told that like A I can change the world um unlock hidden business value, solve problems that we are not able to do before.

But how do we kind of like, how do we do that? Right. Like so, but how do you even start, what sort of considerations you need to make? How do you even go about building your first A I product? So who should first figure out how do we approach my A I product uh because most A I products are built for uh efficient uh optimized uh operational efficiency or reduction in cycle times, most of who and figure out who your customers should be. Would there be my internal teams within the organization? Um or would there be who would be integrating and using my, within my A I product? Right? Like so would it be my external users? For example, I created a, a uh risk engine which would, because I was working in a bank wherein I had to partner with different teams. Uh So that because the product would be not necessarily used by external customers, but it was made for relationship managers who would be working with customer uh external customers uh to manage their loans to manage their financial portfolio. So my customers were uh internal relationship managers who I was building the product for.

So it's very necessary for you to understand who your product uh owner of users would be um who would be your core team because you would mostly be working with data scientists, analytics, data warehouse people. Uh Because that's where your data comes from. The integrations team, like the API S or the CD KS are the ones who build your API S or the CD KS and SDKS, right? Like so until you have your micro services model uh with their architecture team, it's going to be really hard to scale your A I products and mostly um A I product because there is no lot of visual representation like mocks and prototypes. Uh It's really hard to speak to the representation of idea. Uh The identification of problem can be a big challenge as A IP MS are not directly speaking to customers. So they don't know what the customer wants. It all basically depends upon the users of your model. Um So, right, like, so as an A I PM, we always, we feel left out uh because we are always working behind the scenes, there's no motivation or influence because there's no spotlight, nobody recognizes your product, right? Like, so that's a challenge. So how do you feel motivated and keep up with the momentum? And most of all, I think it's really important um to be uh because we are called technical uh because we need to communicate and understand the technical jins to be the best fit with your technical team.

We don't want to be living with the FOMO effect. Uh That doesn't mean that uh A I product managers needs to be technical and it's not, we need to learn rocket science, but not everyone is good at coding. So PM and also P MS don't have just have one role, right? Like they need to manage everything from inception to launch. So definitely there's some extended skills that A I product managers need to have apart from the regular ones. So let's find out, find out what they are. OK. Mhm OK. So moving on uh so the skills of an A I PM business case evaluation you are fit for A I. So start with your problem statements in this phase, you decide what to do and why. So right, like smart product managers combine data uh along with analysis and judgment, ask the right questions, right? Like so ask those five by like why do you need a I? Why do we need to build this product? Why do we need a I? Why do we need automation? What's the problem that we are trying to solve? Right? Like having enough depth to filter out the noise and focus on the worthy po CS. That's very important because they are not like they are not just science experiments uh executives wi I or have really a lot of wishful thinking and they have no data or scaling possibilities, right?

Like so A IP MS need to ensure that A IP O CS must match the organization's reality. Say no, when uh machine learning is not a good candidate, uh make a checklist of things that you would need. Absolutely. Uh for your A I model to be a success, decide how much machine learning is to be used with. Along with other approaches, you might need to, you know, partner, you might, you might need to uh combine it with the rules automation uh right? Like so to enhance that success or to amplify that success. It's just not, not A I will not alone give you that um key results that you're looking for, right? Like so priori first of all, to need to prioritize and manage the pipeline, we need to have all these things in alignment. And then also um yeah, um figure out what your resources and development uh looks like uh organization structure really matters, understand and coordinate organization structure, right?

Like so uh you, you need to have data literacy, domain depth, uh geek creed. All those are important things. Uh We need to partner with the right SME S at different stages. Uh And then uh how do we engage or engage support teams so that we are not roadblocked while we are sprinting. So A I product manager is the glue that keeps all of those A I elements together. Uh speed and frequency of improvement matters. Be agile smart Po CS are more important than ever. Start with your PO CS, create a series of mvps uh like and which are actually lightweight models and also set your priorities and road mapping. Yeah, and do a value stream. Uh What are the features that we need to build as a part of our MVP? And how long it's going to take? How do we plan, how do we prioritize the business, wants everything to be delivered or every set of features to be included in the MVP? No, have a, have a conversation with them. Set up your backlog, um your delivery schedule, your risk that you may encounter your mitigation plan and include trust and empathy, right from the beginning. That's really important. So, understanding also your ro I right, like so do a value stream mapping. So understand what your current process looks like, how long it takes for um for a stakeholder to complete one to get from one state to another. Uh Where are the bottlenecks?

How much of it is the lead time and cycle time measure all of that and then come up with uh uh with uh this is a current process and this is uh like, you know, two week process. How will the enhanced process look like with incubating it with machine learning models? Are we able to remove bottlenecks? Will it reduce the cycle time or the lead time? Uh Do we need automation to drive efficiencies and don't reflect to forget on a few other things is the result that we are aiming for really obtainable. That's one thing that you need to ask and also what have been the reasonable results or the rois of other companies who have implemented uh the same applications of these kinds. So you need a baseline to measure your success against sometimes applications. Uh Sometimes an application or a model is really first of its kind and there won't be any precedent to it. Uh So in most cases, there, there will be some analog A I application uh that will give you some reasonable ro I so you have to just, you know, measure it against that any questions so far. OK. So I guess I just keep moving on.

So another thing is like you have an A I, apart from your A I strategy is a big thing is the platform you would be using, whether it would be a hybrid setup, a cloud setup. And then also what kind of data do you have? So product managers need to have um to be able to understand the fundamental and terminology and concepts of artificial intelligence. Uh It doesn't mean that they need to learn to code nor does it does it mean that they need to talk sharp uh with the IT department? But they, and they also don't need to be data scientists. Uh but they have to be managers and leaders at a conceptual level. At least they need to know how a machine learning algorithm is trained, uh how feature engineering works. So if the team asks you, uh which model features need to be of high priority, for example, than the others among the anomalous set of ones. So you might have a model that is kind of consisting of about thousands of features. So if your team ask, like, how do you prioritize, right? Like, so how does our model make decisions and what would be the priority of your features? You need to be able to uh be in a spot to understand those questions. Um Also what kind of A I approaches and techniques are out there? There are some predictive analytics, machine vision, natural language processing, computer vision image processing.

So which one actually which of these techniques and approaches can be applied uh to your use case because there are tons of uh ones that are available in the industry. Also, which type of algorithm works best for structured data versus unstructured data. Uh For example, is the B model fit for my use case or would it be a regression one? Uh You need to make those tradeoffs, right? Like, so that's where a product manager skills need to grow, especially for A A I product and also the life cycle and the phases of the A I product. So it's understanding uh the the business understanding, the data, understanding and assessing the product needs a conceptual understanding of the life cycle of the product of the uh of the A I. And data science is one of the skill that I think most of the uh product managers are missing uh because they go into A I projects just like as if it's a regular one and they fail to prepare for the constant loop of iteration. So if they do understand this, it's more likely that the C level executives um the report don't, they do not. So aligning leadership with reasonable expectations, it's very uh for an A I project is a separate skill um that needs to be nurtured.

So also product managers often have to make sure that they develop or they can perform that incubation period in a way that also doesn't, doesn't disrupt the user experience in a way that it is detrimental. So they need to understand the challenges that come along with adopting artificial intelligence in an in an enterprise. We have already spoken about RO I also, it can take months to access and clean the necessary data to begin an A I product project. Uh Really, it's not about the model, right like that where we are shipping features out every sprint as the business is used to seeing in some cases, uh the business structure needs to be overall including data in infrastructure and and some uh protocols. All of these are subjected to change in the middle of the project. So in any of these cha challenging situations, which are big surprise for A I product manager, they need to be very good at communicating realistic expectations with the leadership team and they are also going to manage various different themes of the project. So setting expectations with stakeholders on deliverables, timelines, model accuracy and Roy and maintaining a regular clearance of transparent communication right from start. Uh That's really key kick. Uh So next is commercialization. So how do you product is your product?

Uh how do you have the right people, the processes and tools, how will you continuously mo monitor and improve the performance of your product. So involve your subject matter experts right from the beginning during the training process. If you're doing supervised learning, have your subject matter experts who are the domain experts to label data for you. Uh Because that will keep them motivated and involved, right? Like so getting business sign up early on the model accuracy and prediction. If your subject matter experts might help with that kind of a thing too. I've seen A I uh uh artificial intelligence P MS struggle a lot with businesses to reach a target accuracy and precision state. It's hard, we need to negotiate on a sweet spot. If you don't want to spend the entire project, just attaining accuracy and end users. Uh How will your A I team reach? Uh how will your A I product reach the end users? So most of the times you need to pair up with another PM who needs to house your model inside their application. So what would be the format you would be delivering the model as would it be J on file or would you push results directly into a dashboard? Uh work, work on those logistics early enough to give you enough time for the end user uh and the PM to work in tandem to develop those tools that is required to launch your product.

Also don't under underestimate the importance of shared vision, understanding, effective communication to do that. A initiative works best when multiple teams orchestrate work and it also reduces friction, develop your understanding and work through different subcultures uh bringing different roles together to learn about each other.

Uh making data, technical technical folks serve as evangelists, they provide raw access. Uh Next is I think, yeah, model inputs and outputs. There's no way around this have your training data as close as to be in production data as possible. You will need a large amount of data, historical data for creating training and testing data sets with various compositions. You might need to have a data set which has 70% of training data and 30% of testing data. Whereas you might need another data set wherein you have to change it around, you have to test it like because you have to your, your, your printing would be comparing models to models against for accuracy. How would my 7030 model perform with a 6040 or a 5050 or 8020? That's exactly what you need to be testing your hypothesis against. You need a lot of experiment uh until you get that accuracy that would fit your ro I for example, uh computer vision use case like the camera used to capture the image, its position, uh movement lightning, etcetera. You need to match that clearly to uh the in production environment.

So it might be tempting to just use publicly available data sets for your use case uh to kick start your model training, but model trained on general data from the web uh that are built uh uh is performs really different on the data that is built for the purposes. And it will definitely fail in production if you're building it from uh data from the web. So transfer learning or generating uh generating synthetic data might look at an interesting option when you are doing PO CS. But when you have to put something into production, yeah, you just, you really have to be realistic uh to generate uh enough realistic results for your model. And same goes with the output of the model. If you're using supervised learning technique, you need annotated data for your uh mo model outputs, right? Like so in order to train your model, uh labeling data is really expensive and while multiple companies offer services for annotation, um you know, you need to pick one that has got a really strong quality checks in place. Uh Because you need to be aware of uh data privacy security and other concerns, right? Like about people misusing your data, which is proprietary data.

Uh label data often is more valuable for the model itself because any competitor can get access to your label data and can build a similar product or an offering. So make sure that your data cannot provide and can provide the same user benefits. So that's how that uh uh an insight which I have often uh got working with different industries in these teams is that product teams have uh successfully developed A I solutions with an in-house data intern who can actually uh label the data for them and manage the annotation tools or platforms.

So data is the fuel that actually fuels that your model learns from. And there's no good substitution for using data for training that has been captured in the same exact conditions as the production environment. So on the model output getting ambiguous high quality data is more important than spending your time on actually improving the model itself. Again, explainability. Yeah, trust plays a crucial role in customer satisfaction and retention and also overall business success explainable A I gives customers insights into A I decision making. So um A I product managers are really responsible for including explain availability reasons in the model predictions and ensuring trust in the model. People need to know why they were rejected for a loan or why a model made a certain decision. Uh They need explanation around that and we might, we and we as A I product managers need to work around that. We need to respect their reasons for it and have those incorporated in the models. Um We need to build models uh more on explain availability and it should not be uh kind of like a black box which most A I models are um scalability.

Yeah, it uh P MS must evangelize that the product project just more than the business must evangelize the project more just than the business case. Uh They have to understand and promote the merits of A I for adoption and help the company stay competitive in the new A I world and ethics. A big part. And it's often discussed uh whenever we are talking A I because A IP MS need to be ethical um of what uh uh need to have an ethical consideration and a deep concern when they are building A I or machine learning tools. Uh It's up to the A I PM uh to often remember and remind others that with great power comes great responsibility, especially when dealing with data with regulations and privacy issues around it. Make sure you have required the uh necessary approvals and consensus before you actually dive in.

I was in a spot at one time when I was building uh the same risk engine for uh my customers when I was not allowed by, I was stopped by the legal team for using a certain uh data category, right? Like so I was not uh it was financial data of a member and it was not supposed to be used uh apart from whoever like who, who apart from um just, you know, using it to make some business rules decisions. And we were not supposed to use that in our A I model. So our product got scrapped right in the middle. Uh So that's very key to understand your ethics and your privacy and legal constraints around it. So do your homework beforehand and also maintenance and ownership A I products constantly need to be tuned and maintained to the right accuracy. So pay attention early on to identify your model, ownership and work with your stakeholders towards an engagement business model uh to sustain and maintain A I products with the right resources on your team. So either it has to be owned by the data scientists or the A I team or if you are sourcing it off to your or if you are passing the ownership off to your business, make sure that the business has the right capability and resources on their team to actually do tweaks and maintain to the model on an ongoing basis so that they don't disrupt the the ownership doesn't fall on you any questions so far?

Cool. OK. Moving on. So building A I products is often referred to as a vous A I cycle um because there is no ship features out of the door at regular intervals. And it's hard to explain to the stakeholders why A I products need so much of time to be built and implemented. So these are kind of like the mandatory steps, right? Like that um must happen when building A I products. So it starts with the data collection. So preparing customer data from whatever sources that you are using can be really a daunting task uh due to sheer number of display data sources and data silos that exist in organizations uh to build an accurate model. It's really critical to do the right selection of data uh that it's likely to be predictive of the target that you are chasing uh the outcome which you hope the model will predict based on the other data inputs, right, like once you have collected your data, the next step is normalizing your data.

So the next step is where the data analysts and the data scientists typically spend most of their time on analy analyzing the projects, cleaning the and normalizing the dirty data because the data is not stored in clean formats and structures and tables, right? Like you need to kind of like structure it add a quality to it, make it like usable by kind of removing noise, removing the things that we don't need. This is often often requires data scientists to make decisions on data that they may not understand. Like what do they do with the missing data? Sometimes the data is missing uh incomplete data, there are some outliers. So how do we kind of like do we need to include all of that in our data sets? It's not a good idea, right? Like so they that's not good ethics. So uh we spend a lot of time with the data scientists actually doing data normalization. Uh It the data may not be easily correlated uh to proper unit of analysis right, like so in order, uh so in order to predict if a single customer will churn, for example, silo data from these spread sources can't be relied on. So a data scientist will prepare and aggregate the data from all the data sources into a format wherein the machine can interpret it. So this will end up being a very lengthy process and will require a lot of work uh before any machine learning project can even start.

So next is uh data modeling. So the next phase of the machine learning project is to model the data that will be used for prediction. Uh So part of the modeling data for prediction about customers is to combine those disparate data sets uh to paint a proper picture of a single customer. So this includes blending aggregating of silos of data like from web or mobile app or offline data and then putting it together so that they can come up with those data sets like you know, that's called data modeling. So once they have done that they have the training and testing data set, uh they start with the model training and the feature engineering. So after a brand has been deployed and collected and enrichment of a meaningful input data, it's time to put the predictive power to it, right? Like so um to test it. So what do data scientists do at this is like take representative samples of population like for example, um all customers who are anonymous visitors or who have known prospects or you know, set aside portions of data for training.

And the remainder is used for validating of the models after the training is completed. So the key component of this is to uh iterate rapidly and to continuously test new data points that can be delivered from various other data sources. So this is called as feature engineering and also prioritizing which data points need to have a higher allocate higher priority than the based on which the model makes predictions. The last is like deploying models to productions. All of all work to at this point culminates into the final step of deploying a model to production wherein the ability to predict outcomes in the real world is tested. So by this point models should be a able to meet some kind of like threshold or accuracy, right, like that warranties them to be deployed to production. So for this reason, it's really important to interpret model performance with stakeholders to agree on what is the level of risk um that is acceptable for inaccuracy. So some customers behaviors may not be sufficiently predictable enough. And thus, a model may never be able to achieve that accu accuracy to justify deploying it to production.

So in the end, machine learning is just isn't going to replace a digital marketing strategy, but we just rather augment it and enable it, right, like so successful brands will put their customers at the center and then of what they want to do with machine learning. And one of the, one of them is machine learning uh to optimize decision making as a part of that larger initiative. Cool. No questions. OK. So before I conclude, I just wanted to give some tips for A I product managers. So A I and machine learning are here to stay and will continue to transform the way we interact and engage with each other and the world at a larger at large. So you need to figure out where your A I and machine learning can give your business a competitive edge. So they need to be A IP MS need to approach A I initiatives differently by exploring data first to identify and we we business opportunities get a solid handle on your data and modeling technologies. Is your organization really having that appetite to change and transform data, create data lakes data sources.

So you first you digitize your data from um unstructured data sources, like get get on all of the data into one place before you start building your A I models and also get your hands dirty in data, get some data and uh analysis skills like different tools. There's Power B I uh Google Analytics which can help actually to kind of like make uh make those decisions, help you understand, create dash dashboards on the site to see what kind of data is available. Um uh At your uh disposal also communication. Uh A IP MS must understand how to answer uh ask the right questions to the customer, right? Like so and also they must continue to gather that data to fine tune it for ongoing A I executives like effective cross cross collaboration with cross functional teams is what A I brings in A I brings in A I initiatives to life, listen to each other, learn from each other. Um They also need to have that empathy for the customer, right? Like so they might they must leverage A I and machine learning to deepen their understanding of the customer. Have empathy. Look at privacy with empathy, explore new solutions such as differential privacy. What will it take for the users to adopt my solution? Right? Like so use a human uh user centric product centric framework for error measurement and also challenge, facilitate and deliver, right? Like so challenge your business is this uh the best use case for uh A I, right?

Like so do we how can we uh set on your objectives and Kpis earlier uh just do your homework research back and see how many people in the past or how many organizations in the past have benefited from this type of use case or from this type of model? Do they have metrics? What was the implementation strategy? What was their benchmarking? Uh challenge your business to get all those answers to you before you start building your uh A I models and then increase the trust and adoption, right? Like so it's really very critical to have people trust in A I and adopt A I because without trust there wouldn't be a lot of adoption. So I've come across in my experiences, people wherein um in my organization, my team members would not prefer to uh go by the decisions the A I models make and use their own gut sense and experience to make those decisions. So we are not, we are not like using that adoption, we are not encouraging that adoption if we are keeping A I, uh if you are not explaining AAA I to them, right? Like so if you don't have the reasons why A I took this decision or why the model failed to kind of like adjudicate a customer or not make a decision on the customer.

If you're not able to explain that, then our A I adoption will always be slow and then evangelize carefully. Like, you know, you are also look at things that you would like to scale for, right? Like so you have built this product for this A I but then focus on the architecture, focus on the integration, focus on the API how would this be able to, how can I convert it into microservices? So how would other other areas of my business benefit from my model? Is there a way wherein I can scale create that virality create that network effect that 10 next, right? Like the 10 ex experience. So yeah, you have to Evan evangelize it carefully because it takes a lot of time and effort and money to uh create A I models and you just don't want them to be sitting on their shelf. And last of all, perhaps most importantly, uh prepare for failure. Uncertainty is very high in A I initiatives and above all, keep learning and use that knowledge to make self improvements. So, yeah, uh that brings us to the end of uh my talk. Uh So connect me on linkedin. I'm available on linkedin. Uh And then also you can shoot me an email. Um Have any questions regarding A I or product management. Yeah. Feel free to reach out. Uh I'll be most willing to help you. Thanks for taking the time to listen to me. I hope it was beneficial.

And then, yeah, uh keep in touch and happy Wednesday.