Okay. So good evening. Good morning. I'll be talking about how did we move a chaotic IT landscape to a more controlled, intelligent, IT operations landscape through AIOps.So I'm Rabia here. I have about two decades of experience in performance and reliability engineering, and I currently lead Haysari practice currently working for Hexaware. I have shared my, LinkedIn QR code at the end of the deck. I'll be happy to, connect with you, and I will also share my LinkedIn URL in the chat window. Feel free to connect with me to take up any questions offline if I couldn't take up your questions online. So for the next twenty minutes, I'll be talking about quickly what is AIOps, first of all, why does it matter now, and what are the general challenges in traditional IT operations, and how AI solves those key challenges, and, of course, doing a deep dive on a transformation case study for a fintech, how AI completely changed a chaotic landscape into a controlled, intelligent IT operations landscape.

With that said, let me get started with what is AIOX. AIOX is all about using AIML to drive automation to enhance IT operations. If you look at IT operations, right, it is traditionally very reactive, and it is very, I would say, the SRE engineers feel burnt out, and it is more of limited automation. So that's how the landscape is in general. So AIOps helps to transform this reactive way of traditional IT operations into highly intelligent and more predictive and proactive. And how does it do it? It combines the power of technologies like big data, AIML to do to enable automation at IT operations landscape. And AIOps has to be looked in something beyond observability. Right? Observability is is not, just about bringing the telemetry data and enabling an environment where all the data is available in one single pane of glass view, but there is lot more that is going behind.

So when we say AI ops enablement beyond observability, there has to be a contextual insights about the entire IT landscape, which cross functional teams can act on it to solve problems quickly, reducing the meantime to resolve problems in production. And AIOps usually is overlooked all about as tool. But, honestly, it is much beyond a tool. Right? AIOps is about bringing the right processes and the right culture, probably using a SRE kind of a culture. We have to build with an automation first mindset. So AIOps is about more beyond tool, enabling SRE principles, which helps to bring an automation first mindset in the landscape to drive operational excellence. That is all about AIOps. And why does AIOps matters now than any time before? The primary foremost reason is explosion of data. So we have a massive tsunami of data, telemetry data that we collect from IT systems, which is of huge volume, which is beyond the human capacity to analyze.

That's where AI ops is very important to ingest and analyze the data at scale. And today's cloud native distributed architectures and dynamic infrastructure, they are highly complex, and you need a deep visibility across the layers, across the stack to quickly nail down if something is going wrong. And be it customer or business, I think everyone demands always on systems with zero downtime. No compromise. So, hence, we need a better way for faster problem resolution, better detection, better way of diagnosis, better way of remediation. All this can be made possible with AI ops. And thanks to the shift what we see, because of ASR culture now, where it was more traditionally reactive when ASR is driving, sir, some of the enterprises in a very proactive autonomous operations. How do they do? Primarily using SLO driven operations.

So in order to meet stringent SLOs, AIOps helps with predictive insights and proactive way of preventing problems by detecting anomalies upfront before even it reaches the customer. Hence, AIOps definitely betters now in today's complex landscape. If you look at AIOps, it by and large transforms IT operations in four dimensions. The first dimension is definitely reactive to proactive. Right? I mean, it has been always reactive. After the problem arises is when we always looked into the problem, but now it offers a better way of predictive and preventing the incidence before even it impacts the customer. The second dimension is, as I was mentioning, it was all manual, but now, today, we talk about automated mindset, automation first mindset with a complete set of operational tasks being automated. The third dimension is very important because across SDLC persona, or be it customer or technical stakeholders or business stakeholders, everybody expected different kind of matrices, and there are plethora of tools that are being used, very siloed organization.

So now with AIOps, we bring a centralized visibility across the stack, and everybody is able to use the same single source of truth. The last but not the least, goodbye to guesswork and intuition. AIOX is now able to drive data driven decision making, and the culture it is trying to build now actually creates the ecosystem to make IT operational excellence a possibility. So now what are the key challenges in a traditional IT operations? If you look at traditional IT operations, the first and foremost challenge is the lack of unified observability. If you look at the enterprises, they often have three to four different tools. Every tool is giving different set of data. One metric, the other two logs, traces, events, alerts. Different tools giving different set of data.

It takes lot of time for the support engineers to switch between different tools to arrive at, to nail down the problems and investigate the problem. The second important challenge is the operations team gets overloaded with thousands and thousands of alerts. How do they filter the noise? How do they understand the critical incidents? How do they even remove the false alarms? They always burn out, and there is always a delayed detection coming in because everything is manual. And I was as I was saying, infra support team uses a different tool, application support team uses a different tool. So often it leads to finger pointing issues because of the siloed tools. And majority of the incidents are completely handled in a manual way, be it triaging to routing the incidents to fixing. Everything is manual. There is a huge person dependencies.

Of course, because of person dependencies, there is a human error, higher mean time to resolve problems, and there are a lot of inconsistencies. And, of course, there is a reactive way of looking at things, no automation or very limited automation due to which, end of the day, customers get impacted because of long system downtime, making them very unhappy. So now how do AIOps addresses these challenges? The five key challenges. Now with AIOX into the game, it completely revolutionizes how IT deals with these challenges. Observability pipeline with ML models help us to bring in autocorrelated views. All my telemetry data, be it metrics, logs, even traces, all the different telemetry data is correlated, and the autocorrelated views are made available in a single pane of dashboard. Alerts, again, there is alert suppression models, and it does event correlation, and it tries to group the alerts and create one meaningful alert, rises automatically in an ITSM tool for the operations team to even look at it.

The centralized tool with data lake integration often gives custom dashboards for different stakeholders across the domain. So everybody looks into one single tool. There is one single source of truth across the board. And what was done manually, now we have pattern recognition models, and we have, root cause analysis automated, even NLP model supporting a ticket classification to routing. There are a lot more efficiency in the system because of ML and NLP models. And, of course, what was Reacto now, be it incident responses, self healing scripts, gen AI powered a assistance, whatnot, it is more proactive and predictive. So a typical IT operations incident handling workflow looks like this on the on your left side.

Manually monitoring, filtering the noises, manually looking into the validate the right alerts, identifying the incidents, logging into the ITSM tool, escalating to the teams, collaborating, diagnosing the problem, manually executing the run book. It is too manual. But the right side flowchart gives you how AIOps brings a transformation where we can even think about more than 70% of reduction in MTTR. So there is an intelligent monitoring. Even correlation is automated. Anomalies are detected proactively. Root call analysis are done by ML models. There is an automated triage happening where if required, teams are notified. Or if there is a self healing workflow already available, it gets auto executed. So end of the day, the ML models are continuously learning the ecosystem. Right? So over a point, they gain such a great intelligence to do real time root cause analysis and leaving a way for autonomous zero operations.

These are some of the AIOps popular platforms. As I said before, AIOps platform has to be not looked as just using a tool, just throwing an engineering tool into the enterprise. It cannot create the magic that we are talking about. It has to be combined with the power of processes and creating the culture. That's when site reliability engineering principles play a vital role. So SRE and AIOs together, it can create a magic in the enterprises to to bring operational excellence. So now for the next ten minutes, I'll quickly take you through a transformation journey of a fintech enterprise where we were able to transform a highly chaotic landscape into a controlled, intelligent operations over a span of eighteen months. And you can think of chaotic environment with frequent outages, high system downtime, very limited automation, very limited monitoring, high resolution time.

Of course, customer very unhappy due to, everything done manual, so there is a high operational toil and whatnot. And over a point of eighteen months, we were able to build a full stack observability using elastic. We brought SRE culture where we were driving operations in a complete SLO driven way. So we had error budgets to decide. So we were operating with SLOs and SLI with the powerful automation first approaches, which transformed our release and deployment engineering. We had pipelines. We brought in blue green canary deployments. We brought complete infra provisioning under Terraform using IAC. We were able to bring chaos engineering integrated as part of the pipeline. So we were able to think through proactively the potential failure use cases and bring it validated and improve the resiliency characteristics of an application much earlier in the controlled environment.

The outcome, we were able to cut down MTTR by 70%, and incidence got reduced by 80%. I think seventy to eight eighty percent. So now when I speak about all this, it wasn't an overnight journey, and it was able to be made possible with a systematic phased approach. So now, overall, if you look at we started with a very reactive, phase and then slowly moved on to an autonomous phase. By and large, if you look at this transition, the journey comprised of bringing three capabilities. The first capability is, am I able to understand if there is a problem? Am I having an observability to understand what's happening in real time and detect incidents when it happens by bringing down the MTT being time to detect problems? Can we have the capability? That is the first part. The second part is about, okay. Good. We were able to detect it. But can we predict it proactively?

Can we use anomaly detection or it could be identifying the faults? How do I do a change impact analysis upfront proactively and give some intelligent suggestions to the operational team? That's about the predictive capabilities. And third, it's not about just detecting and predicting. How can I mitigate with the power of bringing auto remediation workflows and talking about self healing system, which are completely autonomous, bringing a closed loop automation? So these are the three distinct capabilities that we have to bring when we say AI ops enablement. And when we go when we went through these four phases of level one, two, we were able to bring a full stack observability.

Level three and level four, the shift was very, very tough. So level three, we were able to bring a basic level of automation. Okay. What's next? How do we bring a zero ops autonomous operations? From level three to level four, it was the more challenging journey than the other levels. So how do we do that? We were able to do a three step strategy. The first step is about, can we identify some of the use cases where extreme automation is possible? What do you mean by extreme automation? Can we automate certain use cases which can fully replace the human intervention? For example, alert noise reduction, what I was said, talking about, anomaly detection or automatically routing the incidents with NLP models in place to the respective team or with the predictive, capabilities, with pattern analysis capabilities, can we do a root cause analysis or build a self healing workflows because of the historical pattern analysis?

All this was made possible, which completely replaced human intervention. So that was extreme automation, the number one strategy we used. Number two strategy was what could be the potential use cases where we could use AI in an augmented way that empowers our IT operations team with the intelligence, actionable insight, and helps in decision making. For example, change impact analysis, or it could be the anomaly detection. So AI ML models did lot of heavy lifting, but it gave its recommendation, suggestions with its confidence score for the operations team to act on it. Be it capacity planning or bringing an AI auto a copilot into the game, it definitely skyrocketed the efficiency productivity level of IT operations team, but we did had the operations team to make critical decisions. The third strategy, I think if I don't touch upon the third critical strategy, which is about GenAI and agentic AI, I think it will the the strategy, the transformation looks very incomplete.

Right? So now what were we able to do with GenAI and agentic AI? We were very clear that there were some use cases where we could bring a GenAI agents into the landscape where it could completely skyrocket the way the time at which we were delivering things to reduce by 50 x or more. And there were use cases with AI agents. We were able to completely automate, do it in an autonomous way. For example, SOP creation or knowledge based creation, these activities, by bringing assistant, it helped in ticket analysis, SOP creations, documenting, creating the summary, and whatnot, albeit onboarding the new engineers, creating, supporting to resolve incidents. All this was made possible with Gen AI assistance. But what I'm talking about for agentic AI is, can the agent autonomously do certain act and, again, autonomously, I'm talking about a multistep agent.

It is not just doing one dedicated activity. So that can it have a capability of investigating the symptoms, analyzing the ticket, looking at various data? It it has intelligence to look at what telemetry data and pattern of how things were resolved in the past and solve certain challenges by itself, implement the fixes on its own, and just notify the teams that, hey. This is what I did. And there were some use cases. Definitely, we were very, very clear to take certain decisions having a human in the loop. That's when we stopped having agents just at the level of talking about suggesting, giving suggestions, not going beyond that. So now these three step strategies, extreme automation, AI, augmented automation, and Gen AI agentic AI automation. So these three levels of automation, it actually helped us to transform from level three, proactive to autonomous zero ops culture.

That we were always dreaming it, which was able to be created with the power of our three phased automation approach. Business benefits, I don't need to talk explicitly. We were able to achieve improved uptime, faster way to detect the problem and resolve the problem, of course, the total cost of operations, operational efficiency gain, and the the way we were able to bring automations in as operations as an autonomous zero ops way, definitely, there is a very huge impact on the customer experience.

This is the transformation we are talking about when we say chaos to a controlled intelligent operations. Before I wind up, I want to just leave this message with all of you. AIOps is not just about a tool. It requires a mindset shift. That mind shift shift can come by bringing the right site reliability engineering principles, creating an ecosystem for you, not just throwing good engineering tools will not create this magic. Let's embrace AI ops for driving operational excellence. Thank you so much for listening in. I'm open to questions now. Okay. So, Rina, you have a question. How long did this journey take? Actually, this journey, as I was saying, it it was a eighteen months journey. And the first one year or plus fifteen months, it was more of coming up with lot more strategies on automation that powered us to make the autonomous operation a possibility.

So level three to four was really tough challenging. But, again, overall, it was an eighteen months journey, and we are still in level four. We are still exploring various use cases to bring GenAI and agentic AI into the game to fast forward lot of few more areas where we want to improve. But it has been an eighteen months journey and such a great transformational journey that we've experienced last eighteen months. And, again, it is it varies, enterprise to an enterprise. Right? So it could be a mix of different things. It could be, the organizational culture and the the way the operational SRE teams are empowered in decision making or how powerful is your error budget or policy. So a lot of such dimensions definitely play a role.

Again, the what I'm talking about in the fintech transformation is it took eighteen months, but I can confidently see the same, eighteen months journey in another fintech, enterprise itself may or may not be possible. It primarily lots of ecosystem environment plays a vital role here, which should not be, underestimated. Okay. Christina, I have a question. I'm curious if there were any big failures that actually accelerated your learning and innovation. Absolutely, Christina. Every day was of, I would say, lot of failures to embrace and then act upon. Right? Primarily, I think, the failures, I think more of failures we have to, deal with, when we brought a more focused approach on automation. I think some of the automation strategies where we expected to bring an extreme automation, extreme reduction, it it wasn't the case, honestly, on the ground.

So where some use cases where we planned for a partial two x the productivity, we were, like, quite surprised sometimes. And, actually, a lot more failures we had while going through the Gen AI, agent decay journey, honestly, than the first two phases because the level one and level two was more about adopting a tool and then bringing the right maturity and using the complete tool features, all of it.

I think level three and four definitely was very challenging, and I think, it is a very appreciatable experience as a team we all went through to drive operational excellence for the enterprise. Thank you so much. I would love to connect with all of you, and please feel free to connect with, LinkedIn. I have put my QR code here. I would be very happy to take up questions offline. Thank you so much for listening.