AI over Legacy Systems
Corina Staicu
CPOReviews
The Future of Legacy Systems: How AI is Transforming Enterprise Technology
Welcome to the exciting world where artificial intelligence (AI) meets legacy systems! I'm Corina, chief product officer at Alteus, and today we will delve into how AI is revolutionizing the tech possibilities surrounding legacy systems. With over 18 years of experience in various IT roles, I have witnessed firsthand the evolution of enterprise systems and the challenges organizations face with outdated technology.
Understanding Legacy Systems
So, what exactly is a legacy system? This term can mean different things depending on who you ask:
- Business Perspective: Often seen as old software or outdated technology, like COBOL in the banking sector.
- Developer Perspective: Considered "dying code," as newer technologies and developers hesitate to interact with it.
While legacy systems may still be operational, their ability to adapt and integrate with modern technologies is severely limited, leading to what is known as "technical debt."
The Challenge of Replacement
Many enterprises hesitate to replace legacy systems due to:
- Long time frames — replacement often takes several years.
- Cost overruns — projects can exceed budgets significantly.
- High failure rates — a staggering 70% of replacement projects are not deemed successful.
However, the landscape is changing, thanks to advancements in AI technology.
AI-Assisted Development: The Game Changer
In 2022, industry leaders like Sam Altman predicted that AI would boost programmer efficiency by over 30%. By 2024, Google reported that 50% of its code was being written by AI, and a year later, that figure soared to a staggering 90% at Anthropic. AI is not just a tool; it is becoming integral to the development process:
- Speed: AI-assisted development collapses timeline for system replacement, allowing companies to shift from years to months.
- Efficiency: Existing specifications can be used to enhance the new systems, streamlining the development process.
- Lean Approach: Developers can eliminate unnecessary processes from legacy systems, creating sharper, more efficient solutions.
Two Approaches: Replacement vs. Overlay
Organizations have two primary approaches for dealing with legacy systems:
1. Replacement Thesis
This approach involves building a new system from scratch. With AI's assistance, organizations can:
- Implement much faster timelines.
- Rebuild using existing specifications for essential features.
- Streamline processes, leading to simpler and cleaner solutions.
2. Overlay Thesis
The overlay method adds intelligence atop a legacy system without modifying it. Benefits include:
- Minimal disruption to core operations.
- Quick implementation — pilots can be operational within weeks.
- User-friendly interfaces, allowing natural language interaction.
In both cases, companies need to ensure they have proper governance in place to monitor AI interactions and maintain performance standards.
Case Studies: Practical Implementations
Let’s explore a few case studies to illustrate these concepts:
1. Project Management Tool Revamp
We took a traditional project management tool, a legacy system, and created an interactive user experience for quicker data retrieval:
- Users can input commands naturally, like "show me my vacation dates."
- The AI facilitates real-time interactions, improving productivity.
2. Big Data Insights
In the healthcare sector, we implemented AI on a big data platform, enabling clients to perform complex queries easily:
- Direct inquiries about patient scores lead to rapid insights.
- AIs provided top service analysis without the need for cumbersome reporting.
Centralized Governance with Altais
To ensure effective usage and governance of AI-driven solutions, we developed Altais, a centralized gateway that:
- Enables monitoring and auditing of AI interactions.
- Facilitates the creation of custom agents and applications.
- Provides a user-friendly chat interface that avoids the need for cumbersome training.
This
Video Transcription
Hi, everyone. My name is Corina. And today, we will dive into the session I over legacy systems. I'm Corina. I work at the intersection of AI and enterprise systems.And, in the past years, I've been the chief product officer at Alteus, where we build an AI gateway for enterprises that focuses on governance. I've been working in the IT field for over eighteen years, on different projects in different industries. And in my past positions as CEO, project manager, or team lead, or even programmer, I witnessed a lot of project discussions and implementation of replacement versus overlay or connecting, an existing system. So this is why from this perspective, I thought it was important for us to talk today about how AI is changing the tech possibilities, regarding legacy systems. I work at eSolutions, the company that is building Altais, and, we've been working in projects with customer services for over the past twenty five years.
We're a group of 100 professionals, and we delivered over 300 successful projects. So from this perspective, I will try to present to you what are the current possibilities and how we can witness, these new options that we now have with AI on the table. If we look at the past years, they say that the over 62 per percentage of the enterprise core system are still running on older technology, even older than iPhone. But, this begs a question on how to do it, those enterprise or systems that are considered legacies. What is really an legacy system? Because if you're talking with different people with different roles, you will receive different answers. Common knowledge or if you ask the business, they think it's all software. Old code technology, if you think of banking where Cobalt was back in the seventies, eighties developed and nobody nowadays is trusting to touch it.
But we are still thinking of technical depth. Why replace it if it's even still working? So this is a perspective that is common knowledge. But for the from the developer's perspective, all legacy system is not necessarily that old. So it's a different word attached to it, is dying code. Because the new developers with new technologies, once they don't touch the code anymore and is becoming rigid and nobody has the confidence to touch it, they consider it legacy. So if you have something that is not updated or upgraded or even look at the security releases for it, it's kind of a legacy system for your company. So defining legacy as old receives one answer if you look at it. And if you define it as hard to change, you'll receive other answers and possibilities to approach it.
Back in the days, if you think thought of it, just replace it and look at a statistic in 2023. Most of the industries based on a Gartner research will not dare to do it because, usually takes longer than initially estimated. Even three times I put here on the survey, I also find five years over the five years to to replace something that is old and, big incorporated in the core, business enterprise. Sometimes the cost overrun was exceeding expectations on the long run, and most than 70% would not even declare it as success. But yeah. This was back in 2023. So what changed that we can now so I think of it more confidently? Back in, 2022, Sam Altman promised something that was sounding pretty pretty good. The fact that programmers will be more efficient with AI, more than 30% back then he was saying. And after that, ChargeGPT got on the market.
And because we are a technology company and we use as AI assisted development, we received that news very intrigued and happy to see how we can deliver value to clients and customers much faster. After that, in 2024, other news were coming in. Google declaring that they are writing the code their code with over 25% with AI. Few months later, 50%, so the scale was increasingly higher. And, the last declaration at the the last the end of two thousand twenty five was that more than 90% of the code for Anthropic will be developed by, by the AI. So this AI, assisted development sounds very promising. And also besides the AI assisted development, something else happened, meaning we are now having agents that are not not only AI and models, but also they have other properties like context and tools, and you can provide them with capabilities so that they can do more than just, answer and chat on specific tasks.
So you have on one side AI who is doing the development and at the other side the possibility with agents. Let's look what we can do with both of these new things on the market. We have the replacement thesis, of course, where you approach this legacy system and you replace it. You write it from scratch again. And this is a a good opportunity to be put on the market because now we are talking about months, not years. So AI assisted development is collapsing the timeline. If you have a good team that knows how to use AI because this is also important, they can ship much faster, and the cost versus time at replace has structurally changed. So if you have in your company such a system that is named legacy from whatever reason, you can now request now a new quotation from the market and, see if it's drastically much slower lower than, couple of years ago.
Another benefit for replacing from scratch the the legacy system is that you already have the specs. If you already have the legacy system that is working, you know what the features might be in place and you need it, so it's simple to rebuild them. You also rebuild it you can also rebuild it rebuild it lean, meaning you can go through the processes that are no longer needed and use these capabilities to improve and make process much easier and much concrete and lean and remove the garbage to put it shortly. So this replacement is not hard anymore. And, if you just need some preconditions and if you have the right team, it's much easier to to deliver it with AI. This is a replacement thesis. Next would be the overlay thesis. When you are using AI without touching the legacy system, but you put an you add an intelligence on top of the intel of the legacy system.
In the next slides, we will talk a little bit how in practice we can do that. To give away some preview, we use agents. And, if we overlay, it's it's a good practice. It's a good choice because the system record stays the same. AI is becoming just something on top of it. The change is not affecting the old the old stuff, and, you have smaller risk of perturbing the the business and their work. So this is the the benefit of it because the core operations are not touched. And, how we can do that is having three layers. First layer is interaction layer, when you are having the interaction with the AI, like, as a human via a chat or on different kind of UX interface. After that, you have the intelligence layer where you have the agent or the LLM or whatever two is needed.
And the old system, the legacy system, stays untouched as layer number three. And, just via the AI, you are just communicating with it. The important stuff, the prerequisite here is that you have, for this system, the access to to connect with it via API or directly in the database. It has it needs to have the connection layer. Okay. So if we look at the pros for overlay versus replace is that overlaying is much faster. We are talking about pilots that can be in place from two, three weeks depending on how many use cases are covering versus the the multiple months program if we are just replacing it. When we are doing only the overlay overlay, we don't stop the production or the operation on the old, legacy system or the the system that is no longer living because nobody touches it. Users meet the AI, via the chat or another easy to to understand interface, so you don't need to retrain it because, user are communicating with the legacy system via natural language.
And, the minus on the replacement is that if you are doing a new application from scratch, you are redoing the system, you need to also migrate. Most of the case, you need to migrate the data behind it. So, it's an extra time consuming activities on the left side. So, yeah, it's important for us when we choose something to understand the risk and implications of each options. And, of course, even if you have an overlay over it, over the legacy system, you will still need some kind of governance. Governance, why? Because we have AI models and agents that, can be developed all over the organization. And it's important for you to have some kind of observe to have observability so you can see how they are used and what cause they are, costs they are, they are producing.
And also to have some central policy. Who can use it, what kind of agent, and even auditing to see how the people are using the agents and how they are interacting with it. Are they satisfied? Or this way, via the audit, you can also see if the AI is hallucinating, meaning it is not behaving properly and it's providing not okay answer from the perspective of the users. I prepared here three case studies so we can actually see how this can be done. And we will see, again the slide with interaction layer, intelligence layer, and data system, the legacy system layer. And, I will show you in the following slides how on a use case, we redid the interaction layer and the intelligence layer for a couple of scenarios.
So new application interface connectors and models. The the project setup was simple. Meaning, we have a project management tool that is used in the company by the HR and project managers. They have all the in there, the system has all the, the data regarding the projects and also the, the employees of the company. And I took we took this as a legacy system to to create a new easier interactive UX, an interface, so that people can more quickly, interact with the system and get the data that they need. The proof of concept implementation took a couple of days bringing it to production, depending on the company size size because you need communication and also sessions for the people to to accommodate with it and couple of time to give feedback. But, I will show you in the end how they can use it. So this is a dashboard where you can see you can interact with it, also via prompts.
Show me my vacations date or put let me put some vacations on the date upfront. So you no longer need to log in, enter in old application or if it's a design from the nineties, uploaded to Excel or something like that. You have another interface where you can see actions that are available for you as a user, like request absence or show me my timesheet or, import my project hours. And you can just direct the agent and telling them what to to to do for you. These are examples of, prompts. Book three hours, on the project to evolution, and the agent is already showing me, that he's selecting for me three hours selecting the project, and I just need to click yes. You can do that. And here is, short list where are all my projects that are I've been working on, and he is actively showing me all the projects.
Again, I'm not saying that I cannot do the same things in the legacy system. I can. But, this interaction via this kind of interface is much faster because I'm just clicking and asking a comment and he is providing me with the data instantly and not going through multiple pages and clicks and shows my data. Another way of interacting with the same system with project management because I thought it would might be an example that everyone here attending might understand it very easy. So, the legacy system in this case is the same project management tool. And what we have now is, an chat interface without any add added buttons that is, serving my commands. And here is the Altaus, the product that we've been working on on interface where I'm choosing the connector for that application.
And the extra thing here that you can implement via the agent is that each agent can, guide the user on how what tools they can use and what resources they have the right to access. Meaning, on my account, I can see the list of projects. I can see the list of employees. I can, get the date and time and so on. Depending on each system application, you can extend this kind of capabilities. On on this example, I told him to give me the man days book per per project, and he gave me the number of, man days book per project. And what is important here is that the tool is also you you can look at the reasoning because it's a special agent to see if it's right. Because sometimes when we have the data in, in in other systems, what happens is that the same data might be in different tables.
And that is why it's important at the first phase when you are connecting an agent to, older system or another system is to to test him and to look if he is doing right things. Meaning, now he's looking in the table project with project ID, what what I've told him in internship 2023. And he's also knowing after doing the sum to divide by eight because I have asked him the man days and not the hours. So he's done right in this scenario. Okay. We now saw that we can interact via prompt in a chat or in a new interface. Now it's time to ask ourselves what do we do with big data? Meaning, we have lots of data with a big data platform because in our company, we are working frequently with big data. We have a special division for this.
And this is no longer the scenario necessarily for legacy system, but for any system that has big data for any company that has big data platform and can work with the data. And the beauty of having AI in place on the big data over the big data is that you can build an overlay, and you no longer need to, necessarily do custom reports because because you can directly ask the the agent. Here's an here is an example from, a health care industry company where I can ask him questions about the data that is gathered over different channels. And I'm asking him, give me the patient score measures for the patients back in March 2016 based on his their feedback. And he, after going through, as you can see, a lot of queries in the database, he gives me he gives me the answer that is 8.55 out of 10. When you are going with AI in the the database or in the big data platform, it's important that you also help him, to identify correctly the data.
So this is why, I strongly advise we strongly advise that someone, mostly the business, has the data shaped correctly and put a description on the columns and on the tables so that, the AI gets more insights on which, are the relevant and why are the relevant data in the the data big data platform.
From the confidentiality reasons I've put, I'm I've blurred some of the stuff. But, what I wanted to show here is that it's very nice to have deeper insights because now I'm asking him what are the top 10 most profitable medical services in 2026. So he goes in the database and he sees all the, reported data per each analysis or service and, gives me a top 10. And after that, tries to tell me why the first one might be might be on number one, why why the second one might be on number two. Looking on all the things and also, calculating and analyzing, this. So now if we are going to see this agent, this specific agent is more specialized. And on the big data platform, it's it's really nice to have AI after you have clean data and, a strong governed, big data platform because you can also benefit from, lots of insights and the human analysis is much easier to to be done.
Okay. So we have agents. We saw how we can, implement them and how we the question would be how to govern this. Because, we might have multiple agents. And, the solution or our answer would be to have one gateway, meaning one place where you can see how the agents are behaving and how people are are work interacting with it. So this is the main reason why we build up Altaus. This is, a tool that is designed for, for enterprises because it's having the gateway, meaning the console where the admin console where you can see all models and policies, cost, and audit, audit them. You can see in if you implement this, if you can you can see in, all the enterprise who is using how much and what he's using. After that, you have the agent builder custom agents apps where you can define, test, and deploy versions over, different applications.
And the end part for the users within the enterprise company is the chat. They don't need to learn new interfaces. They only have a chat where they can ask a question and interact with the knowledge base and other company applications. So the idea was to have everything lean, clean, and a human conversational way to interact with the systems. We have multiple models, so no vendor locked in. Public models, also on premise models. We have live observability where via the dashboard, you can see the token success rate per user, per call, per agent. Ajentic AI, this is a new terminology which is trending in the past months because you can build up around the agent. You can harness, like a harness, meaning you you give him context, skills, connections so that the agent is less likely to to do mistakes.
So this, kind of ability we offer via the interface. And you also have budget guardless guardrails defining quota per team or per member, per project, or per company department because we have this kind of categorization so that, there are easy to follow. The agent is pretty much straightforward. Like, I'll tell you this chat. How is it here? How many vacations? Again, this is an example from the PM world. How many vacations, they days team booked for, for next month. This is a question that PMs are finding very useful because you can you can predict workload in the future. And this is an example that AI is already via Altaus chat, via an agent can calculate directly, not going individually to each member and see that, this person has two days, three days, three days, and so on.
So you calculate it, and, DA can, can do the this high kind of work and give you the answer quickly. So, user talks directly with the, with the chat and interacts with the system. Every conversation is governed via the gateway. You can also put policy there so that it, it's careful at language or what kind of prompt is using, so that it bans if, the prompt is not politically correct. Agent discover inside the chat. Yeah. You overlay, it's becoming the front door for the whole organization. Nobody needs to people don't need to, learn all the buttons in the the systems that are already here because the AI can access it and, do the summary and answer it. The part with the agent, building an agent is quicker than before. Now we are talking about days.
It takes longer to decide on what to do than to really do it. This is an extended timeline because usually, approvers approvals take longer than actually build it. This is the end of the conversation. If, you liked it and you would like to have a demo of Altais, just scan the QR code or write an email to me directly. And, we can look at a specific, workflow that you have and, talk with, with you and the team. Okay?
No comments so far – be the first to share your thoughts!