The AI Investment Gap: Why Your Pilots Succeed and Your Deployments Fail by Christina Garcia
Christina Garcia
SVP, Head of Engineering & TechnologyReviews
The Future of AI: Redesigning Human-Machine Partnerships for Success
As organizations across various industries rush to invest in Artificial Intelligence (AI), a concerning trend emerges: many are failing to leverage AI effectively. The problem doesn't lie in the technology itself but in the approach companies are taking towards AI integration. In this blog post, we will explore the key missteps organizations make when implementing AI and how to avoid them for a transformative experience.
Understanding the AI Investment Gap
The first step in solving any problem is identifying it. The issue at hand is what I call the **AI investment gap**. Many organizations fall into the trap of treating AI like any other automation tool, leading to disappointing results.
- Inadequate Integration: Companies often try to inject AI into existing processes instead of redesigning those processes from scratch.
- Failure to Redesign: Organizations overlook the need to fundamentally rethink their workflows; they expect AI to work miracles without making necessary adjustments.
- Neglecting Human-Machine Partnerships: Effective use of AI isn't about replacing humans; it's about creating a partnership that optimally leverages both human and machine capabilities.
A Case Study in Failure
The fintech company Klarna provides a classic example of what goes wrong when organizations mismanage AI adoption. Despite initial success with their AI customer service assistant, which handled millions of inquiries, customer satisfaction plummeted a year later. The root of the issue stemmed from:
- Removing humans without redesigning the processes that remained.
- Failing to ask essential questions about the roles humans would play when AI took over a significant portion of the work.
As a result, Klarna faced a crisis of service quality and was forced to reevaluate its strategy and bring back human employees.
Learning from Success Stories
On the other side of the spectrum, Moderna, a leader in the mRNA vaccine space, showcases the potential of successful AI integration. Their approach included:
- Organizational Redesign: Rather than simply rolling out AI tools, they merged departments to ensure coordinated efforts in AI implementation.
- Holistic Process Evaluation: They reevaluated every business process to incorporate AI effectively, leading to remarkable improvements in efficiency and employee adoption.
The result? Faster manufacturing cycles, increased accuracy, and widespread adoption across the company—evidence that strategic planning and redesign can yield powerful outcomes.
Implementing Effective AI Strategies
To avoid the pitfalls of AI integration and harness its full potential, organizations must ask themselves three critical questions:
- Are we injecting AI into an existing process? If yes, reconsider this approach; it may lead to failure.
- Are we replacing humans without redesigning the work around what's left? This can have long-term negative consequences.
- Is the partnership observable? Ensuring that human roles and AI functions are visible will foster trust and usage.
Closing Thoughts: The Future of AI Is Transformation
The key takeaway is that the future of AI is not found in simply purchasing better tools but in designing better human-machine partnerships. By investing in transformation rather than technology alone, organizations can create a seamless workflow that reimagines existing processes.
As we navigate this rapidly evolving landscape, remember: don’t inject, don’t replace, but redesign your approach to AI. The potential for success lies in your ability to develop a cohesive partnership between humans and machines.
Now is the time to ask the difficult questions, to challenge existing structures, and to embrace the uncomfortable work of redesign. The reward? A truly transformative future driven by AI.
Video Transcription
The AI would read the email, pull the relevant data, generate the quote, and the client rep could then send it. The demo was flawless.The executives in the room were excited. The project sponsors, people who'd been with the company for over ten years were convinced that this would transform our quoting process and help our reps move out of email and into prospecting. I was brought in to help evaluate the technology, how it was built, security, scalability, resilience, and data protection. And I had this sinking feeling in my stomach, not because the technology didn't work, the demo did work, but because I could see exactly how this was going to fail. They were treating AI like every other automation tool they'd ever implemented.
They were going to inject it into step three of their existing eight step process and expect magic to happen. But I was the outsider. I'd been there less than eighteen months. They've been there over a decade. So in my one on ones with my boss who was convinced that this AI was going to change the game, I shared my concerns timidly and very carefully. Six months later, almost nobody was using it. This is what I call the AI investment gap. Here's what happened with that email quoting agent and what's happening in organizations everywhere. We're investing in the technology. We're investing in the pilot, but we aren't investing in the thing that actually matters, reimagining the process from scratch. Instead, we did what we always do with automation.
We looked at our existing process and asked, where can I inject AI to make it faster? But AI isn't RPA. It's not workflow automation. It's not a tool you drop into step three and walk away. AI is a partner, and partnership requires completely rethinking how the work gets done. And here's the thing. This isn't just my company's mistake. Some of the most aggressive AI adopters in the world have made a bigger version of this same mistake. In February 2024, the fintech company Klarna made a global announcement. Their new AI customer service assistant built on OpenAI had handled 2,300,000 conversations in its first month. It was doing the work of 700 full time agents. The CEO projected 40,000,000 in annual savings. Open AI comarketed the launch. It was the case study every consultant wanted to put in the deck. Klarna par paused hiring. They cut their workforce from 5,500 to 3,400.
The CEO went on podcast saying AI could already do the work humans were doing. A year later, customer satisfaction had fallen sharply. Service quality was inconsistent, and Klarna was quietly asking software engineers, designers, and marketing staff to step in and answer customer inquiries. By 2025, the same CEO who said AI could replace people was public publicly announcing that the company had prioritized cost over experience, and they started rehiring humans. Here's what went wrong, and it's this exact same mistake as my email quoting agent, just on a much larger scale. Klarna invested in AI replacement. They didn't invest in AI transformation. They removed the humans, but they never rebuilt the work around what was left. They didn't ask. When the AI handles two thirds of inquiries, what does the remaining one third actually need from a human? What does done look like?
How should this feel when it's working? They optimized for the cost line. They didn't redesign the partnership. And when the AI hit the things only humans can do, empathy, judgment, escalation, the messy edge cases, there was nobody left to do them well. In those early conversations about my email agent, I knew something like this was coming. I watched my team plan to inject AI into an existing process, and I knew it wouldn't work. But here's what I couldn't articulate yet. I couldn't explain why it was different. Now I can. Think about DNA, the double helix. Two strands wound together. You can't separate them and expect either one to function. The strength comes from how they're intertwined. That's what AI adoption requires. Human capabilities and machine capabilities woven together so tightly that you can't extract one without breaking the process.
Not humans using AI tools, not AI replacing human work, but human machine partnership designed at the foundational level. Here's what that actually looks like. Human shift left and right to the edges of the process where we do the work only humans can do. Strategic judgment when patterns break, relationship management and empathy, creative problem solving, deep expertise and context. Machine shift center into the middle where they create leverage, pattern recognition at scale, consistency across thousands of decisions, real time data processing, tireless coordination. Klarna pulled the humans out and let the AI feel the whole shape. We left our existing email workflow in place and tried to bolt AI onto the side of it. Different mistakes, same root cause. Neither one redesigned the partnership.
Both invested in the technology. Neither invested in the transformation. A few months after the email quoting failure, we got another opportunity. Different vendor, different use case, but this time, we approached it completely differently. Inbound calls would come to our carrier help desk. The calls would ring directly to the carrier service agents. The agent would pick up blind. No context. No information. They have to ask every discovery question before they can even start helping. We could have injected AI in the same way we did with email quoting. Let's add an AI assistant that helps the agent during the call. We could have done it Klarna's way. We could have replaced the agents entirely and called it a win. We didn't. Instead, we pulled up. We asked, what's the desired outcome? How should done feel?
We started with a blank canvas. We completely redesigned the process, and this is where the weave actually happens. Human shifted left. Our operations team structured the prompts, defined the rules and thresholds, built in the trust and safety parameters. They reviewed transcripts, modified triggers, refined when the AI should route versus handle. They architected the AI's behavior. AI shifted to the center. When a call comes in, AI handles the initial conversation within those guardrails. It gathers information about routes, loads, validates details. It does the prework, the prescreening, the data gathering and triage that used to take the first five minutes of every call. Human shifted right. The call only routes to a human agent when it needs human judgment, expertise, or empathy. And when it does route, the agent isn't picking up blind anymore.
The AI passes them all of that pre vetted information. Here's what made this work. It was an observable partnership. The operations team could see the transcripts and refine the AI's behavior. The agents could see what the AI did versus what they were doing. There was no overlap. It was a natural team effort. Humans architect the rules. AI executes within those rules. Humans handle the judgment calls. We didn't introduce this as here's a new tool. Have fun. We introduced it as a process change that optimizes their time and gives them control over how the AI behaves. That's the difference between injection, replacement, and transformation. And this isn't just something happening at a three p l in the Midwest. Some of the most successful AI deployments in the world are built on exactly this principle. Look at Moderna, the mRNA company. When they partnered with OpenAI, their CEO didn't say, we're rolling out a chatbot.
He said, we're looking at every business process from legal to research, manufacturing to commercial, and thinking about how to redesign the process with AI. Redesign, not inject, not replace, redesign. And they backed it up structurally. Moderna merged their HR and IT departments into a single function, people and digital tech. And they put one leader in charge of figuring out what work belong to humans and what work belong to AI. They didn't treat it as a tool rollout. They treated it as an organizational redesign. The results, in two months, employees built 750 custom AI assistants across the company. Their legal team hit a 100% adoption. Manufacturing cycle times for new therapies dropped from over ten days to five or six, and error rates fell by 80%. Not because they bought better AI, because they redesigned the work around the partnership. That's what invest in transformation looks like, not technology.
Now let me show you what it looked like for my engineering team because the same pattern works on a smaller scale, and it taught me something that during this case study doesn't actually cover. Two years ago, when we first introduced AI Code Companions, my engineering teams were massive skeptics, and rightly so. The tools weren't as good then as they are today. We didn't force it. We introduced it as a tool like their IDE available if they wanted to try. We created a community of practice, a Slack channel for sharing, and then we watched. Early adopters started posting wins. I used it to scaffold test cases. I had it troll through repos and create documentation that was never in place. We started seeing patterns form naturally. Champions emerged. Natural champions emerged. Engineers who figured out how to work with the AI effectively. They weren't just using it to write code faster. They were shifting how they worked.
And that's when we learned something Moderna's story doesn't tell you. We can't always design the perfect process upfront. Sometimes you need to introduce the capability, observe the patterns, learn from the champions, and then redesign the process around what's actually working. As the tools got better, we took what we learned from these champions, and we started discussing delivery process shifts, not just tool adoption. We shifted to expecting engineers to be deeply involved in discovery with product, to fully flush out features with complete acceptance criteria, functional specifications, locked UX designs, and now a small team of engineers can build a plan and drive their AI companions as agents through spec driven development.
Engineers shift left architecting, technical design, working with product, defining a solution, and outcomes. Engineer shift right, code reviews, testing, ensuring quality of the outcome, and AI shift center, writing the code, generating the scaffolding, producing the documentation. It's less about who writes the most line of codes and more about the technical design, the code review, and the testing with an awareness of the outcome that's needed. Three different stories, three different scales, same pattern, pull up, understand the outcome, design or observe the weave between human and machine, then commit to the redesign. So here's what I want you to walk out of here and do. If you're leading an AI initiative right now, whether it's a pilot or you're planning a deployment, stop and ask these three questions.
One, are we injecting AI into an existing process? If the answer is yes, you might be about to repeat my email quoting mistake. Two, are we replacing humans without redesigning the work around what's left? If yes, you might be about to repeat Klarna's mistake, and trust me, that one is much more expensive. Three, is the partnership observable? Can humans see what the AI is doing? Can they shape it? If they can't see it, they won't trust it. And if they don't trust it, they won't use it. Pull the team together and ask, what's the desired outcome? What does done look like? How should this feel when it's working?
And then design the complete flow, understanding where human judgment creates value, where machine consistency creates leverage, and how to weave them together so the partnership is visible to everyone involved. Because here's what's at stake if you don't. You'll invest hundreds of thousands of dollars, sometimes millions, in technology that works perfectly in demos and sits unused in production. You'll watch talented people revert to old habits because the new process doesn't make sense. You'll cut the wrong roles and have to rehire six months later. You'll see pilot after pilot succeed and deployment after deployment fail, and you won't know why. We've gotten really good at investing in IT technology. What we haven't learned is how to invest in AI transformation. The messy, uncomfortable work of starting from a blank canvas and weaving human and machine capabilities together at the DNA level. So speak up. Ask the hard questions. Don't inject.
Don't replace. Redesign because the future isn't about buying better AI tools. It's about designing better human machine partnerships. Thank you.
No comments so far – be the first to share your thoughts!