So before we begin the session, I'm Hema. I'm a senior technical program manager at Microsoft.Most of my work is focused on deploying hardware as globally all over about 60 plus Azure regions, and I have led a hardware deployment program for about three years now. I'm really excited to be here and to talk about something that often stays behind the scenes but is absolutely essential, the hardware infrastructure, the backbone of AI workloads. And let's get the session started. Thanks again for being here. I'm really excited to walk you through a topic that's both foundational and also fast evolving, the infrastructure behind the modern artificial intelligence. We hear a lot about what AI can do from generating text, images to powering Copilot or autonomous systems, self driving cars, and whatnot.

But what we don't always see is what's exactly happening in the background, and that's where things get really interesting and incredibly important. Behind every smart system is a layered stack of hardware, software, and engineering decisions that make it all possible. This session is going to be about those layers, how they are evolved, the challenges they face, and where innovation is taking them next. Today, I'll share a high level view of AI infrastructure, how it has grown, and where it's headed, and what it means for organizations building and deploying AI at scale. Let's jump in. So let me ask a question before we proceed with this. As soon as you hear the word artificial intelligence, what comes to your mind? Can I have some answers in the chat? The word AI. Anything that comes to your mind as soon as you you hear the word AI. Anything at all? Chatbot.

Yes. Robot. Yes. Machine learning. Yes. LLM. Of course. Awesome. Thanks for all the answers. But AI is not just about writing code or feeding data. Right? It's about enabling that intelligence to run effectively. Let's get one thing straight. Even me, I asked you the question, but if someone is asking what AI is, I'll immediately think about data and coding and everything. But let's get this straight. Artificial intelligence is not magic. It's math. It's really big, complex math. Let's think about race car. It needs the right engine. Of course, it's need it needs the right engine to run, but it also needs a track for the race to be completed. Similarly, AI needs the right infrastructure to run.

So when you're training a large model, let's say, one with hundreds of billions of parameters, it's like trying to teach ourselves every book in a library, but all at once. It requires staggering compute resources. When we think about AI, we often picture algorithms and data and everything we mentioned in the chat right now. Those are really crucial, and I'm not denying that. But AI is not just code and data. It's deeply dependent on the hardware that runs it. The truth is compute power defines how fast, how big, how far we can take artificial intelligence. Training today's large models, let's think about GPUs and multimodal systems, requires massive parallel processing across thousands of GPUs or specialized actuators. And once models are trained, they need to run efficiently. Inference, the part where AI makes predictions, demands low latency. Right?

If we have to ask a question and wait for weeks, we will not be asking the question at all. Correct? So, it needs low latency and high throughput systems, especially in real time applications. Without the right hardware, progress actually slows down. Training can take months. Let's imagine asking, AI app to generate a image and wait for weeks for the image to be generated or ask a question and we have to wait for days for the answer to be generated. Without the right hardware, that's what will happen. That's why we are seeing a shift to specialized AI chips designed not just for general computing. Right? Especially for AI and not for the these chips don't exactly save sorry. Yeah. These chips don't just save time and power. They accelerate time to insight. So we are saving we are seeing thousands and thousands of parallel processing, not even thousands. It's it's it's really huge of parallel processing happening simultaneously.

Ultimately, hardware infrastructure is what translates AI's promise to usable reality. It makes AI accessible not just for research labs, but also for industries like health care, finance, manufacturing, education, and everything in Birmi. So, hardware is not just a support layer to AI. It's a core enabler of the modern artificial intelligence. So where it all started? Today, we hear about AI everywhere from headlines to every home and wherever we go, the grocery store. Imagine everywhere is about AI. But what many people don't realize is that AI did not become a buzzword overnight. It has been slowly and steadily evolving for over eighty years now. Much of that progress was only possible thanks to the simultaneous evolution of computer hardware too. Let's see where it all started.

Let's begin in the year 1950, where AI began with logic and rules, like trying to teach machines to think through checklists. But the computers of that time, imagine how big they were, huge and slow. Think about room sized machines running on vacuum tubes. A good example is the early chess programs that followed rule based logic. So it can follow the rules, but it cannot it cannot learn. It just follows the rules of chess and then plays it. Let's jump into the year nineteen eighties. That's when the expert system started. So AI that mimicked human decision making with the if then logic. Thanks to faster microprocessors and early computers, these systems could use this these systems were able to do useful work. Like Mycin, it's a medical expert systems.

It was able to diagnose infections, but with the if then logic. But still, no learning, just smart rule following. Let's go to the year nineteen nineties where our machine learning arrives. This decade was a turning point. Computers started learning from data, not just following rules like the year nineteen eighties or nineteen fifties. Machine learning models like decision trees, SPMs became popular. Does it ring a bell? It's been a long time since we started hearing decision trees or SPMs because now we are all about LLM, AI training, and everything. Right? So nineteen nineties is when they learn patterns over time instead of hardcoded rules. This made things like spam filters. Did they think junk email is possible? That's that that's all the that was all happening in the year nineteen nineties. Let's go to the year February. Internet. Yes. AI got a boost from the Internet.

Huge datasets were now available. And with cloud computing starting to grow, models could be trained across multiple machines. A great example is the early versions of translate, which use statistical models to guess the best translation. Next comes the year February, deep learning breakthroughs. So this is when AI really started making headlines, Thanks to the powerful GPUs originally designed for gaming, deep learning took off with these GPUs. We saw the breakthroughs of face recognition and then speech language. AI was able to see and hear in a meaningful way thanks to the GPUs designed for gaming. And now we are here in the present, twenty twenties. AI went mainstream today with foundation models like Chargeberry. AI can understand and generate human language, code, images, and much more.

Specialized hardwares, such as TPUs and AI chips in mobile phones, have made real time AI accessible to almost everyone. So what's beyond this? Looking ahead, we are heading towards AI that can learn over time, plan, and reason almost like a human. But to make that leap, we will need next generation hardware, quantum computers, and brain inspired chips. The hardware has to catch up to this ambition. So it took almost eighty years from where it started to be here to move from labs to living rooms, what seems sudden is actually the result of decades of quiet progress powered by better, faster, and smarter machines at every step as you can see. AI evolved, so did hardware. Let's see the connection with a simple example of modern AI infrastructure and where hardware actually relates to every step of modern AI infrastructure. Of course, artificial intelligence does not exist in a vacuum. It's built on physical machines.

Every component in the AI infrastructure leans on hardware to do its job, from data storage to real time responses. Let's start with the first block here, data. Data needs a place to live. Large hard drives, solid state drives, or Cloud servers hold this vast amount of information. Fast storage means it can easily find what it needs without delay. So we get low latency and then get our answers as soon as possible. That's because data has a place to live, which is hardware. And let's move on to compute. Compute in really generic terms means training and running AI. This is really compute heavy. GPUs, TPUs, they are like supercharged engines built to perform thousands of calculations in parallel, much faster than regular CPUs. So these GPUs and TPUs coexist if compute have to happen. Let's also move on to frameworks. AI software frameworks are designed to use hardware efficiently.

For instance, let's think about TensorFlow. They can automatically spread tasks across multiple GPUs, making training faster and cheaper. Modern AI models, like transformers, rely on processing many pieces of data simultaneously, something that's only possible with hardware that supports that supports parallel computation like GPUs. And training. Training run across many machines working together. Data centers are filled with racks of powerful servers connected with high speed networks. This distributed hardware speeds up the learning process. Once trained, AI model lives on Cloud servers and small edge devices like our phones, smart speakers, and even wearables. These edge devices have specialized chips to run AI without needing the constant Internet access. Finally, AI, of course, needs ongoing care. You have to check if it's still accurate, fair, and most importantly, unbiased. Monitoring tools can help catch issues before they become big problems.

AI health checks are really important, and they run on servers that continuously gather metrics. This hardware enables engineers to catch the issues early and keep AI reliable. So overall, AI is the software brain brain and hardware is the physical foundation. And without physical foundation, our modern artificial intelligence infrastructure will not exist. But just like every other technology, the AI also has some major roadblocks. And as AI is scaling in the pace it's scaling today, so do the challenges. They span across technical, operational, governance layers. Let me break this down. Sourcing is a first hurdle. High performance GPUs and TPUs are in massive demand, but supply is tightly controlled. Getting the exact hardware needed on time is a growing challenge. And in terms of supply chain, much of the world's fabrication is localized in certain regions. That means a delay or disruption in one area can affect deployment globally.

And to that, the logistics of shipping and component dependencies, it's a multilayered puzzle. And performance wise, keeping system cool, efficient, fully utilized, request careful planning, or provisioning can lead to cost overruns while underutilization waste the capacity. It's like you not utilizing hardware but spending so much money on it. And moving on to governance, sustainability is now a core concern. Learning running large scale AI models consume vast amounts of energy. We are all under pressure to reduce emissions, increase energy efficiency, and adopt cleaner power resources. Scaling AI infrastructure is as much about facilities management and software strategy as it is about hardware specifications. Like any breakthrough technology, moving from lab to deployment, let's say electricity, let's say Internet, AI also is facing scaling challenges.

These are not problems unique to any one company, but shared industry frontiers that are driving a way of infrastructure innovation. The good news, these challenges are now being understood, and many in innovations we'll discuss in the next slide are directly targeting them. AI infrastructure is evolving rapidly, which we covered for over eighty years now what happened. But there are known industry wide constraints from chip availability to sustainability. Fortunately, innovation across hardware, software, and deployment models is targeting these challenges head on. Like I mentioned and like any transformative wave, electricity, the Internet, the smartphones, AI brings both promise and also challenges. And just like those earlier revolutions, the problems are being met with real time innovations.

AI infrastructure, as I was mentioning since the start of the session, is evolving rapidly and hardware is no longer general purpose. It's being purpose built. We are seeing unified memory designs and matrix optimized chip architectures, reducing training time dramatically. Low power accelerators are shrinking AI from Cloud data centers down to the edge. Let's think about powering cameras, sensors, even wearables, like I mentioned. Looking ahead, we will move toward more modular, efficient models, faster to train, easier to go on, and we'll get smarter and smaller. We will see it work more privately and locally. Unlike electricity or the Internet, it will quietly become part of everything, powering decisions, systems, and everyday experiences, often without us even noticing.

It's important to recognize that AI progress is inseparable from the hardware advancements. From high bandwidth interconnects to dedicated AI course, these innovations don't just make AI faster, They unlock entirely new use cases. Hardware does not just support AI. It defines what's possible. And that's the message I wanna give through my session here. Just wanna uncover what's in the background of AI and its infrastructure. Thank you.