The Journey of Data Science - How Did It Iterate the Way to AI

Ling Zhang
Data Science & AI Leader
Automatic Summary

2023: The Year of AI and its Transformative Power

In recent years, the advancements in Artificial Intelligence (AI) are transforming our world as we know it. As generators and consumers of technology, we are integrating AI into our lives more than ever before. Be it chatbots or smart home appliances, AI is not just revolutionizing the technology industry, but altering our understanding of professions and daily life.

Equipping ourselves with the knowledge and understanding of AI's journey led by data science is pivotal. Let us delve into this transition and also learn a bit about how AI has come to play a critical role in our lives.

Understanding the origin and evolution of AI

Having over 20 years of experience in data analytics and AI, I have contributed to AI and data science strategies for companies and built a data-driven model. My mission lies in empowering companies to make data-driven decisions and excel through technological advances in AI.

The Relationship Between AI and Data Science

At an elementary level, AI is a simulation of human intelligence by machines and computer systems. This involves learning, reasoning, decision-making, and problem-solving. Often confused with data science or machine learning, AI includes these areas while encompassing much more. To simplify - "A robot combines AI, machine learning, and hardware."

Five Stages of Data Science Evolution

As we dive deeper into AI's journey, there are five stages of the data science evolution to consider. More details on the same are as follows:

  1. Pillars of Fundamentals: Business problems, algorithmic approaches, data relevance and volume, computing power, and strategy.
  2. Progression of algorithms: From frequency-based methods in 1863 to AI and machine learning in recent times.
  3. AI waves: The evolution began with rule-based methods, followed by machine learning, deep learning, and is now in the era of reinforced learning.
  4. Levels of Analytics: Developing from no data or analysis to machine-driven decision-making.
  5. Data Science Evolution: From silo-based practices to automation based on theoretical foundations.

Current State and The Future of AI

This year has been deemed the "clean year" for AI and the world at large. Despite economic turbulences, AI's growth has been unstoppable. With the advent of large language models and AI being increasingly used in tools and applications, the scope of opportunities has substantially expanded.

A Look at the Opportunities

As AI continues to evolve and grow, opportunities in various sectors are coming to the fore. Our focus should not be limited to marketing, sales, or customer assistance. Optimizing operations, improving efficiencies, and enabling better decision-making processes are the keys to leveraging AI.

Preparing for the Future

AI's future looks incredibly promising if we position it for the benefit of humanity. AI is changing our everyday lives, redefining what it means to be human, and the very essence of work and life. As we gear up for this future, focusing on what makes us unique as humans — like our emotions, passions, and curiosity — remain of utmost importance.

Key Takeaways

For those working in data science or AI, the following lessons are crucial:

  • Lay a solid foundation in data and algorithm technology
  • Start small and gradually build on your skills
  • Collaborate with others and learn from them
  • Seek partnerships and create synergy
  • When you become an expert, be supportive and mentor others

As AI's future takes flight, individuals working in the AI field should also expect career growth parallel to these technological advancements.


Video Transcription

And 2023 is, is a year of A I. We all know A I is transforming the world as both generators and the consumers of technology. We all interact with the chat box and you may have some smart toaster at home, robot cleaner.And also you may take a self driverless car to work someday. We may greet into Robert shake hands to them on the street. So A I is not only changing, almost every single profession is also changing our understanding. What work is A I is redefining every company and our daily life. So how did A I come to here today? My talk is really about the journey of data science. How did it it lead the way to A I before I jump to the campaign. First thing, let's have a brief introduction about myself. I have about over 20 years experience in data analytics, design data science. A I, my focus is really developed for A I data science strategies for companies build the data driven products, build the teams. I'm passionate about pe people, a leader with a spirit to consistently identify opportunities and the challenge assumption for universal I hold the two approved patent two books. One is co-authored with others in the data area. Another one help people in their personal growth. Grew to first in Korea.

My mission really help companies empower data in f the decisions a growth which optional excellence through the science of A I in life. I want to help people achieve their full stay in life in the career. Data science of A I. My favorite code is try not to become a main success but rather become a man of value. OK. This is today's outline we start from with what is A I. The focus is really five stages of data science evolution and the current state of A I lessons were learned. So what is A I at a very high level A I is the simulation of human intelligence by machines and the computer system that include the learning, reading self collection. We all know A I is a multidiscipline field. So here is the ta to share what's the relationship A I with computer science, data science and machine learning. So what is the robot in my simple formula? I would like to say robot is put A I machine learning in the center, data science hardware. What does a robot do? Basically, they collect the consuming information like human, they learn reasoning, automated process decision making, even communicated with machines and people. So this is the five stages. Personally, I think uh data science evolution have gone through. Let's start with the six pillars of fundamentals.

This include business problems. We know everyone facing problems in life. So the business problem may be from simple to complicated we so one followed another. How can we let machine help us? This is the means how can we use red algorithm with red assumption and conditions to formula the B problem? So we also need the data, data needed to complete relate. This is a very popular fourways volume veracity velocity veracity. Also we also need computing power to process massive data that comes to distribute the parallel computing to the current quantum computing. The tragedy is really try to develop systems by leverage data theory technology altogether to solve a unique business problem. Build the product and service last but not least is people. People is on the driver seat on each of these pillar. People collaborated each other for innovation. So let's zoom in for the algorithm B is a problem. We started very early back to the 1863 probability based method.

Really one method, one learning one England physician actually using simple frequency count identified the location Chola disease outbreak in 1854 at London. Later on we use the linear method to nonlinear. Also numerical forecast is one application. It's come to assembled method assembled method basically is the same method.

But through multi iterative learning by focus giving the last snapshot of data such as grounding boosting random forest. Also around the 2000 people think about how can that machines reading like human that's come to much layered learning neural using neural network then come to now is standard to A I. Basically it's yes machines interact with humans is not only one method, multiple method, multiple iteration and also many one time learning a big model provide the money. So if we look at this, it's really from simple pattern detection comes to very human way to solve, to build the product to solve the problem. So if we look at the speed development, very beginning is very slow, then come 20 year, eight year, six year. So let's talk about the four A I waves at the very beginning is rule based method. Basically humans include the rules hard coded, the less the machine to things thus come to machine learning automated that using deep learning currently is reinforced learning, put human in the loop, humans can teach machines to become smarter.

So basically the evolution begins with the baby steps, the crawling then progress is flame. So the journey of 1000 miles begins with one step. Now let's look at the file level analytics. What's the relationship, the analytics with data and the decision. So we started with the very beginning.

No data, no analysis, purely human gut fail basic decision. Then we use a small data identify what happened. That dashboard based approach then comes to OK. We want to know why it happened that's root cut analysis. Basically we do A B testing is clear. This level three is really part of strategic planning. We need to understand what will happen. That's really model fitting start some level four is basically decision support. But otherwise what's the best solution we use simulation to analysis. The approach currently is really decision in intelligence machines do the self learning leverage cognitive computing to upgrade all these kind of steps together to the automated decision making. So from database perspective, we go through subtracted the database to data warehouse data lake to data lake house, we started from monitoring to data governance and currently observe viability understanding n gritty detail what happened to the data. So the decisions we already mentioned as A B at the very beginning, no inside, no data that the pure human. So we leverage in inside gradually from less inside the more human intinction to become less human instinct, instinct, the more insight that analytic come to A I humans working together when the machines become smarter, actually humans become also smarter.

And because humans make a machine becomes matter at a very high level. When we reach very high level analytics maturity ac should have a well established data character. Unlock the full potential of data. Everyone use data analytics to make decisions. It's not limited only a small part of people. The first stage actually, when we have this, after we laid the foundation theoretical, we gone through silo based practice. If we look at the data science project lefty involve eight components and the related business objective whether from people, data processing algorithm analytic processes, they are all everyone do their own thing. The challenge really comes slow, less efficient, over 85% product, they feel high cost, low profit how to solve this. This is come to stage three A I emerging modulation. Basically we try to do high level abstracting abstraction upgrade a similar function together to be reused such as data pipeline, data process feature engineering. And we pack all for the business use case with the different modules pipelines together to make it easier reuse accessible is the approach actually people in here involved in the data design, machine learning process, they collab each other, try to identify reusable template and some company are also build a center of science to communicate the best practice organized training program, create an internal marketplace with reusable template and guidelines.

Then we come to automation basically leverage the people process technology to infuse A I machine learning into business process. The automation process is really leverage the fallout. Basically it's studying first to build the product then data for the data pipeline and engineer and comes the machine learning part A I MLS model. It include this six component involved. So II I know I go through very fast because there are so many stuff. As a summary, this is the four stages of data science E the progress is not linear is zigzag but it's reached the point huge breakthrough. Next, I'm trying to talk about what's the current state? Let's focus, challenge changes, opportunities, future development. We study the GT for release.

This year is really grandly clean year for the world and A I. But still there are a lot of t because China GDP base is a large language model. The training is so expensive is over a million dollars, consuming a lot of resources and energy and their potential bias well known. And it's hard to explain. And the number misuse A I is also ring maybe because of this challenge us government actually invest more in A I research and the design about 2 billion. Another trend that is very interesting industry actually takes the lead to produce small machine learning models because it's easier to access real time data, computer power with enough money than academic. A lot of spot up, they come up to try to make A I more easier to understand transparent. The opportunity is also huge. I would like focus is really not only for companies, it's for the personals such as how can we use A I power, the automation to do all kinds of things.

Not only marketing sales, customer assistant, also think about how we are going to optimize all these kind of operations. There are a lot lot of innovation opportunities. The development will continue to improve the A I it used in every area such as a nose smartphones also help to make A I understand human emotions. So the F of A I is really looks amazing if we put a, if we use A I for the benefit of human put ha in the center. At the beginning, I said A I is redefining every company and our everyday life. The essence of what it means to be a human under the two meaning of work and life. I believe a beautiful future is waiting for us where we focus on what make us unique as humans, such as our vision, curiosity, our passion, our dreams, our emotion impedance. Finally, let's talk about what we learned that this like A I. If we work in data science A I career, we also need to lay a solid foundation to build our data and our algorithm technology foundations. Secondly, we need a collective siloed approach to build ourselves. Start with the under demand, beginning something easier. Third, learn to collaboration, be humble. You can't do everything. Stand on the shoulders of the band, learn from others, collaborated from others to adopt others approach with them. The first part is partnership with others.

Be proactive, leverage the power of create with them. The last part is when you become expertise, your success, multiply your success, be supportive, coach, mentor, others build a flying will leave Alexi. I think the future is if all of this, your future is also flying like a I right now. So this is my talk today. I go very fast. Thank you everyone. I'm not sure. Is there any question? Should I stop? Oh, I can't see any question here. Top Charity.