Not every network alarm is an anomaly, and Orange's models must tell which
Orange's network equipment produces a few terabytes of data per country each month, and the anomalies that signal trouble are rare and come with no labels. Orange Innovation data scientist Marzena Ołubek, named among Poland's TOP 100 Women in AI in 2022, shows how her team built unsupervised detection on Orange Romania's alarm data. So which of five models catches the anomalies that matter?
See how ARIMA, isolation forest, LOF, Poisson and autoencoders compare on real telco alarms
Marzena Ołubek, a Data Scientist at Orange Innovation, explains how Orange uses unsupervised machine learning to detect anomalies in the volume of network alarms, using fault management data from Orange Romania. She walks through feature engineering, compares ARIMA, isolation forest, local outlier factor, a Poisson model and an autoencoder, and shows how the use case runs as Vertex AI pipelines on Google Cloud Platform.
“So, at first, think about when somebody calls you and your call drops, or when your internet is broken down. I think it's very frustrating, even for me, so I think for you also.”
“Because not each alarm is an anomaly. We are focusing on anomalies in volume.”
What you will learn
Explain why rare events, class imbalance and missing labels push network anomaly detection toward unsupervised methods.
Engineer model-ready features from raw alarms by filtering per alarm type, splitting into time windows and aggregating per network level.
Detect anomalies by comparing a model's predicted alarm volume with real values against a threshold.
Choose between ARIMA, isolation forest, local outlier factor, a Poisson model and an autoencoder based on whether data is continuous or sparse.
Map an end-to-end anomaly detection workflow onto Google Cloud with Cloud Storage, Vertex AI pipelines, Artifact Registry, BigQuery and Looker.
Key takeaways from the speaker
Data-driven transformation with 3 pillars: AI + Automation + Cloudification of Network data
Anomaly detection is a key functionality feeding the root cause and the resolution
GCP as a reference platform for data pipeline development
Details
Level
Intermediate
Format
Tech talk
Technical depth
Practitioner
For
Early career, Mid-career, Senior individual contributor
Length
24:09
Published
May 2023
Questions this talk answers
Why does Orange use unsupervised machine learning to find network anomalies?
Anomalies occur very rarely, creating real class imbalance, and when they do occur the data has no labels, so supervised training is not possible.
How are raw network alarms turned into input for the models?
Alarms are filtered by type, such as power alarms, partitioned into time windows like three minutes, and counted per site, region and OSS level to form the model's input table.
How does a model decide that a point is an anomaly?
The model predicts the most probable alarm volume from past data, the difference between prediction and real value is calculated, and points exceeding a set threshold are flagged as outliers.
Which anomaly detection model worked best?
All models caught the highest peaks, but only ARIMA caught the lowest ones, the Poisson model fitted sparse data best, isolation forest flagged too many anomalies and needs tuning, and local outlier factor did well on both continuous and sparse data.
How is the use case deployed on Google Cloud Platform?
Input data sits in Cloud Storage, a Vertex AI training pipeline trains a model registered in Artifact Registry, an inference pipeline stores results in BigQuery, and Looker dashboards visualize them.
About this talk
Have you ever wondered how AI is used in the telco network area?
Orange, as data-driven and AI-powered telco company, place Data and AI at the heart of our innovations for Smarter Networks. In that case, network management is supported by Machine Learning models for detecting anomalies based on different types of network data. During this presentation we will zoom on Predictive Network Maintenance based on network data. Our aim is to shorten root cause analysis with Explainable AI to improve network experts daily job.
Good morning, everybody. My name is Marzena Ołubek. I'm from Orange Innovation, Data and AI, and today I will tell you about the anomaly detection in network based on GCP Vertex AI. So, at first, think about when somebody calls you and your call drops, or when your internet is broken down. I think it's very frustrating, even for me, so I think for you also.
So, today I'm going to tell you how Orange is dealing with this network malfunction. First of all, I would like to say about the agenda. So, at first, I will say a few words about Orange Innovation, Data and AI, so my department when I'm working on, and then I will show you my use case. So, anomaly detection in the network, and then I will show you how this use case was developed on Google Cloud Platform.
So, starting from the first, why we need in Orange Data and AI. First of all, our mission is making people's everyday lives better better through AI. It means that we are helping our customers and our colleagues, for example, network experts, in their everyday life, because we are preventing the network malfunction, and we are providing products and services within the machine learning and artificial intelligence, solutions, in order to faster, in order to faster, faster, faster, faster, prevent the network malfunction. So, our goal is for Orange to became data and AI driven, with AI at scale. It means that we are focusing on solutions that can be replicable.
For other countries. Our ambition is to place data and AI at the heart of our innovation model. So, what is our innovation model? It has three, four strategic pillars for data and AI. First of all, it's smarter networks. So, we are building building network that should be adaptive, efficient, sustainable, and very secure with AI. So, it should be smart. Another strategic pillar for us is reinvented customer experience and management, where, for example, we are doing models with propensity to buy, that should be open city to buy, churn reduction, and other customer journey with AI. We are also using data and AI for greater operational efficiency, to be more profitable with AI. And all those three pillars must be within responsible and sustainable AI. So, we are using data and we are using data and data. So, we are using data and AI. So, in data and AI, now we will focus on smarter networks. So, in that pillar, we are focusing on network data. So, what are the network data? Network data are those data that
that comes from network equipment. So, what are the biggest challenges in network data? So, what are the biggest challenges in network data? First of all, this is a very big data because we are we are dealing with few terabytes. We are dealing with few terabytes data per one country per month. So, it's a very huge amount of data. It's a very big data. They are coming from different countries. So, we have different methodologies for aggregation data.
And those network data. So, we have different methodologies for aggregation data. So, we have to deal with that problem. And the most important challenge is to detect anomalies. And those anomalies occurs relatively rarely. It means that we have a real class imbalance. So, it's very hard to define those anomalies because they are very, very, very rarely. And moreover, if they occur, in our data, we don't have labels. So, we are focusing on unsupervised methods of machine learning.
So, we are focusing on unsupervised methods of machine learning because of the lack of the labels. So, this is the word of network data. So, what we are focusing on unsupervised methods of machine learning. So, what can we do with those data? So, what we are focusing on unsupervised methods of machine learning? So, I will present you my use case where I'm working on in orange. So, anomaly detection based on alarms and volumetric.
What does it mean? So, those orange ones are systems that are on premise in orange. And this blue one is Google one because we have a partnership with Google. So, first of all, we have network equipments that produce the data, network data. Which are gathered and managed in data management layer. And those data, this is our source of the data into AI ML based anomaly detection systems. And those solutions are built in Google Cloud platform. So, in this blue one. And those solutions are built in Google Cloud platform. So, in this blue one.
So, the output of machine learning models that are detecting anomalies are pushed into the incident management system that are triggering the ticketing system. And this all, this process is done automatically. So, our goal is to enrich this incident management system. So, what we are starting with data. So, what are the data. Data are coming from some kind of umbrella. So, the data warehouse.
So, what we are starting with data. So, what we are starting with data. So, what we are starting with the data. What are the data. Data are coming from some kind of umbrella. So, the data warehouse. So, what we are starting with data. So, what we are starting with data. So, what we are starting with data. What are the data. Data are coming from some kind of umbrella. So, the data warehouse.
The data source that is managing the network data. It's fault management umbrella. And those data in this use case are coming from Orange Romania. And we are focusing on alarms. And we are dividing those alarms into different type of alarms. For example, power alarms. It's one of the type. And we are focusing on alarms. And we are dividing those alarms into different type of alarms. For example, power alarms. It's one of the type.
And so, this is our input. And so, this is our input of the data. And our goal is to detect anomalies in volume of alarms. Because not each alarm is an anomaly. We are focusing on anomalies in volume. So, for example, detecting the peaks of the alarms. So, how we are doing it. How we are doing it. First of all, we are starting with a feature engineering. What does it mean? That we are, first of all, filtering the data per type of the alarm. So, let's focus on one type of the alarms. So, power alarms.
And it will be those orange dots. So, we are filtering those power alarms. So, at the second step, we are partitioning data. So, splitting those data, those alarms, by time. For example, into three minutes time window. So, in the third step. So, in the third step. We are aggregating those data per levels of network. Because we have few levels of network equipments. Very low. So, each site.
So, each site. Site is the lowest type of network equipment. So, BTS. Based Telco Station. And this is the site. Region. And this is the region. Geographical region. In Romania. And there are 44 regions. And OSS. There are three or four OSS. There are three or four OSS regions in Orange Romania. So, we are then counting those alarms. Within the time window. This is the three minute time window. And per each site. Each region. Each OSS.
So we are having some kind of alarm scanner. And this table is an input to our model. So how we are doing those data modeling? First of all, what is our goal? It's to detect anomaly. So what is the anomaly? Anomaly is some kind of an outlier in data. So it is defined as an observation or subset of observation, which appears to be inconsistent with the rest of the data.
So we are looking for non-natural behavior of the data, which is not the same as it was before. And how we are detecting those anomalies. We are fitting data to machine learning models, which models I will tell you in a minute. So we are predicting the volume of the alarm. So the most probable volume of the alarm is based on the data that was in the past.
And then we are calculating, for example, the difference between the model output and the real data. So when the model is well fitted, when the data are well fitted to the model and the predicted value is the same as it was in the real. So there is no anomaly. But when we are predicting the value, which should be not anomalous value and the real data is far, far higher.
So it's supposed to be an anomaly because we are calculating the difference between the model output and the real data. After setting a threshold, we detect the points that are exceeding those thresholds as outliers. And here we have some few methods that we are using in our use case to detect anomalies. First of all is the time series forecasting method, so ARIMA, autoregressive integrated moving average.
And it is very, it was very good for the continuous data. But we also tried another. Models. Based, for example, on decision tree, which is isolation forest. So it's building the decision trees based on the one feature and. Dividing. those elements that are. Those points in a tree that are far, far away. So the first points of the tree seems to be the most anomalous.
We also tried the density based. Models like local outlier factor. And. Some statistical approach based on the Poisson distribution because of the. Because the Poisson distribution is Poisson law. Seems to be. estimating the. The. The rare events and our alarms are very, very rare events. That's why we. We're trying the Poisson law. Calculating the probability of the.
Rare events. And at the end, we also tried some neural networks. So autoencoder. This is the. The model that. The structure. The. The architecture of this. Neural networks. Which. Decodes. And encodes. And then decodes the data. And those data that have the. Higher reconstruction errors. Are seems to be. The most anomalous data. So. Those are the results.
Of. Of our research. As you can see. On. On the graphs. Sorry. All those models. Are. Detecting the highest peaks. The Arima. Poisson. Isolation forest. Or local. Applier factor. But for example. Only Arima. Is detecting. The lowest anomalies. It's also important. Because for example. When the. Network equipments. Are not. Producing any alarms. It also can be an anomaly.
So. We are looking for the. Highest peaks. But. For some kind of the network. Equipments. It's very important. To have a look also. For the. Lowest peaks. That's why Arima. Is good for it. Poisson model. Was. The best. For. A very sparse data. So. For example. We have some. Kind of a traffic. Alarms. That were more sparse.
Than power. And. In that situation. Poisson law. Was. Better. Fitted. To the. Poisson. Distribution. Isolation forest. Was. Detecting. Much. So. Anomalies. That. It. May. Need. To. Hypertune. This model. To lower. The threshold. And. Locale. Flyer. Factor. Was. Have. Nice. Results. Even. For. More. Continuous. And. Sparse. Data. So. To sum up. Different. Machine learning. Models. Output. Results. All.
For those. Models. Detects. At the highest peaks. But. Really. To. Validate. Those. Models. We. Can. Do it. After. The. Feedback. Loop. Annotations. Are. The. Information. From. For example. Network. Expert. That. Will. Confirm. Or. Not. If. The. Anomaly. Detected. By. Model. Was. True. Or. Not. So. This. Is. The. Important. Step. To. Check. If. Those. Models. To. Tune. Yes.
And. How. We. Have. Develop. Deployed. Our. Use. Case. On. Google. Cloud. Platform. And. First. Of. All. We. Have. An. Input. Data. It's. Stored. In. Cloud. Storage. Google. And. Then. We. Are. Pre-processing. The. Data. And. Training. Model. This. Is. The. Training. Pipeline. In. Vertex. AI. The. Model. When. It's. Pre-trained. Saved. And. Registered. In. Artifact. Registry. And. So. This.
Is. The. First. Pipeline. And. Here. But. We. Can. Have. Also. Inference. Pipeline. So. We. Are. Calculating. Predicting. The. New. Values. Based. On. The. Pre-trained. Model. And. Then. We. Are. Storing. Our. Results. In. BigQuery. And. We. Also. Have. A. Dashboard. In. Data. Looker. To. Visualize. The. Data. All. Pipelines. I. Managed. In. Google. Cloud. Platform. A. Few. Words. About.
Those. Pipelines. Helps. To. Automatically. Train. And. Predict. Our. Machine. Learning. Models. This. Is. The. End. To. Process. Starting. From. Loading. The. Data. Feature. Engineering. Training. Predicting. And. Saving. Results. So. All. The. Steps. Are. Done. Step. By. And. It's. Easy. Tool. For. Data. Scientists. We. Can. Have. Also. A. Dashboard. This. Is. Done. In. Data. Looker. So. We. Have.
Separate. Dashboard. Per. Alarm. Type. Model. Because. We. Have. A few. Models. Per. Dimension. Meets. Those. Network. Like. Site. Name. Or. The. Highest. Level. Of. Network. Equipment. And. The. Per. Time. Window. For. Example. And we can have three minutes time window when we are counting those alarms and or do the experiment with any other time window. What are the most important facts I want you to remember from my speech is that Orange is a date and the eye driven company because they tend the eyes in our heart of the innovation.
And. With our partnership with Google, we are building a smarter network using the cloud based solutions. So we are using BigQuery as a big data tool and we are building pipelines in Vertex AI and those pipelines, I mean machine learning pipelines. As. And we have and we are focusing on unsupervised machine learning models for anomaly detection as we don't have or have so much, not so much labors in our data.
So thank you for listening to me. If you have any questions, don't hesitate to to contact me on LinkedIn or or here now on on chat. Thank you. We'll see you next time.
Transcribed automatically; names checked against the speaker record. Select a time to jump there.
Article based on this talk
How do you detect network anomalies when nothing is labelled?
Marzena Ołubek explains how Orange turns raw network alarms into unsupervised anomaly detection running on Google Cloud.
In short
Network anomalies at Orange are rare, buried in terabytes of data and almost never labelled, so the company relies on unsupervised machine learning to find them. Marzena Ołubek, a Data Scientist at Orange Innovation, shows how her team turns fault management alarms from Orange Romania into an anomaly detection workflow, compares five modelling approaches and deploys the whole flow on Google Cloud Platform with Vertex AI.
Key takeaways
Data-driven transformation rests on three pillars: AI, automation and cloudification of network data.
Anomaly detection is a key functionality feeding root cause analysis and resolution.
GCP serves as a reference platform for data pipeline development.
Why a telco puts AI at the heart of the network
Ołubek opens with a familiar frustration: a dropped call or a broken internet connection. Her department, Orange Innovation Data and AI, exists to prevent those malfunctions, with a mission of making people's everyday lives better through AI, for customers and for colleagues such as network experts. She describes Orange's goal of becoming data and AI driven with AI at scale, which in practice means building solutions that can travel across the group's countries.
"It means that we are focusing on solutions that can be replicable." (1:21)
She lists the strategic pillars of Orange's data and AI innovation model: smarter networks that are adaptive, efficient, sustainable and secure; reinvented customer experience, including propensity-to-buy and churn reduction models; and greater operational efficiency. All of them, she stresses, sit inside responsible and sustainable AI. The rest of the talk zooms in on smarter networks.
The trouble with network data
Network data comes straight from network equipment, and Ołubek names its challenges plainly. Volumes are huge: a few terabytes per country per month, as she cited. Data arrives from different countries with different aggregation methodologies. Most importantly, the anomalies that matter are rare.
"It means that we have a real class imbalance." (5:15)
Even when anomalies do appear, no labels come with them, which is why the team works with unsupervised methods. The architecture she presents connects on-premise Orange systems with Google Cloud: network equipment produces data, a data management layer gathers it, AI and ML anomaly detection runs on Google Cloud Platform, and model outputs flow automatically into the incident management system that triggers tickets.
From raw alarms to a model-ready table
The use case draws on a fault management data source holding alarms from Orange Romania, divided into types such as power alarms. The target is not each individual alarm but unusual volumes, such as sudden peaks.
"Because not each alarm is an anomaly. We are focusing on anomalies in volume." (8:58)
Feature engineering follows three steps. First, filter alarms by type. Second, partition them by time, for example into three-minute windows. Third, aggregate them by network level: the site (the base station), the geographical region (44 in Romania, as she cited) and the OSS level (three or four regions). Counting alarms per window and per level produces the table the models consume.
Defining and catching an anomaly
Ołubek defines an anomaly as an outlier: an observation, or a subset of observations, inconsistent with the rest of the data.
"So we are looking for non-natural behavior of the data, which is not the same as it was before." (12:18)
The method is to fit a model that predicts the most probable alarm volume from past data, then compare that prediction with what actually happened. A small gap means no anomaly; a far higher real value signals one. "After setting a threshold, we detect the points that are exceeding those thresholds as outliers." (13:41)
The team tested five approaches: ARIMA time series forecasting, isolation forest, local outlier factor, a statistical model based on the Poisson distribution (suited to rare events) and an autoencoder neural network that flags data with high reconstruction error. The results she presented differ in useful ways. All models caught the highest peaks, but only ARIMA caught the lowest ones, which matters because equipment that stops producing alarms can also be anomalous. The Poisson model fitted sparse data, such as traffic alarms, best. Isolation forest flagged so many anomalies that it needs hyperparameter tuning, while local outlier factor gave good results on both continuous and sparse data.
Real validation, she explains, only comes through a feedback loop: network experts annotate whether each detected anomaly was true or not, and those annotations are then used to check and tune the models.
Running it on Google Cloud
The deployment stores input data in Google Cloud Storage. A Vertex AI training pipeline preprocesses the data and trains the model, which is saved and registered in Artifact Registry. A separate inference pipeline predicts new values from the trained model and writes results to BigQuery, and Looker dashboards break the output down per alarm type, model, network dimension and time window. Ołubek describes the pipelines as an easy tool for data scientists that automates the end-to-end process, from loading data and feature engineering to training, predicting and saving results.
"So we are using BigQuery as a big data tool and we are building pipelines in Vertex AI and those pipelines, I mean machine learning pipelines." (22:46)
Her closing message ties it together: Orange places data and AI at the heart of its innovation, builds smarter networks with cloud-based solutions through its partnership with Google, and leans on unsupervised models because labelled anomalies are scarce.
Watch the talk
AI for Anomaly detection in network area, by Marzena Ołubek (24 minutes): https://www.womentech.net/video/ai-anomaly-detection-in-network-area
About the speaker
Marzena Ołubek is a Data Scientist at Orange Innovation with more than 10 years of experience in data mining, analytics, modelling and machine learning, and an educational background in advanced econometrics. As an R&D expert she applies AI and ML to network management through predictive network maintenance, and in 2022 she was named among the TOP 100 Women in AI in Poland.
Questions this talk answers
Why does Orange use unsupervised machine learning to find network anomalies?
How are raw network alarms turned into input for the models?
How does a model decide that a point is an anomaly?
Which anomaly detection model worked best?
How is the use case deployed on Google Cloud Platform?
Drafted with AI from the talk transcript. Editor review pending.
Frameworks and ideas mentioned
ARIMA
Isolation Forest
Local Outlier Factor
Poisson distribution model
Autoencoder
Resources mentioned
Vertex AI pipelines
Cite this talk
Marzena Ołubek. "AI for Anomaly detection in network area." WomenTech Network, 2023. https://www.womentech.net/video/ai-anomaly-detection-in-network-area