0:0024:09

AI for Anomaly detection in network area

Marzena Ołubek 24:09 2023

Not every network alarm is an anomaly, and Orange's models must tell which

Orange's network equipment produces a few terabytes of data per country each month, and the anomalies that signal trouble are rare and come with no labels. Orange Innovation data scientist Marzena Ołubek, named among Poland's TOP 100 Women in AI in 2022, shows how her team built unsupervised detection on Orange Romania's alarm data. So which of five models catches the anomalies that matter?

See how ARIMA, isolation forest, LOF, Poisson and autoencoders compare on real telco alarms

Watch on YouTube

In short

Marzena Ołubek, a Data Scientist at Orange Innovation, explains how Orange uses unsupervised machine learning to detect anomalies in the volume of network alarms, using fault management data from Orange Romania. She walks through feature engineering, compares ARIMA, isolation forest, local outlier factor, a Poisson model and an autoencoder, and shows how the use case runs as Vertex AI pipelines on Google Cloud Platform.

“So, at first, think about when somebody calls you and your call drops, or when your internet is broken down. I think it's very frustrating, even for me, so I think for you also.”
“Because not each alarm is an anomaly. We are focusing on anomalies in volume.”

What you will learn

  • Explain why rare events, class imbalance and missing labels push network anomaly detection toward unsupervised methods.
  • Engineer model-ready features from raw alarms by filtering per alarm type, splitting into time windows and aggregating per network level.
  • Detect anomalies by comparing a model's predicted alarm volume with real values against a threshold.
  • Choose between ARIMA, isolation forest, local outlier factor, a Poisson model and an autoencoder based on whether data is continuous or sparse.
  • Map an end-to-end anomaly detection workflow onto Google Cloud with Cloud Storage, Vertex AI pipelines, Artifact Registry, BigQuery and Looker.

Key takeaways from the speaker

  • Data-driven transformation with 3 pillars: AI + Automation + Cloudification of Network data
  • Anomaly detection is a key functionality feeding the root cause and the resolution
  • GCP as a reference platform for data pipeline development

Details

Level
Intermediate
Format
Tech talk
Technical depth
Practitioner
For
Early career, Mid-career, Senior individual contributor
Length
24:09
Published
May 2023

Questions this talk answers

Why does Orange use unsupervised machine learning to find network anomalies?

Anomalies occur very rarely, creating real class imbalance, and when they do occur the data has no labels, so supervised training is not possible.

How are raw network alarms turned into input for the models?

Alarms are filtered by type, such as power alarms, partitioned into time windows like three minutes, and counted per site, region and OSS level to form the model's input table.

How does a model decide that a point is an anomaly?

The model predicts the most probable alarm volume from past data, the difference between prediction and real value is calculated, and points exceeding a set threshold are flagged as outliers.

Which anomaly detection model worked best?

All models caught the highest peaks, but only ARIMA caught the lowest ones, the Poisson model fitted sparse data best, isolation forest flagged too many anomalies and needs tuning, and local outlier factor did well on both continuous and sparse data.

How is the use case deployed on Google Cloud Platform?

Input data sits in Cloud Storage, a Vertex AI training pipeline trains a model registered in Artifact Registry, an inference pipeline stores results in BigQuery, and Looker dashboards visualize them.

About this talk

Have you ever wondered how AI is used in the telco network area? Orange, as data-driven and AI-powered telco company, place Data and AI at the heart of our innovations for Smarter Networks. In that case, network management is supported by Machine Learning models for detecting anomalies based on different types of network data. During this presentation we will zoom on Predictive Network Maintenance based on network data. Our aim is to shorten root cause analysis with Explainable AI to improve network experts daily job.

Topics

Related talks

Browse the library