0:0016:29

Adversarial attacks and defense mechanisms against deep point processes

Samira Khorshidi 16:29 2022

Noise you cannot see can make a model miss a stop sign

Two stop signs look identical to people, yet a sliver of invisible noise is enough to convince a model that one of them is something else. Samira Khorshidi, a computer science PhD candidate researching trustworthy machine learning at IUPUI, studies the same weakness in deep models that forecast crime, disease spread and viral content. If those forecasts can be fooled or shaken by shocks like a pandemic, how do you make them robust?

Learn how point processes work, how adversarial attacks exploit them and which defenses help

Watch on YouTube

In short

Samira Khorshidi introduces her PhD research on deep learning models of point processes, the event patterns behind crime, disease spread and viral content, and on how adversarial attacks and natural shocks such as a pandemic affect them. She argues that studying attacks and defenses is central to trustworthy AI and outlines defenses such as adversarial training, alternative architectures and generative adversarial networks, while flagging the environmental cost of ever larger models.

“And I'm saying that because I believe that machine learning models should be like trustworthy and we should have a like a machine learning model that is explainable responsible for the decision making and also safe and robust.”
“And it's important because like the whole field the point processes is hard to understand itself and then when it comes to deep learning models, there is always the risk of adversarial attacks, regardless regardless of the performance and accuracy of these models.”

What you will learn

  • Explain what point processes are and where they are used, from criminology to epidemiology and natural disasters.
  • Recognize how perturbations invisible to humans can mislead deep learning models.
  • Compare defenses such as adversarial training, alternative architectures and generative adversarial networks.
  • Distinguish attack-specific from attack-agnostic robustness when evaluating a model.
  • Weigh data noise, overfitting risk and environmental cost when building larger models.

Key takeaways from the speaker

  • Adversarial attacks and defense mechanisms against deep point processes are a matter of trustworthy ai and machine learning.
  • Adversarial Attack is an important step towards understanding the deep learning models

Details

Level
Intermediate
Format
Talk
Technical depth
Introductory technical
For
Student, Early career, Senior individual contributor
Length
16:29
Published
July 2022

Questions this talk answers

What is a point process?

It describes how a pattern of points in time or space might have been generated, explaining the cause behind the arrangement. Point processes can be deterministic or stochastic, like crimes, disease spread and wildfires.

What is an adversarial attack on a machine learning model?

It is an input with a perturbation humans cannot perceive that still misleads the model. In the speaker's example, tiny noise makes a model misread a stop sign.

How can deep learning models be made more robust to adversarial attacks?

Options include adversarial training with adversarial samples, alternative architectures such as convolutional instead of dense layers, and generative adversarial networks. Teams should state whether robustness is attack-specific or attack-agnostic.

Why does the nature of the data matter for robustness?

Random, noisy event data makes deep learning models prone to overfitting. Practitioners need to measure the uncertainty and sensitivity of their models.

What cost of large deep learning models is often overlooked?

The CO2 produced by big companies' data centers and the effect of their deep learning models on the global climate.

About this talk

Adversarial attacks and defense mechanisms against deep point processes are a matter of trustworthy ai and machine learning. In this research, accuracy, cybersecurity, and trust come together. More recently, neural network-based approaches to modeling point processes have received attention from the research community. These methods attempt to learn non-parametrically key components of point processes and their intensity to capture real event patterns better than parametric models. However, deep learning approaches run the risk of overparameterizing models and overfitting noisy real-world data. In addition, there is a lack of research into how robust such models are to natural shocks to systems, e.g., how a pandemic impacts deep point process forecasts of crime and adversarial attacks. My research focuses on modeling stochastic point processes using deep learning approaches and examining their robustness against natural shocks and adversarial attacks.

Topics

Related talks

Browse the library