0:0024:24

Computer Vision for Augmented Reality & Virtual Reality

Tilottama Goswami 24:24 2023

Without computer vision, augmented and virtual reality cannot exist

A digital ring lands on a finger the camera has just located, a dinosaur appears in a garden, and a treadmill tilts as an on-screen guide leads the climb, yet every effect depends on a machine first seeing the world. Computer vision researcher Professor Tilottama Goswami, with 23 years across academia, research and industry, unpacks what sits underneath. Which vision tasks must run before any digital object can sit convincingly in a real scene?

Learn the four types of AR, the vision tasks behind them and the tools to start building

Watch on YouTube

In short

Tilottama Goswami, Professor at Vasavi College of Engineering, gives a broad introduction to how computer vision underpins augmented and virtual reality. She defines the metaverse, VR and the four types of AR, walks through the computer vision tasks and SLAM that make immersive content possible, and shows applications and development tools, arguing that AR and VR cannot exist without computer vision.

“After that, I'll provide the list of the fruits. So the whole concept of the computer vision is see, observe, understand, and then act.”
“Now coming to the augmented reality, it is basically your real environment and the digital content is overlaid at the top of it to get an interactive experience. So it is augmenting the real world.”

What you will learn

  • Distinguish the metaverse, virtual reality, augmented reality and extended reality, and name the three pillars of presence.
  • Identify the four types of augmented reality: marker-based, markerless, projection-based and superimposition-based.
  • Map core computer vision tasks such as detection, tracking, segmentation and feature extraction to what AR and VR software must do.
  • Explain how SLAM lets markerless AR track a user continuously and build a virtual map of the environment.
  • Choose a starting point among common AR and VR development tools such as Unity, Vuforia, ARKit, ARCore, Wikitude and OpenCV.

Key takeaways from the speaker

  • The talk is about the emerging technology of the future
  • The session will discuss potential application domains of AR/VR Technology
  • The talk will focus on how Computer vision along with virtual and augmented reality work together to build a system for achieving sophisticated interactive content.
  • The session briefs about the AR/VR Development environment

Details

Level
Beginner
Format
Tech talk
Technical depth
Introductory technical
For
Student, Early career
Length
24:24
Published
June 2023

Questions this talk answers

What is the metaverse?

It is a virtual universe, beyond the universe, where humans and avatars coexist as a community. It rests on three pillars: the feeling of presence of oneself, of the space around you and of others.

What types of augmented reality are there?

Goswami describes four types: marker-based, markerless, projection-based and superimposition-based. She also shows a GPS location-based example from tourism.

Why does AR and VR depend on computer vision?

AR and VR software must first classify, detect, recognize, track, segment and extract features from a scene before any digital content can be placed. In her words, computer vision is the fundamental of it.

How does tracking differ between marker-based and markerless AR?

Marker-based AR tracks a fixed user location by looking for markers, while markerless AR continuously tracks the user wherever they go. With SLAM, that continuous tracking can also build a virtual map of the environment.

Which tools are used to develop AR and VR applications?

She names Unity and Vuforia as popular choices, ARKit from Apple, ARCore from Google, Wikitude, and OpenCV for the computer vision side.

About this talk

The objective of the talk "CV for AR/VR" is to give a broad overview of the role of computer vision in Augumented Reality and Virtual Reality. Various E-Commerce industries are paving ways towards mass adoption of AR and VR technologies to add the real-time visual effects to their products to provide better customer experience, improve brand perception and improve sales. Participants will learn the basic concepts of the technologies and the tools involved. The talk will focus on how Computer vision and virtual reality work together to build a system for achieving sophisticated interactive content.

Topics

Related talks

Browse the library