Bringing Computer Vision to Hidden Objects
A Tutorial on Millimeter-Wave Imaging and Reconstruction of Occluded Scenes
Millimeter-wave (mmWave) signals, such as those used in 5G, 6G, and next-generation WiFi, are a unique modality that can travel through many everyday occlusions (e.g. cardboard, fabric, fog, etc), allowing them to sense objects or scenes that are hidden from view. This unique capability has sparked recent interest in the computer vision community and beyond for using these signals to enable novel perception tasks with applications spanning autonomous driving, robotics, shipping and logistics, and more. The goal of this tutorial is to introduce audience members to this modality, and equip them with the knowledge needed to begin research in this area. We will cover both fundamental millimeter-wave imaging concepts, as well as recent, state-of-the-art methods. We will discuss different applications of millimeter-wave sensing, including a deep-dive into two areas: through-occlusion 3D object reconstruction, and all-weather scene understanding. We will additionally cover existing datasets, benchmarks, and tools so that audience members new to this area can begin research in this field. This tutorial is designed to be accessible for an audience with no prior millimeter-wave experience. Topics to be covered include:
Timeline
| Time | Details |
|---|---|
| 8:00 - 8:15 AM | Introduction |
| 8:15 - 9:15 AM | Basics of Millimeter-Wave Signals How They Differ From Visible Light and What They Can Do We will start by covering various applications of mmWave sensing, fundamental properties of mmWave signals, classical imaging approaches and limitations of these approaches. |
| 9:15 - 10:00 AM | Advanced Through-Occlusion Object Reconstruction Using Surface Normal Estimates for Through-Occlusion 3D Object Reconstruction and Completion Next, we will discuss a recent state-of-the-art mmWave reconstruction method, including how it opens many exciting directions for future work. |
| 10:00 - 10:15 AM | Coffee Break |
| 10:15 - 11:00 AM | Enabling Visual-Quality Scene Understanding in Poor Visibility We will discuss how mmWave signals can be used to reconstruct lidar-like scenes in poor visibility, and how this can be used to enable safer autonomous vehicles. |
| 11:00 - 11:45 AM | How to Begin Millimeter-Wave Research A Tour of Existing Datasets, Tools, and Open Problems We will conclue by discussing how researchers can start research with mmWave signals, discussing existing datasets, benchmarks, and tools they can use to get started. This will include a hands-on demonstration for setting up and using this tool. We will also discuss several different open problems in the field that researchers can begin . |
Assistant Professor
University of Pennsylvania
PhD Student
Massachusetts Institute of Technology
PhD Student
École Polytechnique Fédérale de Lausanne