Research
Quanta Photography for Motion De-blurring
The above shows a frame of a video of a bursting balloon. The raw data consists of about 1000 photons in 50 frames. Our method is a combination of a change-point event detection and a burst photography motion correction. It is able to reconstruct the balloon on the right
Single-photon avalanche diodes (SPADs) are a rapidly developing image sensing technology with extreme lowlight sensitivity and picosecond timing resolution. These unique capabilities have enabled SPADs to be used in applications like LiDAR, non-line-of-sight imaging and fluorescence microscopy that require imaging in photon-starved scenarios. In this work we harness these capabilities for dealing with motion blur in a passive imaging setting in low illumination conditions. Our key insight is that the data captured by a SPAD array camera can be represented as a 3D spatio-temporal tensor of photon detection events which can be integrated along arbitrary spatio-temporal trajectories with dynamically varying integration windows, depending on scene motion. We propose an algorithm that estimates pixel motion from photon timestamp data and dynamically adapts the integration windows to minimize motion blur. Our simulation results show the applicability of this algorithm to a variety of motion profiles including translation, rotation and local object motion. We also demonstrate the real-world feasibility of our method on data captured using a 32 ×32 SPAD camera.
NLOS
What is Non-Line-of-Sight Imaging?
Single photon cameras are fast enough to image the motion of light. This allows us to analyze light transport in way that are not possible with an ordinary camera. By measuring and inverting multibounce light transport in a scene, we can for example reconstruct images of scenes from indirect light after it has reflected off relay surfaces. In other words: Our NLOS imaging system can project a virtual camera onto any surface. Our reconstruction shows us what this virtual projected camera sees. This makes it possible to see into rooms through windows, into caves, or around obstacles on the road
Non-Line-of-Sight Imaging using Phasor Field Virtual Wave Optics
Single-Photon 3D Imaging
Single-photon avalanche diodes (SPADs) are an emerging image sensor technology. Due to their unique ability to precisely time-tag individual photons, SPADs have great potential for high-resolution long-range LiDAR systems. We address challenges that arrise when SPAD pixels face strong ambient background light.
We also address the important challenge of SPAD data compression. SPAD pixels are capable of capturing large amounts of raw data. To make high resolution SPAD LiDAR systems with millions of pixels possible, this data needs to be compressed before it can be transferred off the camera chip for storage or processing.
Read more
Passive Inter-Photon Imaging
Digital camera pixels capture images by converting incident light energy into an analog electrical current, and then digitizing it into a binary representation. This direct measurement method, while conceptually simple, suffers from limited dynamic range: electronic noise dominates under low illumination, and the pixel’s finite full-well capacity causes saturation under bright illumination. Here we show that inter-photon timing information captured by a dead-time-limited single-photon detector can be used to estimate scene intensity over a much higher range of brightness levels. We experimentally demonstrate imaging scenes with a dynamic range of over ten million to one. The proposed techniques, aided by the emergence of single-photon sensors such as single-photon avalanche diodes (SPADs) with picosecond timing resolution, will have implications for a wide range of imaging applications: robotics, consumer photography, astronomy, microscopy and biomedical imaging.
Read more
Surgical Fluorescence Imaging
Medical vision problems provide exception challenges to our cameras and algorithms. We are developing methods that utilize single photon cameras for fluorescence imaging in fluorescence guided surgery, fluorescence lifetime imaging, and other types of radiation detection. Our imaging systems have to deal with ambient light, motion, and weak signals present in the operating room and our demonstrated methods are ideal for compensating these effects.
The application of advanced imaging techniques outside the laboratory is often complicated by background light. These techniques, such as fluorescence imaging require a dark environment, which is problematic in many industrial and medical applications. With transient lighting, it is possible to capture images on systems with light sensitive detectors and cameras while maintaining normal room illumination. This is achieved through quick switching light sources and detectors to provide time separation for multiple detectors and illumination in a common space. We have demonstrated this method on a two photon microscope and are currently developing systems for fluorescence guided surgery.
Computational Radiation Cameras
Detecting X-Rays, Protons, and Neutrons can be achieved with scintillators that absorb the particle and emit fluorescence light that can be detected. We are investigating novel detector setups using advanced scintillator materials and single photon cameras for improved radiation detection for applications in nuclear nonproliferation and medical imaging.