Studying optimization for neuromorphic imaging and digital twins

Harbir Antil (PI), director of the Center for Mathematics and Artificial Intelligence (CMAI), professor of Mathematical Sciences, and Rainald Löhner (co-PI), director of Computational Fluid Dynamics Lab, professor of Physics and Astronomy, received funding from the Air Force Office of Scientific Research (AFOSR), under the prestigious DURIP program, to establish a neuromorphic imaging and digital twins lab with capabilities to design new optimization algorithms.

This project will set up the Neuromorphic Imaging and Digital Twins Lab—a first of its kind physical lab in the country under the Center for Mathematics and Artificial Intelligence (CMAI) at George Mason University. This lab will contain equipment for neuromorphic imaging, neuromorphic computing, and digital twins that will not only support research at Mason but support the needs of local organizations and beyond.

"Unlike traditional cameras that detect pixel intensity synchronously, neuromorphic sensors only detect 'changes' at pixels where a change is occurring asynchronously. This enables neuromorphic sensors to sample at a micro-second level and efficiently capture the dynamics," said Antil. "This will be useful for images/videos captured by phone, planes, drones, or other objects subject to motion blur, high contrast environment, and environment with access to limited power and memory, so that they can collect higher quality imaging."

The project will be a testbed for new algorithms and subsequent deployment of this hardware in the field. Using the generated data, the algorithms will ultimately assist in object recognition, motion detection, and scene understanding.

"We're currently procuring neuromorphic cameras, drones, and audio sensors now that will generate data and allow our team to begin testing," Antil said.

With the acquired hardware, the researchers will create an open environment that will be accessible to several neighboring institutions, including academia, national labs (e.g., U.S. Naval Research Lab Washington DC, AFRL) industries, and historically Black colleges and universities (HBCUs). This lab will also serve as a national resource. Significant impact is expected on Air Force, Space Force (imaging, object tracking), Civil Engineering (structures such as bridges), Machine Learning (neuromorphic computing), etc.

Mason will host workshops and summer programs to train researchers on this hardware.

The funding (direct) amount is $357,868 from the Air Force Office of Scientific Research (AFOSR) for this project. Funding began in April 2024 and will end in late March 2025.

For questions, please reach out to Harbir Antil: hantil@gmu.edu.

Provided by George Mason University