Researchers develop camera inspired by the human eye

A team led by University of Maryland computer scientists have invented a camera mechanism that improves how robots see and react to the world around them. Inspired by how the human eye works, their camera system mimics the tiny involuntary movements used by the eye to maintain clear and stable vision over time. The team’s prototyping and testing of the camera — called the Artificial Microsaccade-Enhanced Event Camera (AMI-EV) — was detailed in a paper published in the journal Science Robotics in May 2024.

“Event cameras are a relatively new technology better at tracking moving objects than traditional cameras, but today’s event cameras struggle to capture sharp, blur-free images when there’s a lot of motion involved,” said the paper’s lead author Botao He, a computer science PhD student at UMD. “It’s a big problem because robots and many other technologies — such as self-driving cars — rely on accurate and timely images to react correctly to a changing environment. So, we asked ourselves: How do humans and animals make sure their vision stays focused on a moving object?”

For He’s team, the answer was microsaccades, small and quick eye movements that involuntarily occur when a person tries to focus their view. Through these minute yet continuous movements, the human eye can keep focus on an object and its visual textures — such as colour, depth and shadowing — accurately over time.

“We figured that just like how our eyes need those tiny movements to stay focused, a camera could use a similar principle to capture clear and accurate images without motion-caused blurring,” he said.

The team successfully replicated microsaccades by inserting a rotating prism inside the AMI-EV to redirect light beams captured by the lens. The continuous rotational movement of the prism simulated the movements naturally occurring within a human eye, allowing the camera to stabilise the textures of a recorded object just as a human would. The team then developed software to compensate for the prism’s movement within the AMI-EV to consolidate stable images from the shifting lights.

A diagram of the novel camera system (AMI-EV). Credit: UMIACS Computer Vision Laboratory.

Study co-author Yiannis Aloimonos, a professor of computer science at UMD, views the team’s invention as a big step forward in the realm of robotic vision.

“Our eyes take pictures of the world around us and those pictures are sent to our brain, where the images are analysed. Perception happens through that process and that’s how we understand the world,” explained Aloimonos, who is also director of the Computer Vision Laboratory at the University of Maryland Institute for Advanced Computer Studies (UMIACS). “When you’re working with robots, replace the eyes with a camera and the brain with a computer. Better cameras mean better perception and reactions for robots.”

In early testing, AMI-EV was able to capture and display movement accurately in a variety of contexts, including human pulse detection and rapidly moving shape identification. The researchers also found that AMI-EV could capture motion in tens of thousands of frames per second, outperforming most typically available commercial cameras, which capture 30 to 1000 frames per second on average. This smoother and more realistic depiction of motion could prove to be pivotal in anything from creating more immersive augmented reality experiences and better security monitoring to improving how astronomers capture images in space.

A demonstration of how microsaccades counteract visual fading. After a few seconds of fixation (staring) on the red spot in this static image, the background details of this image begin to visually fade. This is because microsaccades have been suppressed during this time and the eye cannot provide effective visual stimulation to prevent peripheral fading. Credit: UMIACS Computer Vision Laboratory. For a larger image click here.

“Our novel camera system can solve many specific problems, like helping a self-driving car figure out what on the road is a human and what isn’t,” Aloimonos said. “As a result, it has many applications that much of the general public already interacts with, like autonomous driving systems or even smartphone cameras. We believe that our novel camera system is paving the way for more advanced and capable systems to come.”

The paper, ‘Microsaccade-inspired event camera for robotics’, was published in Science Robotics on 29 May 2024.

Top image credit: iStock.com/Vertigo3d