Bugs and bats inspire creation of bionic super 3D cameras

The cameras were developed by a pair of bioengineers at the University of California, Los Angeles (UCLA) and a former postdoctoral scholar. Computational image processing powers the cameras and can decipher the size and shape of objects hidden around corners or behind other items.

The researchers say the technology could incorporated into autonomous vehicles or medical imaging tools “with sensing capabilities far beyond what is considered state of the art today”.

In developing the cameras, the team drew inspiration from two natural phenomena found in flies and bats.

In the dark, bats can visualise a vibrant picture of their surroundings by using echolocation, or sonar. Their high-frequency squeaks bounce off their surroundings and return to be picked up by their ears.

The minuscule differences in how long it takes for the echo to reach the nocturnal animals and the intensity of the sound tell them in real time where things are, what’s in the way and the proximity of potential prey.

Many insects have geometric-shaped compound eyes, in which each “eye” is composed of hundreds to tens of thousands of individual units for sight — making it possible to see the same thing from multiple lines of sight.

For example, flies’ bulbous compound eyes give them a near-360-degree view even though their eyes have a fixed focus length, making it difficult for them to see anything far away, such as a fly-swatter held aloft.

With this knowledge, the UCLA-led team set out to design a high-performance 3D camera system with advanced capabilities that leverage these advantages but also address nature’s shortcomings.

“While the idea itself has been tried, seeing across a range of distances and around occlusions has been a major hurdle,” said Liang Gao, an associate professor of bioengineering at the UCLA Samueli School of Engineering.

“To address that, we developed a novel computational imaging framework, which for the first time enables the acquisition of a wide panoramic view with simple optics and a small array of sensors.”

Called “Compact Light-field Photography,” or CLIP, the framework allows the camera system to “see” with an extended depth range and around objects.

In experiments, the researchers showed that their system can “see” hidden objects that are not spotted by conventional 3D cameras.

Using seven lidar cameras with CLIP, the array takes a lower-resolution image of the scene, processes what individual cameras see, then reconstructs the combined scene in high-resolution 3D imaging.

The researchers showed the camera system could image a complex 3D scene with several objects, all set at different distances.

“If you’re covering one eye and looking at your laptop computer, and there’s a coffee mug just slightly hidden behind it, you might not see it, because the laptop blocks the view,” Gao explained.

“But if you use both eyes, you’ll notice you’ll get a better view of the object. That’s sort of what’s happening here, but now imagine seeing the mug with an insect’s compound eye. Now multiple views of it are possible.”

According to Gao, CLIP helps the camera array make sense of what’s hidden in a similar manner. Combined with lidar, the system can achieve the bat echolocation effect so one can sense a hidden object by how long it takes for light to bounce back to the camera.