If you’ve ever used a pair of night-vision goggles, you’re familiar with the fluorescent display tinging the nighttime world in shades of green. The reason why you see everything in one color versus any other is because of the human eye. We’re most sensitive to light at wavelengths around 555 nanometers—which appears as a bright green. So under the green hue of night vision, we can perceive and distinguish leaves on a tree or a person climbing a building, better than we could under other colors.
But now the monochrome technology is about to get a technicolor upgrade.
In a new study published Wednesday in the journal PLOS One, researchers at the University of California, Irvine used machine learning to transform what you see through a night vision scope or camera into a veritable rainbow of colors. This game-changing development could benefit not just the military, but also medical technologies, healthcare, and even more niche tasks like art restoration.
ADVERTISEMENT
But to understand how the new night vision tech works, it’s important to first understand how human vision works. People are can also see in the visible light spectrum, which runs from around 380 nanometers (where the color purple is situated) to 740 nanometers (where we can see red). Sensors in the eye, called cone cells, absorb the energy from these wavelengths and generate an electrical impulse carried to the brain, where the perception of color is created. Another group of sensors called rod cells deal with grayscale, helping us see in low-light conditions.
Understandably, seeing is nigh impossible when we’re in pitch darkness with no light source around. That’s where another type of wavelength right next to the visible light spectrum comes in handy: infrared. Night vision picks up infrared to create the image you see on a display. But as mentioned earlier, you can only recreate objects well in green.
There are technologies that get around this by using ultra-sensitive cameras that detect and amplify visible light instead of infrared. However, relying on visible light can harm sensitive tissues like the eye or some types of delicate biological samples in a lab, Dr. Andrew Browne, an ophthalmologist and biomedical engineer at UC Irvine who led the study, told The Daily Beast.
In recent years, scientists have been turning to machine learning to see in the dark. Browne and his team also decided to turn to the emerging field, taking neural networks—computer programs that are like an artificial brain—and feeding them information about different colors based on hundreds of printed pictures.
“The way neural networks are trained is just like if I gave you 100 pictures of a person’s face and I circle the nose in every single one of those pictures, then the neural network would learn to recognize labeled objects,” Browne explained. “What we did with [our] neural network is we gave it hundreds of pictures containing data on the visible and infrared spectrum.”
The UC Irvine researchers used infrared images of three different wavelengths and deep learning to predict the visible spectrum image.
Andrew BrowneArmed with its freshly-learned knowledge, the neural networks were asked to reconstruct the color of the images shown—now taken by a night vision camera—and the results weren’t too shabby.
“There is some variability because you can put them side-by-side and see some differences here and there,” said Browne. “[But they’re] basically indistinguishable like you wouldn’t even know you were looking at a predicted image.”
Browne is quick to add that while this proof-of-concept is a great first step in improving night vision, the neural network’s predictions are limited to its storehouse of data.
“Let’s say that if the goal is to take a whole bunch of swatches of paint and predict colors, it might perform well if it’s a fixed [set of] swatches, but if I start getting into real-world examples, the neural networks perform as well as the data they’re fed,” he said.
Nevertheless, the researchers are working on incorporating more training data sets and improving the neural networks’ computer hardware so they can collect and store more data. The military would obviously be interested in this type of technology, but it could prove useful in eye surgery, where night vision could be used to protect sensitive retinal tissue from light damage. It could even be helpful for art restoration where visible light can deteriorate the art.
“Artificial neural networks are something that is going to support a host of different scientific application endeavors. And it’s why they’re a very powerful tool in the context of medical care, they can enhance a clinician’s ability to function,” said Browne. “In the context of new technologies, they can enhance the performance of that technology to perform a specific task.”
