Teaching AI to accurately colorize marine plankton images

Teaching AI to accurately colorize marine plankton images
Credit: LI Jianping

Plankton are fundamental to the marine ecosystem and an indispensable means of modern marine ecological management. Since previous studies have shown that zooplankton are insensitive to long wavelength (i.e., red) light, such light is often used to make images of plankton as part of the ecological management process. Unfortunately, red light only produces grayscale images of plankton, thus losing information about their true color.

In response to this problem, a research team led by Dr. LI Jianping from the Shenzhen Institute of Advanced Technology (SIAT) of the Chinese Academy of Sciences (CAS) recently developed an automatic plankton colorization algorithm, the IsPlanktonCLR network, based on deep convolutional neural networks.

The algorithm is trained to convert gray images acquired under red lighting into high-fidelity color images. It can automatically and truthfully “dye” marine plankton grayscale images shot underwater into their natural colors, and the colorization effect is very close to human eye perception.

The study was released in the European Conference on Computer Vision (ECCV), which was held in Israel from October 23 to 27.

Researchers from Xiamen University and the Harbin Institute of Technology (Shenzhen) were also involved in the research.

The IsPlanktonCLR network uses a two-path structure with a self-guidance function, a customized color palette and a stepwise focusing loss function to achieve automatic colorization of plankton grayscale images. It shows excellent accuracy and fidelity in color restoration for rare species and key parts of common species.

In order to train and develop the IsPlanktonCLR algorithm, the team constructed a dataset containing thousands of color-gray plankton image pairs. Using this dataset, the team not only trained the colorization algorithm, but also compared its performance with state-of-the-art colorization algorithms. The experimental results showed that the IsPlanktonCLR algorithm offered the best results not only in terms of human visual perception but also in terms of peak signal to noise ratio (PSNR), structural similarity (SSIM), Fréchet inception distance (FID) and other classical metrics frequently used in machine vision.

In addition, existing colorization algorithms generally lack objective and quantitative evaluation metrics. Existing evaluation metrics not only have difficulty simulating the color perception ability of human vision but also have difficulty objectively evaluating the color similarity between the colored image and the ground-truth image.

Therefore, the researchers proposed a color similarity evaluation index, Color Dissimilarity (CDSIM), which combines a color histogram, color aggregation vector, color correlogram and color gradient. Through the tests on marine plankton and natural scene images, the researchers verified that CDSIM is not only effective in evaluating the effect of colorization algorithms but is also more suitable for image colorization evaluation in the field of scientific imaging than other existing metrics.

“The development of IsPlanktonCLR provides a new artificial intelligence solution for ocean imaging instruments to obtain accurate and real observation results,” said Dr. LI. “This provides new technical means for us understanding the ocean.”

DOI: 10.1007/978-3-031-19839-7_13
Article Title: Colorization for in situ Marine Plankton Images
Article Publication Date: 27-Oct-2022

Media Contact

ZHANG Xiaomin
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
xm.zhang@siat.ac.cn

www.siat.ac.cn

Media Contact

ZHANG Xiaomin
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences

All latest news from the category: Information Technology

Here you can find a summary of innovations in the fields of information and data processing and up-to-date developments on IT equipment and hardware.

This area covers topics such as IT services, IT architectures, IT management and telecommunications.

Back to home

Comments (0)

Write a comment

Newest articles

New organoid with all key pancreas cells

Researchers from the Organoid group (previously Clevers group) at the Hubrecht Institute have developed a new organoid that mimics the human fetal pancreas, offering a clearer view of its early development….

Unlocking the potential of nickel

New study reveals how to use single atoms to turn CO2 into valuable chemical resources. Nickel and nitrogen co-doped carbon (Ni-N-C) catalysts have shown exceptional performance in converting CO2 into…

‘Spooky action’ at a very short distance

Scientists map out quantum entanglement in protons. Particles streaming from collisions offer insight into dynamic interactions and collective behavior of quarks and gluons. Scientists at the U.S. Department of Energy’s…