结合语义分割的灰度图像彩色化

Colorization of grayscale images combined with semantic segmentation

  • 摘要: 图像彩色化是计算机视觉领域备受关注的核心问题之一,近年来吸引了越来越多的研究者。彩色化技术提高了人眼对灰度图像,特别是微光图像中目标的识别率和场景的理解力,但目前彩色化方法仍然存在颜色溢出、颜色偏差和边界模糊等问题。为解决上述问题,提出了一种结合语义分割的灰度图像彩色化方法。该方法设计了一个由分类子网络、语义分割子网络组成的彩色化网络,将语义分割子网络和分类子网络提取的特征融合到彩色化网络中,引导彩色化网络为图像中每个对象和背景赋予准确的颜色;将结合空间和通道的注意力机制引入语义分割子网络,获得更准确的语义分割结果,提高彩色化网络对图像边界的感知能力和对细节的处理能力。为验证本文方法的有效性,通过多像素光子计数器(MPPC)实验平台,在低照度环境下拍摄了多组不同场景的微光图像,并对其进行彩色化验证,结果表明,本文方法在峰值信噪比(peak signal-to-noise ratio,PSNR)、结构相似性(structural similarity index measure,SSIM)、感知相似度(learned perceptual image patch similarity,LPIPS)上分别提升6.12%、11.65%、4.99%。

     

    Abstract: Image colorization is one of the core problems of great interest in computer vision and has attracted more and more researchers in recent years. The colorization technique improves the recognition rate and scene comprehension of targets of human eyes in grayscale images, especially in low-light-level (LLL) images. However, the current colorization methods still have problems such as color bleeding, color deviation and boundary blurring. To solve the above problems, a colorization method for grayscale images combined with semantic segmentation was proposed. A colorization network composed of a classification sub-network and a semantic segmentation sub-network was designed to fuse the features extracted from the semantic segmentation sub-network and the classification sub-network into the colorization network, and guide the colorization network to assign an accurate color to each object and background in the image. Then the attention mechanism combining with the space and channel was introduced into the semantic segmentation sub-network to obtain a more accurate semantic segmentation results, thereby improving the perception ability of image boundaries and the processing ability of details of the colorization network. To verify the effectiveness of the proposed method, several sets of LLL images of different scenes were captured under LLL environment and colorized by multi-pixel photon counter (MPPC) experimental platform. The experimental results show that the peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS) of the proposed method is improved by 6.12%, 11.65%, and 4.99%, respectively.

     

/

返回文章
返回