Attention-based Multimodal Image Feature Fusion Module for Transmission Line Detection

Abstract

IEEETransmission line (TL) inspection is important for ensuring a stable supply of electricity to rural areas. Currently, there are several TL detection approaches based on computer vision; however, they have limitations owing to background clutter in visible light images. This study presents a novel multimodal image feature fusion module that utilizes both visible light and infrared images to enhance TL detection performance. The proposed module consists of a multi-branch feature extraction (MFE) block followed by a channel-wise attention (CA) block. The first block extracts the representative features of each modal input using multiple branches. The outputs of the MFE block are jointly aggregated into an attention vector in the CA block. Finally, the attention vector re-calibrates each input feature of the proposed module. To reduce the number of additional parameters due to the insertion of the module, we introduced a channel-shrink factor in the MFE block and utilized a 1 x 1 convolution in the CA block. Comparison experiments with various augmented conditions of day, night, fog, and snow were conducted on a real-world dataset, which we constructed by visible light and infrared images. The results showed that the proposed module outperformed not only the case of single modal input but the state-of-the-art fusion methods, regardless of the baseline networks. Additionally, the proposed module showed effectiveness in terms of capacity when the baseline network has a large number of weight parameters.