Abstract
Gradient-weighted Class Activation Mapping (Grad-CAM) has been a successful technique to produce visual explanation for CNN-based models. In this paper, we verify its applicability in the task of remote sensing image classification. The results show Grad-CAM gives contradictory localization of the important regions for some remote sensing images. To solve this problem, we propose a new strategy, bidirectional gradient verification (BiGradV), to rectify the visual explanation produced by Grad-CAM. The BiGradV is based on the fact both positive and negative gradients can be sensitive to class discrimination of remote sensing images. It designs an internal feature map occlusion with confidence drop decision to verifying which directional gradients works for certain class. The verified gradients are then used to gain the correct visual explanation. Experiments on both remote sensing image dataset and general image datasets demonstrate our proposed strategy is effective and generalized. It could provide a good complement to the Grad-CAM based methods.