DrishtiXAI: a Bengali image captioning system with explainable AI

Abstract

This study presents DrishtiXAI, an explainable Bengali image captioning system designed to support visually impaired users through culturally relevant image descriptions and text-to-speech technology. A dataset of 18,539 Bengali image-caption pairs was developed, and two deep learning approaches were evaluated. The InceptionV3-GRU model with Bahdanau attention outperformed the EfficientNet-B4-Transformer model in reliability, achieving a mean BLEU score of 0.5157 on a separate dataset. Attention maps provided visual explanations of the image regions used to generate captions, while a mobile application enabled accessible use. A user study of 20 participants demonstrated positive ratings for clarity and supportiveness, highlighting the potential of transparent and culturally appropriate AI for Bengali-speaking communities.