ChartSight: an automated scheme for assisting visually impaired in understanding scientific charts

Abstract

The visual or Non-Textual components like charts, graphs, and plots are frequently used to represent the latent information in digital documents. These components bolster in better comprehension of the underlying complex information. However, these data visualization techniques are of not much use to visually impaired. Visually impaired people, especially in developing countries, rely on braille, tactile, or other conventional tools for reading purposes. Through these approaches, the understanding of Non-Textual components is a burdensome process with serious limitations. In this paper, we present ChartSight, an automated and interactive chart understanding system. ChartSight extracts and classifies the document images into different chart categories, and then uses heuristics-based content extraction methods optimized for line and bar charts. It finally represents the summarized content in audio format to the visually impaired users. We have presented a densely connected convolution network-based data-driven scheme for the chart classification problem, which shows comparatively better performance with the baseline models. Multiple datasets of chart images are used for the performance analysis. A comparative analysis of supporting features has also been performed with the other existing approaches.