A Low-Cost Smartphone-Based Sign Language Translator Using Computer Vision for Real-Time Deaf–Hearing Communication

Computer Science and Engineering

Communication barriers between deaf or mute individuals and hearing people remain a major challenge in daily life. Traditional communication methods such as interpreters or sensor-based gloves are often expensive, require specialized hardware, and are not always available in real-time situations. Recent advances in computer vision and mobile computing have opened new possibilities for developing accessible assistive technologies. This paper presents a low-cost smartphone-based sign language translation system designed to facilitate communication between deaf individuals and hearing people. The proposed system utilizes the smartphone camera to capture hand gestures and applies Media Pipe hand tracking for extracting hand landmarks. These features are then processed using a deep learning classification model to recognize sign language gestures. The recognized gestures are converted into text and subsequently transformed into speech using a text-to-speech module, enabling real-time interaction. The proposed approach focuses on affordability, portability, and ease of use by eliminating the need for specialized hardware such as sensor gloves or external devices. Experimental evaluation was performed using a dataset of commonly used Indian Sign Language gestures. The system achieved high recognition accuracy while maintaining real-time performance on a standard smartphone device. The results demonstrate that smartphone-based gesture recognition systems can provide an effective and scalable solution for improving accessibility and enabling inclusive communication through human–computer interaction.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.