Translation of Afan Oromo Sign Language to Afan Oromo Text Using Deep Learning Approach

Abstract

Sign language is a mode of communication used by individuals with hearing impairments and those with hearing difficulties to interact with one another and with individuals who can hear. However, Afan Oromo Sign Language has not been thoroughly studied, with no publicly available datasets and limited research efforts. Most existing studies on Afan Oromo Sign Language translation focus on sign recognition rather than complete sign-to-text translation. This research seeks to address this gap by developing an automatic translation system that converts Afan Oromo Sign Language into Afan Oromo text using LSTM and Transformer encoder decoder models. A custom dataset was prepared by extracting image frames from videos of signed phrases, ensuring a signer-independent translation system. The study implemented two deep learning architectures: an LSTM-based sequence-to-sequence model and a Transformer-based encoder-decoder model. These models were evaluated using Word Error Rate (WER) and ROUGE (ROUGE-1, ROUGE-2, and ROUGE-L) scores. Results indicate that the LSTM model achieves lower WER (best: 18.34%), suggesting better sequence alignment, whereas the Transformer model excels in text generation quality, with ROUGE-1 reaching 58.95%. However, the higher WER of the Transformer model (42.17% - 48.49%) highlights challenges in structural consistency between sign sequences and textual outputs. This study marks a significant step forward in translating Afan Oromo Sign Language phrases and sentences and offers essential insights for developing low-resource sign language translation systems. The findings highlight the necessity for larger datasets, hybrid models, and linguistic adaptation techniques to improve translation accuracy and accessibility for the Deaf community.