SignMate was inspired by the need to make communication more inclusive for the 466 million people worldwide living with disabling hearing loss. We wanted to create a tool that would help bridge the communication gap by enabling people to learn American Sign Language (ASL) in an interactive and engaging way.
SignMate is a web app that generates random ASL videos of common English words. The user attempts to identify the word shown in the video by typing it out. If successful, they receive positive feedback and can continue with a new set of videos. If unsuccessful, the app provides a failure warning, highlighting the missed words and encouraging improvement.
We built SignMate using React for the front-end and Node.js for the back-end, ensuring smooth interaction and functionality. For the ASL dataset, we used a Kaggle dataset, which provided a variety of ASL images and videos for common English words. The app uses these resources to generate random ASL videos and evaluate user input.
One of the major challenges we faced was ensuring the accuracy of user input in matching the ASL gestures with the correct words. Additionally, working with video data from Kaggle required optimization for quick loading times while maintaining video quality.
We are proud of building an interactive web app that not only serves as a learning tool but also fosters inclusivity. The app provides immediate feedback, which helps users improve their ASL skills. It was exciting to integrate the ASL video dataset and create a smooth user experience.
Through this project, we gained valuable experience in working with large datasets, optimizing video content for web apps, and handling real-time user interactions. We also learned about the importance of accessibility and inclusivity in digital applications.
Next, we plan to expand the app’s features by incorporating more advanced ASL vocabulary, improving the dataset to include more diverse signs, and possibly adding speech recognition to allow for a more immersive learning experience. We also aim to enhance the mobile version for greater accessibility.