Bridging Family Gaps for Deaf Teens

After creating the Learning Mode for Sign Pals, I expanded the project with a competitive Duel Mode where players can challenge each other. Today, I'll share how I developed a duel system that allows you to pick your opponent! Design Goals My key goals for the opponent selection visuals were: Intuitive selection of opponents Engaging animations to create excitement Support for both profile-based opponents and guest players I've always believed that competition can be a powerful motivator for learning, which is why I was excited to add this Duel Mode. The opponent selection screen needed to set the tone—creating that pre-match excitement you feel before friendly competition begins. VS Animation System The standout feature is the animated VS graphic that appears when an opponent is selected, adding visual excitement and signaling that a match is being prepared. I'm particularly proud of this animation system. The subtle bounce effect took me longer than I'd like to ...

After choosing UDP as the optimal communication protocol for my sign language learning game, I've finally successfully implemented the full communication pipeline between the Python hand detection system and the Unity game interface. In this blog, I will go through on how I build my learning mode! Python Side Implementation On the Python side, I use TensorFlow Lite with a pre-trained model to detect hand signs from webcam input. When a sign is detected with confidence, I extract the letter and confidence score, then send this information to Unity: Unity Side Implementation For the Unity side, I used the UDP receiver script provided in the YouTube tutorial and customized it to fit my specific needs. The script already contained most of the functionality I needed, which saved me significant development time. Thank you so much Mr. Murtaza! My customizations focused on: Adapting the script to work with my specific JSON data format Adding the event system to notify game components ab...

After successfully implementing hand sign detection and classification in Python, my next challenge was connecting this system to Unity for a more interactive learning experience. At first, I saw a video from the same YouTube channel that showed how to connect Python to Unity using UDP. This inspired me to explore network protocols more deeply to understand what would work best for my project. Initial Discovery: UDP from the YouTube Tutorial When I first started researching how to connect my Python classifier to Unity, I came across a tutorial video from the same YouTube channel ( Murtaza’s Workshop - Robotics and AI)  that taught me about hand detection. The video suggested using UDP (User Datagram Protocol) for this connection. This piqued my interest, but before implementing it, I wanted to understand why UDP was recommended over TCP (Transmission Control Protocol). My Understanding of the Two Protocols Through my research, I found that TCP and UDP represent two different philos...