Approach #1 - Interactive storyline

Finding Solutions

My inspiration came from several sources that had captured my imagination:

The first was "The Void Between," an interactive experience created with GDevelop that blew me away with its use of actual filmed content. The way it seamlessly blended real footage with interactive elements felt revolutionary to me. It wasn't just a game—it was an interactive movie where your choices actually mattered.

This concept reminded me of Netflix's interactive movies, where audiences can choose their own storyline and watch different outcomes unfold based on their decisions. Shows like "Black Mirror: Bandersnatch" had shown me how powerful this medium could be for engaging viewers in ways traditional media couldn't.

The Interactive Movie Concept

My vision was to create an interactive storyline where users would follow characters through situations where sign language naturally occurred. Instead of abstract lessons or mechanical practice, learners would be immersed in real scenarios where communication through sign language was essential to the story.

I imagined scenes like:

  • A family dinner where a deaf member joins the conversation
  • A workplace scenario where colleagues need to communicate across a noisy environment
  • Social situations where sign language becomes the bridge between hearing and deaf communities

The interactive element would come from viewers making choices about how to respond in these situations, with different sign language options leading to different story outcomes. It would be education disguised as entertainment—or perhaps entertainment that happened to be educational.

Why GDevelop?

I chose GDevelop for this initial approach because of its flexibility with multimedia content. After seeing what "The Void Between" accomplished, I knew GDevelop could handle the combination of video content and interactive elements I wanted to create.

GDevelop's visual scripting system also felt approachable for creating branching narratives without getting bogged down in complex coding. I could focus on the storytelling and educational aspects while the engine handled the technical implementation of choice-based interactions.


Implementation

For a small demo prototype, I prepared 4 scenes with 4 video clip scenes. 
Each scene has 2 buttons to go to another scene. This small prototype is to test the basic functionality













Simple Scripting














Limitations

As I began working on this concept, I quickly encountered some significant challenges:

Video Quality Issues: My initial test footage was disappointingly low quality, and the interactive storyline concept depended heavily on professional-looking video content.

Production Requirements: For a final year project, I needed high-quality video production, which required professional filming equipment, skilled actors, well-written scripts, and proper post-production—resources I didn't have access to as a student.

Resource Constraints: The scope was simply too much for a student project. I was essentially trying to produce short films while developing an interactive framework—two completely different skill sets requiring significant time and expertise.

These practical limitations forced me to reconsider what was actually achievable within my constraints as a student developer.

Insights

The more I worked on this approach, the more I realized that while interactive storytelling is powerful for engagement, it wasn't the most effective method for actually teaching sign language. Learning sign language is fundamentally about developing muscle memory and visual recognition—skills that require active practice rather than passive observation.

I began to see that what users really needed was:

  • Direct feedback on their hand movements
  • Immediate correction when forming signs incorrectly
  • Progressive skill building through practice
  • Social interaction and game-like motivation

These realizations led me to completely reconsider my approach. While the interactive storyline concept was creatively exciting, it wasn't serving the core educational needs of my target audience.

References

Comments

Post a Comment

Popular posts from this blog

UDP vs TCP

After successfully implementing hand sign detection and classification in Python, my next challenge was connecting this system to Unity for a more interactive learning experience. At first, I saw a video from the same YouTube channel that showed how to connect Python to Unity using UDP. This inspired me to explore network protocols more deeply to understand what would work best for my project. Initial Discovery: UDP from the YouTube Tutorial When I first started researching how to connect my Python classifier to Unity, I came across a tutorial video from the same YouTube channel ( Murtaza’s Workshop - Robotics and AI)  that taught me about hand detection. The video suggested using UDP (User Datagram Protocol) for this connection. This piqued my interest, but before implementing it, I wanted to understand why UDP was recommended over TCP (Transmission Control Protocol). My Understanding of the Two Protocols Through my research, I found that TCP and UDP represent two different philos...
Read more

Initial Research

Image
Bridging Family Gaps for Deaf Teenagers Communication is one of the most important aspects of building strong family relationships. However, for many deaf teenagers raised in hearing families, communication is often one of the biggest barriers. Everyday conversations, emotional expressions, and even simple family interactions can become sources of frustration or isolation when a shared language is missing. These experiences may go unnoticed by hearing family members, but for deaf teens, they can have long-term effects on emotional well-being, self-identity, and social development. Inspiration This project was inspired by the Korean drama Twinkling Watermelon . In the story, a young deaf girl is discouraged by her mother from learning sign language. Despite this, her boyfriend takes the initiative to learn sign language so that he can communicate with her and eventually teaches her as well. How Sweet! Later in the story, he becomes deaf due to an accident. The two eventually marry and r...
Read more

Final Approach: Python OpenCV

Image
After experimenting with 3 different approaches, I realized that fully implementing them would be too risky. Each required a deep technical understanding and advanced knowledge of complex systems that I don’t yet possess. While the concepts were promising, I ultimately decided to take a more practical route that aligned with my current skills and ensured the project would be functional and deliverable within my timeline. Luckily, when I decided to broaden my research and look into more practical solutions, I came across a fascinating YouTube tutorial on building a hand sign detection system specifically for American Sign Language (ASL) alphabets that offered a more systematic approach. Learning From Mr. Murtaza The tutorial I discovered from the YouTube channel called "Murtaza's Workshop - Robotics and AI" presented a comprehensive step-by-step guide for building a robust hand sign detection system using ASL alphabets A, B, and C. What caught my attention was its two-step...
Read more