User Testing Results

Gathering Feedback

After months of development work on my sign language learning game "Sign Pals," I conducted my final round of user testing to evaluate the complete game experience. Between May 15-17, 2025, I worked with six participants who played through the entire game and provided detailed feedback on all aspects, from the opening animation to the competitive duel mode.

Sign Pals was designed specifically for hearing family members and friends who want to learn sign language to better communicate with deaf loved ones. This audience focus shaped everything from the game's approachable design to its social gameplay elements, encouraging family members to learn together in a fun, low-pressure environment.

I gathered in-depth feedback from a small but diverse group of testers who represented my target audience of hearing individuals interested in learning sign language. Their feedback provides valuable insight into how well Sign Pals achieves its goal of making sign language learning accessible and enjoyable for families and friends of deaf individuals.

In this blog post, I'll share what I learned from this testing phase and how it will influence the future development of Sign Pals.

Testing Process

Participants were asked to play through the entire game experience, including:

1. The opening animation
2. Creating a player profile
3. Navigating the mode selection screen
4. Trying both Learning Mode and Duel Mode

After playing, they completed a detailed feedback form evaluating various aspects of the game on a scale of 1-5 and providing written comments. The form covered everything from game flow understanding to detection accuracy and overall experience.

Ethical Considerations: Participant Consent

To ensure my user testing followed ethical research practices, I created formal documentation for all participants. Prior to beginning any testing sessions, each participant received a Participation Information Sheet and signed an Informed Consent Form.

Participation Information sheet

The information sheet provided details about:

• The project title: "Bridging Family Gaps for Deaf Members"
• My role as principal investigator
• The purpose of the study: evaluating Sign Pals as a tool to help hearing family members learn sign language to communicate with deaf loved ones
• What would happen during the 10-15 minute testing session
• Data collection practices and confidentiality measures
• Participants' rights, including the voluntary nature of participation

Informed Consent Form

Each participant signed a consent form confirming they:

• Had read and understood the information provided
• Had opportunities to ask questions
• Understood their participation was voluntary
• Agreed to play through the application and provide feedback
• Understood how their data would be used
• Had the option to allow or decline any photographs

This formal consent process was important not only for academic research standards but also for establishing trust with participants. By clearly communicating how their feedback would be used and protected, participants felt more comfortable providing honest assessments of the application.

All testing data will be deleted upon completion of the project by June 13, 2025, as promised in the consent documentation.

The structured approach to obtaining informed consent reflects my commitment to conducting user testing that respects participants' privacy and autonomy while gathering the valuable feedback needed to improve Sign Pals.

Key Findings: What Worked Well

Strong Overall Experience

The most encouraging result was that players genuinely enjoyed the game. On a scale of 1-5, the average overall experience rating was 4.8, with five out of six participants giving it a perfect 5!

Intuitive Game Flow

Players found the game flow intuitive and easy to understand, giving it an average score of 4.7 out of 5. One player commented, "Flow was good and quick to understand," while another wrote, "Very impressive, the colors are eye pleasing."

Clear Instructions

Instruction clarity received an average rating of 4.8 out of 5, indicating that players understood what they were supposed to do at each stage of the game.

Learning Mode Detection Accuracy

The accuracy of the sign detection in Learning Mode received positive feedback with an average score of 4.3 out of 5.

Visual Design and UI

The visual design of the game received unanimous praise:

• "Very cute starting screen and a simplistic studious style"
• "Functionality, cute animations and idea of the game"
• "The artstyle and aesthetic is very pleasing"
• "It looks clean and neat for the UI"

Education Value

Every single participant confirmed they learned something new about sign language from playing the game. One player specifically mentioned, "Yes, I learned sign language ABC," while another noted they learned "new alphabets for sign language."

Area for Improvement

Despite the positive feedback, testing also revealed several areas that need improvement:

Duel Mode Detection Accuracy

Duel Mode received a lower average score of 3.3 out of 5 for detection accuracy. Multiple players mentioned issues:

• "Takes too long for the next sign to appear"
• "What letter was I supposed to sign (couldn't really 'see' the letter)"
• "The system might be over sensitive"

This will be a priority area for refinement in the next development phase.

Camera View

Two participants noted issues with the camera view, with one specifically mentioning that it was "flipped." This is a relatively simple fix that will improve the user experience significantly.

Pacing Issues

A few players mentioned that the game felt slow at times:

• "Pacing feels a bit slow at times"
• "It can be slow when it comes to learning mode or when you move on to the next round"

Technical Performances

One player mentioned lag during gameplay, which will need to be addressed to ensure a smooth experience for all users.

What Players Liked Most

When asked what they liked most about the game, participants mentioned:

• "Functionality, cute animations and idea of the game"
• "The idea of detecting hand signs in a gamification"
• "The duel mode"
• "The artstyle and aesthetic"
• "Very beginner friendly"

The Social Aspect

One particularly encouraging finding was that all participants enjoyed the social aspect of the game. When asked "Did you have fun playing the game with other people?" everyone responded positively.

This validates one of my core design goals: creating an experience that makes learning sign language a social, enjoyable activity rather than a solitary exercise.

Would They Play Again?

Perhaps the most important validation came from the final questions. When asked if they would play the game again and if they would recommend it to others wanting to learn sign language, 100% of participants said yes to both questions.

Comments like "Great idea, was very innovative and a creative app" and "Very beginner friendly :D" suggest that Sign Pals is achieving its goal of making sign language learning accessible and enjoyable.

Future Improvements Based on Feedback

Based on the testing results, I've identified several key improvements for future development:

1. Enhance Duel Mode: Improve visibility of the target letter, reduce waiting time between rounds, and refine detection sensitivity.
2. Optimize Performance: Address lag issues to ensure smooth gameplay.
3. Expand Content: Multiple testers suggested adding more sign language content beyond the initial alphabet (numbers, common words, etc.).
4. Add Transition Animations: One player specifically suggested adding "transition animation between scenes" to enhance the flow.
5. Improve Camera Settings: Fix the mirrored camera view and add tips for ensuring players have a solid color/clear background.
6. Adjust Game Pacing: Fine-tune the timing in both Learning Mode and Duel Mode to keep the experience engaging.

Implementing User Feedback: Quick Improvements

After analyzing the user testing results, I immediately implemented several key improvements to address the most common feedback points. Rather than waiting for a major update, I prioritized changes that could significantly enhance the user experience with minimal development time.

1. Background Detection Tips

Several testers experienced inconsistent hand detection, which I traced back to complex backgrounds interfering with the recognition system. This issue revealed a technical limitation in my setup: the Python backend that handles sign detection would occasionally crash when processing complex visual backgrounds, requiring me to restart it mid-testing session.

To address this limitation, I added a prominent "Pro Tip" message at the start of both Learning and Duel modes that reads: "For best results, play in front of a plain background." It improves the user experience by enhancing detection accuracy and also reduces the likelihood of backend crashes, creating a more stable experience. 

2. Improved Game Pacing

The most consistent piece of feedback was that the game felt too slow at times. Players specifically mentioned delays between successful sign detection and the next sign appearing, as well as lengthy transition times between Duel Mode rounds.

I addressed this by reducing the delay between successful sign detection and the next prompt by nearly halving the transition times between rounds in Duel Mode from 3 seconds to 1.5 seconds. These timing adjustments make the game feel more responsive and engaging while still giving players enough time to process feedback.

Conclusion

Overall, the user testing phase was incredibly valuable. It validated many of my design decisions while highlighting specific areas where improvements are needed. The overwhelmingly positive reception gives me confidence that Sign Pals is on the right track.

As one tester summed it up: "Game modes such as these could help make people learn very quickly." This reinforces my belief in the power of gamification for educational purposes.

For future enhancements, I envision expanding Sign Pals beyond the alphabet to include common phrases and conversational elements that would be most useful for family communication. I'm also exploring potential integrations with more advanced camera systems that could improve detection accuracy without requiring specific background conditions.

Looking toward future development, Sign Pals has the potential to grow into a comprehensive sign language learning platform with additional features like progress tracking across multiple users, customizable learning paths, and perhaps even community features where users can share their learning journeys. The foundation established in this project provides a solid base for these exciting possibilities.

User Testing Pictures

User Testing Results

https://docs.google.com/spreadsheets/d/18sb0sclPBa5-1-EnukoaE7J_BqwCuGfDJgCAvngYZlk/edit?usp=sharing