#25. MJ’s Thesis — User Testing

<hello,w>
5 min readMar 28, 2020

Understanding actual First-year undergraduates and high schools.

This is documentation of Minjung Kim’s thesis in Interaction Design (IxD) from School of Visual Arts, New York, USA (Sep 2019 — May 2020).

Context
I added a training part that uses AR to bring to enrichment resources for female students who want to pursue engineering careers in the U.S. I’ve conducted User Testing with four participants so far. I will add over five more observations by this week to look out the AR ideation.

Location
This week’s status quo: Conduct UT, and Gather Feedback
As I’m considering the AR aspects as a part of self-training by using their surroundings, I would love to show a low-fi clickable prototype of it.

Insights
1. People like the concept of providing a basic AR quiz about engineering which tells them what they could be good at and still want to reach out to mentors after playing it.

2. Setting a time limit would make it fun because it feels like there’s a challenge. But if the user runs out the time, it shouldn’t be GAME OVER. Instead, I can show the female mentor’s hint notifications of help.

FIG. 1.1 User Testing Organizing Chart by Minjung Kim

Figure 1.1 illustrates the results of post-test with insights and quotes from the real users who used the first version of <hello,w> prototype. Taking a look at qualitative metrics to understand the reasons behind some issues and discover how to overcome them was the main purpose. Three color codes, blue, yellow, and red represents each cell based on severity. Yellow color illustrates new critical ideas that came from participants for completing a task. Red color means some serious issues which are frustrating for most participants. Blue color identities minor issues which can be irritating but won’t drive users away. Overall, people gave positive feedback about the concept of providing basic engineering AR games which tells them what they could have interest in from now on and understood well reaching out to mentors after playing the fundamental games. And setting a time limit would make it fun because it feels like there’s a challenge.

Gathering feedback from real users is a crucial part of this new prototype for a mobile platform, especially for the next generation. The way in which UT(Usability Testing) can gather the feedback varies depending on what kind of questions the researcher and observator are trying to answer and act. Qualitative usability testing is widely considered to be one of the most effective user research methods for gathering feedback, due to the great amount of insight this can gather from such a small group of users. This UT method focuses on observing users behaviors and better understanding their decisions by asking a series of follow up questions about a clickable prototype of <hello,w>. For this reason, having an understandable and interactive live digital prototype for them to interact with and representative tasks to complete is essential.

Based on research, adding a training part that uses AR was decided to bring to enrichment resources on the mobile application for female students who want to pursue engineering careers in the U.S. This UT conducted by Minjung Kim with six participants. Considering the AR aspects as a part of self-training by using their surroundings and sharing opinions and personal challenges with mentors while pursuing engineering careers as a female student were the key observations. The UT data has been analyzed by providing a low-fi clickable live prototype by Minjung Kim.

Research observations

FIG. 1.2 User Testing Observations about AR training game by Minjung Kim

Overall

1. “The overall experience is quite intuitive and interesting. One thing I would include in games is linking a URL that you can learn about more courses related to the games. It helps to teach you the skills. You were introduced to students.” “It seems a good app to encourage women especially in engineering.” — Aditri F, 21, Carnegie Mellon University Student

2. “I really love the concept of it and everything, especially how a simple game can tell you what you could be good at. For someone who is still unsure/not confident like me, something like that helps to boost my interest in engineering and motivation to pursue it.” — Yuka F, 18, High School Student

3. “It feels more interactive and it helps you feel more connected.” “It’s easier because it compiles all the different resources into one app and also mostly focuses on female mentors. Because on Instagram or Facebook you will have to search for that community and you don’t know if they want to get random questions or not.” — Lou F, 19, Georgia Tech, Neurosciences, Second year Student

How would you compare <hello, w> to competitors?

1. “It feels more interactive and it helps you feel more connected.” “It’s easier because it compiles all the different resources into one app and also mostly focused on female mentors. Because on Instagram or Facebook you will have to search for that community and you don’t know they want to get random questions or not.” — Aditri F, 21, Carnegie Mellon University Student

AR training

1. “So far, it looks really fun because it’s interactive. I’m wondering how the surrounding is incorporated.” — Yuka F, 18, High School Student

2, “The icons and labels make sense.” “A little info button would be added to know more about this object.” — Aditri F, 21, Carnegie Mellon University Student

3. “I’m assuming that if I will click the items under More Resources, it will extend and show more details.” — Yuka F, 18, High School Student

Virtual Video Mentoring

1. “If I can see some relationship between a mentor and her role model, I can understand how Joy influenced others.” Jess F, 17, High School Student

2. “Editing option in my question is what I exactly wanted to do. I would edit my question just by tapping the bubble. You may want to add tap to edit text somewhere in the bottom.” — Yuka F, 18, High School Student

3. “I have one question though. I shoot the message to a mentor, is there any like a chatroom where I go to see my questions and responses?” — Yuka F, 18, High School Student

--

--

<hello,w>

A platform with AR games and mentors encourages young women to pursue engineering career paths