User Testing at TEDxBeaconStreet

I previously mentioned a game concept I was working on with my group in T550 and T522. We are trying to address the STEM gap by creating STEAM learning games for kids. Our project presentation for T522 (Innovation by Design: Projects in Educational Technology) is this Friday. While we were getting great feedback from our peers, we really wanted to get feedback from our target users – kids.

We were able to user test this past weekend at TEDxBeaconStreet.

It was a little hard at first because we were competing for their attention. There were so many awesome booths near us – legos, interactive sand boxes, art materials – and we didn’t want to take away from their experiences with those things.

We were able to get feedback from about 15 kids ranging from ages 5-11, so that was good. We had put together a portfolio folder with each screen, and had kids tell us what they thought as we flipped through the pages.

Overall, the response we received was positive. They really liked the concept of being able to upgrade and customize their ship after each level. For the most part, they thought the interface and gameplay was intuitive.

They also gave us some pointers, such as how some of our designs didn’t make sense. In our STEAM lab, we had a screws icon but no screwdriver. They also said it would be cool to get to “choose their own monster” that they battle. We came away with a lot of neat ideas for more advanced levels.

We also learned a lot from hearing their thought processes as they looked at each screen. For one, it seemed no one was really paying attention to the menu bar. They also thought they had to use all the crates on the dock. Things like this give us a better idea of how to change our designs.

Another thing I learned from this experience is the importance of user test survey tools. Ours wasn’t the best. One of our goals with these games is to teach kids about math and science concepts, so we wanted to get a feel for where kids were at.

Most of the user testing was done with one of us asking kids questions, and the other taking notes. We originally started by showing the kids a happy/sad face scale and asking the following questions:

  • How do you feel about science?
  • How do you feel about math?

The first few kids we asked picked the happiest face each time, and we were starting to wonder if we were getting accurate feedback as some were taking a while to respond. We thought maybe they felt like they are supposed to love math and science, and felt like they should feel the happy face no matter what.

We decided to experiment and change our technique. This time, we asked the questions first before showing them the scale. Then we showed them the scale. We got much different results. I think this method allowed students to think about their feelings first, and then that made it easier to choose a face.

The other thing we realized is that we should user test with other student demographics. While there was some diversity in the user demographics at this event, the majority of students there seemed to come from more affluent communities and their parents were very educated. Most of the students we talked to felt confident with STEM subjects and had no trouble figuring out our game play.

Based on the research we did for our project, minority students typically struggle with STEM subjects and we would like to get feedback from them, since those are the students we are trying to help.

This is a pretty fun project and we’re going to try and pitch it to the i-Lab or HIVE.

2 thoughts on “User Testing at TEDxBeaconStreet

  1. This is AMAZING! I was at Tedx Beacon Street on Saturday evening and it was an incredible experience. I am sure it was a terrific platform to test and showcase your work! I am glad you are considering pitching your idea at the iLab or HIVE in the Spring!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s