MetteTesting

Do they understand the game?
Is the point system enough motivation to visit the websites?
Do the hero and villain character have any value?
If they are the villain do they still want to do the missions?
Do they really enjoy this version of the archive more than the original?

What are you going to test?

I want to start with making a paper version, not 3D yet but just images. After this I want to make a small 3D version and make it into a website.

I want to test it preferably in the Awesome Space in Utrecht.

Who will be your testers?

The members from Awesome space Utrecht that want to participate. And the most different kind of people.
(Age, gender, job etc.)

How will you document your test?

I want to make a voice memo, and I want to film/photograph their (hand) movements. Also I would like to keep an eye on their facial expressions, and if I have the time, write this down.

What are you going to test?

Maytal gave me the feedback that I should write more technical usability questions, and less conceptual questions.

- Open the original archive
This is the current web archive. We searched for a retro gaming related website.
1. What would you do if you had to open it?
2. How would you go to another year?
3. And how would you go back to the archive?
4. What are your first impressions of this website?

My test questions:
1. This is the homescreen of the game, you can read the introduction first.
2. What would you do if you had to start the game?
3. What would you do if you had to move forward?
4. Now if you had to go around the corner?
5. (spirit pops up) What do you see?
6. What would you do now?
7. And if you had to grab the heart?
8. Did you see what just happened?
9. What direction would you go to now?
(Did you see the dead end?)
10. What would you do now?
11. You have to read what is on the screen on the wall,
what do you do?
12. You can read the mission.
13. We’re not gonna do the entire mission,
but how would you enter the mission?
14. How would you go back to the year 2012?
15. So imagine you did the task, and now you have to go back
to the archive, what would you do?
16. Can you tell me what you see, or what has changed
now that you won the mission?

Which prototype are you making?

For the prototype I am going to sketch out the frames and scenario's, and put these in Adobe Xd. I'll also add the text and buttons that actually work as well. I changed my plan because Maytal gave me the feedback to really make it as simple as possible, it's only to test the basic usability, it doesn't have to be a pretty mockup already.

How will you document your test?

Instead of making an audio memo and notes, I'm going to make
a checklist before hand, so I can check the box of what is happening, and make the possibility to take notes. And I would like to take pictures of the testing proces of course. The sheet with the checklist is going to be for every user one,
So I can fill in their name, age and other information if necessary.

Which prototype are you making?

Where will you test?

Sadly we are not allowed to go into the Awesome Space building. So we had to come up with a different approach. We decided to make the prototypes digital, so that we can test it with the Awesome Space members one by one, so they don't influence each other, and on Zoom. For me this works out pretty well, because my game/interface is supposed to be used on PC, at the users home. So they are in the right environment already while testing the interface.

Who will be your testers?

Five Awesome Space members.

When asking Kristian to go to another year on the CURRENT archive, he gave a completely different answer than we expected. He said he's go to the captures link, and then navigate from there. Because the timeline didn't work specific enough for him. This was because there wasn't for example specifically June 21st to click on, but only a graphical timeline without numbers/dates.
Apart from that the user experience was clear to him, and he did mention that this archive won't be hard to improve.

Kristian saw the spirit, and I asked him what he would do. He answered he would click on it (trying to attack it in some way). I knew this wasn't possible, so did it anyway, and of course nothing happened. When I asked him what he would do then, he said click on the arrow to go back. (The "good" answer.)

When I asked him on the intersection which way he would go, he said he would normally go and explore the dead end. And also he noticed that the navigation had changed (on the map) so he knew he was facing the intersection from the beginning the other way. Eventually he decided to go to the right (the "right" answer) because I told him there is really nothing to see in the dead end. And keeping in mind the time, this was okay.

Testing with Kristian

A helpful comment that he made was that he said that instead of only placing a check mark on the screen, maybe I should also change the color of the screen. This is very helpful advice! It makes perfect sense that it's more clear that you have achieved or completed something if the color also changes.

Overall he said he liked the gameplay/challenge aspect of the interface. Because you have to do a challenge in order to navigate through the archive. He also said the first impression of the game was that it reminded him of Wolfenstein 3D, which was my goal. But I don't want it to look like that game too much so I have to keep an eye on that.

Accomplished feedback

Testing with Dervis went pretty well. When showing him the current archive and asking
him how he would navigate to different years, he said he would probably click the arrows next to the year. But preferably he would want to search on a year and date, like with a drop down menu.

At the beginning I asked him what he would
do if he would have to go forward. He replied that he probably would go to the left, because when he looked at the map he saw that was the most logical route to the cross. This makes sense, and I'm happy he noticed the map and immediately made use out of it. But of course I still needed him to go straight forward
for the encounter with the spirit.

When he encountered the spirit he said he would go back, so he didn't try to attack
it like Kristian did. He did say that he saw
a window at the background, which is the screen. But it is still white so I'll have
to change that to match the actual look of
the screen (black and green with
a question mark).

When he entered the mission, and was in a website of the archive, I asked him how he would go to another year. He seemed a bit confused, and said that he would go to the payback machine (so I think it wasn't completely clear that he was already in the payback machine, I should put the logo there as well).

After a while he noticed the year 2016 in the top of the screen, he had to laugh a bit himself and said that he now sees the huge 2016 on the screen. So he seemed to be surprised that he overlooked it. He said he was looking at the bussy website so much, he didn't notice the year.

At the end I asked him about his first impression, and he also said the gameplay aspect is really fun, and that the challenge really engages you. Also he said the design is very minimal and clean, and probably his mom could even play this game.
Which to me sounds like he thought the usability was great! So this was a huge compliment to me.

Because of privacy reasons I'm not uploading the video of the testing with Tjeerd. However I do have the testing from on the right, and I'll write about what happened during the testing of the prototype. For this testing round I used the newest designed version of the prototype.

With the spirit encounter it took a while for Tjeerd to notice that the health bar went down. He saw the spirit first, then the arcade machine in the background, and then he saw the health bar when I said 'on the left side of the screen'.

When I asked him what he would do now that he saw the spirit, he said he would try to attack it by clicking it or moving forward. Then he said he would click the arcade machine, or hit the spacebar. So to go back with clicking the arrow didn't occur to him.

At the intersection I asked him what direction he would go to, which would tell me if he saw the map and if he knew how to use it. At first the Zoom bar was over the map, but when he replaced that he answered he would go back, to go to the cross on the map. I went back to show him he would run into the spirit again.

He noticed the spirit, and then answered to go back again, and to the right.

Once we were at the part where I asked him what had changed now that he achieved the mission, he noticed that the colour of the screen had changed color. But he also said that after playing another mission/visiting another website he would probably not remember what colour it was before. So he advised to put a check mark on the screen, which was there in the first prototype.

For first impressions he said the navigation was a bit confusing. So for example in the end screen you are pretty far away from the next corner, but the next click would be to the left, he would try to move forward one more time before going to the left. Ofcourse this is something that has to do with prototyping, because for the actual game I would make the navigation so that the longer you hold/click the arrow the longer you would walk that direction, and when you let go you stop walking.

He also mentioned the arrow that is pointing down could be a u-turn instead, so that it's clear that you turn around.

Emiel was already quite familiar with the
web archive, he used it before. But whenever he typed in a URL that he was looking for,
he wouldn't find the website he meant.

While testing my prototype I asked him what he would do if he had to move around the corner, and he said he would try to walk forward a bit more first (so the same issue Tjeerd was talking about), and then go to the left.

When he was at the spirit encounter, he did not notice the health bar, (like Tjeerd also). But when I asked him what he would do next,
he did respond that he would try to move backwards.

The testing went quite well, something interesting he mentioned was that when he had to grab the heart, he wouldn't click on it, but walk over it like in Minecraft or 3D shooters. In these PC games to collect items you have to walk over them. This might be something to keep in mind.

After he grabbed the heart I asked what direction he would go to to see if he
noticed the map.

Instead he answered he would go the right,
but because he wanted to know what the arcade machine was about, which he remembered from the spirit encounter scene.

In the end when he achieved the mission,
I asked him if he could tell me what changed. He said the screen turned green, but also
that the health bar was now purple/pink.
I explained to him that his was purple the whole time, except when he grabbed the heart and the health bar went up. He agreed but I think he did not notice the health bar before.

He didn't want to give a first impression immediately because he wanted to see the other interfaces first, so he could compare them.
I thought this was pretty funny but of course was not really the point, but I guess that is what happens when we test all of the interfaces of our group at once.

In the end he did mention that he liked that
we all have different approaches, and that
he would have to use the archives a while first, before choosing which one he
likes best.

Roel was also already familiar with the web archive, and that you can go back in time with it. For privacy reasons the video of testing with Roel will also not be uploaded.

He encountered the spirit, and said what he noticed. After this I asked him what he would do next, and he answered he would turn around and go to the right towards the cross. So he already planned his route towards the arcade machine in the background. This means the map was clear to him, and the goal.

The testing with Roel went really well, so there isn't a lot to write about. He did mention something very interesting in the end. I asked about his first impressions, and he said first of all that it was a fun concept. He called it an old school dungeon crawler.

He also said he was a little surprised with the spirit turning up. He liked the spirit, but it didn't give him any indication of the moment that it might struck him down.

Old school dungeon crawler game

He said so far it seems kind of forgiving, but maybe in the long run when you're really in to it and it's your last bit of health, and you get a ghost turn up out of nowhere that might be a little frustrating. So maybe it should be on the map, or other ways of indicating that it's coming.
I thought the small square before you turn around the corner where the spirit is hiding, might be a hint enough.

Some of the testers tried to attack the spirit instead of going back.

Instead of placing a checkmark in the achieved screen, I might also want to change the color to make it more clear.

The website in the arcade machine/mission was so bussy that one of the testers didn't notice the year in the top of the screen.

One of the testers didn't quite understand that the mission he was in, was also the payback machine. So to make this clear I placed the logo in the bar at the top as well.

After testing with prototype 1, I tested in the second round if only changing the color of the achieved screen to green was enough of a indicator that you achieved the mission. But one of the testers mentioned I might want to use a checkmark on the screen as well.

Some of the testers didn't really spot the health bar, or the health bar going down when they encountered the spirit. They only saw the health bar go up and turn green when they grabbed the heart.

Testers still tried to attack the spirit instead of going back.

One of the testers mentioned I might change the arrow that is pointing down into a u-turn symbol/arrow. So that it is more clear that you turn around and walk the other way.
To grab the heart some testers tried to walk over it to grab it, because this is how you collect items in some other big PC games.
The fact that there is no indication when the spirit pops up can get frustrating, when you are far in the game and don't have much health left.