Based on the knowledge gained from pilot study, 2 phases of the system were decided - Learning Phase & Game Phase (for practice)

It was important to test our idea in a low fidelity prototype. So, I simulated my living room.

Low fidelity prototype was made to check the validity of our idea. A real room was simulated as our virtual environment. Objects had been labelled with Swedish text. Wizard-of-oz technique was used for this user study. 3 participants were asked to explore the room. A soft music was on in the background. As the participants explored the room, they saw textual cues (other language) over some object. When they looked at it, the music stopped playing and they heard an audio cue (other language) saying the object’s name. After this exercise the participants were tested.

Result of the Study: All of the participants remembered the object’s name in the other language very well. They liked the music as well. One of them asked "How will you make a person walk in VR?"

The next quest was to find the perfect interaction tool with the VR system.

The question asked by one of the participants made me think about interaction. Interaction was difficult (Remember, this was late 2015 and VR was very new. The only easy way to interact with VR environment was by using the keyboard, which was not at all user friendly). We explored two options - Leap Motion and Myo Band.

Myo band was decided to be used over leap motion as it was a better input for VR environment. Medium fidelity prototypes, also known as digital prototype was built to find the user’s reaction to VR environment. An open VR environment was created.

User Study: I conducted an informal user study with 3 participants. The users were asked to move in VE by wearing Myo band. They were asked to look at a cube and read the word displayed. We tested our hardware (Oculus and Myo) in this environment.

User Feedback: The users felt uncomfortable navigating in the VE wearing Myo band they complained that the gestures used for navigating were difficult to maintain in VE. This was an attempt to find the easy of interaction with Virtual environment.

I created the 3D apartment environment

I created the 3D environment. It was an apartment having living room, bedroom, kitchen and balcony. The space was created so that the user can move around and interact with the objects. 10 objects were placed in the apartment. Myo interaction was also built. We tested this system on 3 users.

3 New Participants were asked to use the VR system. Some notable user feedback were -

• “Music is good, but should be instrument - no lyrics” The user liked the concept of music but suggested it should be instrumental as it will not interfere with the pronunciation of the words.

• “Oculus is too heavy”. The DK2 headset was heavy and wired. This caused discomfort.

• "Need for practice environment”. The user felt very uncomfortable directly dealing with the new VE without any practice. A need for practice environment became necessary.

• “Objects don’t need English translations, but should be easily recognizable”

• “Slower pronunciation” The pronunciation were fast for some users.

A crucial user feedback was taken into account

The following changes were done based on user feedbacks.

• Instrumental Music was added as in the learning environment.

• Practice Room was created. This was an empty room with a painting on the wall as seen in Fig 3-10

• Easy to recognize without English translation – 10 Swedish objects were chosen.

We decided to do a pilot study run before conducting the major user study

Final User Testing was done before Data collection. One participant tested the system. After using the system, a small assessment was done to check how many words he remembered. Below tables shows the performance of that participant.

Overview of the System

• Phase 1: Learn how to interact with VR environment using Myo Band and Oculus rift in the practice environment.

• Phase 2: Freely roam a virtual apartment space. When looking at a relevant object, text will be shown of what it is in the Swedish language along with audio cues.

• • Phase 3: Game session in which user is given the word and they must go find the object.

Final VR System was made as per user feedback. The interaction using Myo band was adjusted as well to keep the system user friendly.

I designed the user study (A/B Testing) in the following manner.

• There will be two groups of participants. One group will study the word using traditional flashcard method. Other group will learn words using our virtual reality system. (Reason: It was important to measure the effectiveness of VR system over traditional method to prove the hypothesis)

• Then immediately after learning the words, the participants will give a test.

• After one week, the participants will be called and asked to give the test again.

• After completing the test, they will fill out the questionnaire & give informal feedback.

Student volunteers were chosen as participants for the user study

36 student volunteers were chosen as participants out of which nineteen participants successfully completed the user study. They were randomly placed in either the traditional or virtual reality group.

Traditional group participants were first given a list of ten Swedish words and their English equivalents and told to study them for up to five minutes. The words were read out loud to them, repeated upon request. Then, they were given flash cards corresponding to the words, and told to study them until they felt they were ready for the test.

Virtual reality group participants entered a virtual practice environment where they could become accustomed to navigating virtual reality. Then, they entered the virtual apartment and were allowed to explore freely for up to five minutes, looking at objects to see and hear the corresponding Swedish words. Afterward, they entered the second phase, in which the Swedish words were displayed and pronounced and participants had to point to the corresponding object, until each was complete.

Experimental Results - Objective Results

We measured the scores from the initial test (the test given immediately after the study) & post test (the test given one week later). We also calculated the word memory retention.

Subjective Results

Ratings were much higher for virtual reality than for the traditional method, with the majority of participants rating virtual reality as “Very Enjoyable” & "Very Effective".

What did we learn?