Table of Contents

Part 1: The Users

Part 2: The Test Process

Part 3: Findings

For Each Task

How many users were able to complete it?

Which areas / UI elements worked well for users?

For All Tasks

Which areas / UI elements were challenging for the users?

In which areas / UI elements did the users make errors?

In which area / UI elements did the users feel frustrated?

In Summary

Tasks/features/activities the users completely ignored (were not able to find/locate).

List several interesting quotes and sentences the users said during the test.

Give a few interesting/surprising examples from the “Think Aloud.”

Part 4: Analysis

Jakob Nielsen’s Usability Goals

Analyze Viber’s usability based on Nielsen’s usability goals.

In your opinion, what is the usability level of Viber (1 hard - easy 5)?

Don Norman’s Interaction Design Principles

Additional Problems

Recommendations

Sources Referenced

Part 1: The Users

Six users were tested, as our group received special permission to include three students. Two users were middle aged (users 1, 4) and the rest were in their twenties (users 2, 3, 5, 6). Three males and three females were tested, the middle aged users being one of each. The middle aged users work as an editor and banker and have about seven years of experience with smartphones, but considered themselves not very tech oriented (users 1, 4). The twenty year old users are a top waiter, and architecture, engineering, and psychology students with jobs and internships who have used smartphones for a similar or longer amount of time than the middle aged users, and considered themselves very tech oriented (users 2, 3, 5, 6). Users came from the US, Switzerland, and Israel.

Part 2: The Test Process

Recording equipment was set up before the users entered the room. Users were read the introduction and understood that the user testing was for the app’s design rather than their smartphone skills. Users were also asked the warm up questions and their answers were recorded and documented. Next, users were asked to do each task separately, and then rate the task on a 1 to 5 scale, from hard or frustrating being a score of 1/5, to simple and enjoyable being a score of 5/5. During the test, users were reminded to “think aloud” and interesting quotes were documented. As each user was asked to rate each task before moving onto the next one, satisfaction level was determined for each task. After the test, users were asked the summary questions to understand their general level of satisfaction with the app.

Part 3: Findings

For Each Task

1. How many users were able to complete it?

All users except for user 4 completed the tasks without help, albeit some with frustration, and rated each task after completing them. User 4 did not complete task 5 (stickers) because he could not figure out how to get the stickers.

1. Which areas / UI elements worked well for users?

Task 1 (finding and downloading viber), task 3 (sending a message), and task 6 (making a call) for the most part worked intuitively and gave satisfaction to all users. After this, users had different levels of satisfaction with UI elements. For task 2 (signing up), users 3, 5, and 6 enjoyed the quick and user-friendly process. For task 4 (viber out), user 4 was pleased that this was an option.

For All Tasks

1. Which areas / UI elements were challenging for the users?

No UI elements were challenging for half of the users (users 2, 5, 6).

Task 3 (sending a message) was difficult for some users, as they had trouble figuring out which of their contacts had the viber app.

Task 4 (viber out) was hard for some users to find. For example, some users first went to the main menu where they may only buy viber out credits but did not have the option to send a viber out call to a contact; they found the viber out call menu option within a contact page afterward.

Task 5 (stickers) was challenging for some of the users for different reasons. One user could not view the preview stickers within the sticker packages as the images did not show up on her phone, and it took her a long time to decide which package to download as she was deciding by the main image (user 1). Another user could not find the stickers for two minutes and eventually gave up on the task even though he had finally found them (user 4). Further, a user had trouble figuring out how to send a sticker to a contact. She first tried to send it within the sticker market, then left the app on purpose, then re-entered the app but instead of looking through the Chats tab to send a sticker to someone she had already tried to call via viber out, she scrolled through the Contacts tab (user 1). Users who had trouble finding and sending stickers were only the middle aged users.

1. In which areas / UI elements did the users make errors?

Most users did not make errors. Certain users made an error on task 6 (call a contact) and called through Viber Out first (users 1, 2). Both users who made this error hit “back” right away, and then completed the task without further errors.

1. In which area / UI elements did the users feel frustrated?

In task 2 (signing up), half the users did not like that viber sent the code through a phone call and were frustrated when trying to answer it. They felt invaded from the way they saw that viber found all their contacts because they did not understand that they were giving the app these permissions (users 1, 2, 4). In addition, one user thought that viber should not have to ask his number, as he had downloaded the app onto his phone (user 2). Another user was frustrated about giving viber access to his microphone, as he was concerned about giving the app permission to listen in on his private calls (user 4). Lastly, some users were unsure about whether to input their real name or a pseudo name when signing up.

Task 4 (viber out) was frustrating for many users as they did not want to pay for Viber Out. The issue was not only being asked to pay for this feature after attempting the call; most users were upset about the idea of “free to download; pay to use” features of any app, and would rather they not be an option at all. Further, most users felt that Viber Out was an unnecessary feature.

Task 5 (stickers) was frustrating for most users for multiple reasons. A main reason was the confusion over downloading the Rude Rider sticker package and not understanding that they had downloaded it when the app referred them to the Rude Rider game in the app store. Other reasons were that the sticker market was hard to find (user 4); the stickers were “ugly” and a “nonsensical” idea (user 4); the stickers had to be downloaded (user 6); the default sticker tab was not an option for “most downloaded” (user 2); and there was no search button within the Sticker Market (user 2). In fact, user 2 spent more time looking for a Sticker Market search in all sections and menus of the app than it took him to scroll for the Rude Rider package after giving up on the search.

Task 6 (call a contact) was frustrating for some users because they could not figure out the difference between Viber Out call and Outgoing call options - and when they figured out the difference, thought the Viber Out option was unnecessary and were upset at the confusion. Also, some users were frustrated in the delay when speaking to a contact through Outgoing Call.

In Summary

1. Tasks/features/activities the users completely ignored (were not able to find/locate).

All users except for one completed each task. User 4 ignored task 5 (stickers) after being unable to find the Sticker Market and then the Rude Rider sticker package for two minutes.

1. List several interesting quotes and sentences the users said during the test.

In task 4 (viber out), user 2 did not like seeing the word “credits” and used the phrase “spooks me.” User 6 called Viber Out “unnecessary.”

In task 5 (stickers), user 2 did not like that downloading the Rude Rider sticker package sent him to the internet and then to the app store. User 2 said the sentence, “Oh. Oh my god. This is a 1,” the number 1 being the lowest score on usability. User 3 said, “Haha do they think I buy stickers?” User 4 said, “Who on earth could like those stickers? Ah maybe my children.” User 5 called the Sticker Market “tacky” and said, “What was Vber thinking? I’m sticking to Emoji’s.”

At the end of the test, user 2 uninstalled the app while saying “fuck you.”

1. Give a few interesting/surprising examples from the “Think Aloud.”

In task 2 (signing up), user 4 said, “everyone just wants to know all my details, do I have to accept?” and “If i give access to my microphone they would be able to record all my private conversations, right?”

General think alouds included “Easy, easy, easy” from user 3, “Boring, boring, boring” from user 5, and “Oh no what did i press now, how do I get back?” from user 4.

After the test, user 3 said, “what’s the challenge, this is the same procedure as with any other communication app.”

Part 4: Analysis

Jakob Nielsen’s Usability Goals

1. Analyze Viber’s usability based on Nielsen’s usability goals.

Nielsen’s usability goals include learnability, memorability, efficiency, errors, and satisfaction. Learnability is how easy it is for users to accomplish basic tasks the first time that they encounter the design. Learnability for tasks 1-3 were quick for all users. Learnability for tasks 4-6 were more difficult, as users had confusion between making a free call and credits call, or with finding stickers.

Memorability, according to Nielsen, is how easily users can reestablish proficiency after returning to the design after a period of not using it. As this project only covers a one-time testing, memorability is ignored in reference to usability.

Efficiency refers to how quickly users perform tasks once they have learned the design. Most users’ efficiency was high (quick), as the app was intuitive to them. Users who had low learnability did not necessarily have low (slow) efficiency: even though they had trouble figuring out how to perform certain tasks, it did not take them long to do so. Efficiency was lower (slower) for the middle aged users, as it took them longer to perform each task. In general, efficiency for tasks 4-6 were lower (slower) than for tasks 1-3. The default options for tasks 4-6 did not give the best experience for the users, and there were increased chances for them to make mistakes.

Errors in usability include how many errors a user makes, how severe they are, and how easily users can recover from them. Most errors occurred for task 6 (call a contact), as users attempted to call through Viber Out before finding Outgoing call. Both users who made this error hit “back” right away from a page asking them to buy credits, and then completed the task without further errors. This error was not severe, but it may have been if the users had already bought Viber Credit and the calls went through.

Satisfaction refers to how pleasant it is to use the design. All users were asked in the test summary what they thought of the app, how they would explain it to someone else, and if they would recommend it to a friend. One user was excited about the app, kept it installed on her phone, and said she would rave about it to her friends (user 3). Three users had moderate satisfaction with the app, but nonetheless kept it installed on their phones and stated that they would only use it occasionally (users 1, 5, 6). Two users had very low satisfaction with the app and uninstalled it immediately after testing (users 2, 4).

1. In your opinion, what is the usability level of Viber (1 hard - easy 5)?

The usability level of Viber among our user testers included scores ranging from 1/5 to 4/5, with an average score of 2/5. In our opinion, the usability level of Viber is moderate.

Don Norman’s Interaction Design Principles

Don Norman’s interaction design principles include affordances and signifiers, mapping and conceptual models, and feedback. Affordances occur first when interacting with a design. They are an intuitive mental model, and one’s first hint for a quick decision in what the machine is supposed to be used for (before any action). Affordances are not designed; a UI expert must tap into existing affordances that are acquired from early life based on people’s subjective mental models. Affordances in Viber are few, and may be part of the reason for many users’ complications in using the app. On the testers’ parts, the moving swipe motions for picking up or ignoring a user’s call had an affordance aspect (they were in fact a cross between an affordance and a signifier): While nowadays, mobile swipe tasks are a signifier, this viber design is an affordance as well because the swipe tasks are in intuitive motion in the directions that they must be swiped. Sequential affordances may have occurred at the beginning stages from downloading the app to inputting one’s phone number to receiving a self-activating code and finding one’s contacts - and this may be part of the reason that the user testers had fewer issues with beginning tasks than with later tasks.

Signifiers are learned conventions that have been accepted in the digital world (or in the physical world if none apply in the digital world). Viber uses signifiers from other voip and chat apps. First, the Viber icon is a signifier for a couple of reasons. The image of a phone is a signifier for the type of phones people used before mobile phones and demonstrates calling; and the speech bubble is a signifier for messaging. Google Hangouts, Facebook Messenger, and Whatsapp are chat apps that use the speech bubble in their icons; Whatsapp is also a VOIP app that similarly uses the phone image in its icon. The signifiers in Viber’s icon hint at Viber’s purpose, and most users had an idea of the type of tasks that would be expected of them whether they had heard of Viber or not. Like Whatsapp’s main “chats,” “contacts,” and “calls” tabs, Viber has similarly named tabs on its main page: most users use Whatsapp and did not get lost when attempting to find something within those tabs in Viber. Like Line and Messenger’s stickers and sticker packs, Viber has a Sticker Market which users understood if they had used stickers from Line or Messenger in the past. Like Skype’s Skype Credit option, Viber has a Viber Out credit option. However, users only understood the Viber Out option if they had not only heard of skype credit, but had ever used it as well. Most users that we tested had either never used this credit convention or thought it was unnecessary, and were challenged by, made errors in, or were frustrated by the Viber Out task. Like Snapchat’s timed picture and video option, Viber has Viber Wink (although this feature is outside the bounds of this assignment). Like general mobile phone and voip apps, Viber has a plus button and number pad to add or dial phone numbers; regular text, voice message, call, and facetime (video call) options; and a conventional search button above the Chats, Contacts, and Calls tabs. Viber also includes a main menu that is situated conventionally at the top corner and is shaped with the conventional three horizontal lines; as well as sub menus within contact pages and contact chats, with icons situated conventionally at the top right corner and shaped with the conventional three vertical dots that symbolize “more.” The menus included essential things that users wanted to find, and speeded up efficiency. While learnability may have been slow for some users, efficiency was not as low. Viber’s heavy use of signifiers may have helped most users quickly identify the available options on each page.

Mapping is the relationship between a control (a user’s action) and its result (the system’s function). Viber’s mapping is mature. The in-app buttons did what they were supposed to do. Pressing numbers on the keypad sent the same numbers by the system. Pressing a tab moved the app screen to that page. Pressing the main menu shot it out from the side. Pressing a sub menu opened a list from the icon. Pressing any button brought the user to the correct feature. One exception within the app was the Rude Rider sticker package “download” button. The download button did what it was supposed to do, but it also brought the user to the app store to download a game at the same time - mapping that the users did not know they were getting when they pressed “download” for the stickers. When pressing phone buttons to control the app, however, mapping sometimes went awry. A few times when users pressed the phone’s “back” button instead of Viber’s back button, the screen either did nothing or exited the app, when the user had wanted to go to a directly previous and increasingly main screen within the app (such as from a friend’s contact page to the list of all contacts in the Contacts tab).

A conceptual model is a simplified explanation of how something works. It does not have to be complete or accurate as long as it is useful. It also does not have to be written in a manual or spoken aloud as long as it resides in the minds of the people who are using the product. From the picture in the app store, setup, and main screens, Viber mostly has an intuitive conceptual layout. However, different people may hold different mental models of the same item, which may explain why some users had no trouble navigating the app and others had more difficulty. Also, Norman explains that the same person may have multiple mental models of the same item, each dealing with a different aspect of its operation: the models can even be in conflict. Users had different mental models of Viber’s Outgoing call and Viber Out call features, and while learning the app these mental models were often in conflict. Many users could not tell the difference between the two types of calls during the learning phase, and had trouble with tasks 4 (viber out) and 6 (call a friend). Some searched in the main menu to make a Viber Out call instead of looking for it from a contact page. Users were also frustrated that the two ways to call people were not more clearly differentiated in case of errors. Users had poor mental models of the app for other reasons as well. Some users mentioned that they did not understand the link between the app’s name and its use. One user also mentioned that in the Sticker Market for task 5, the “top” stickers tab provided a poor conceptual model of the tab as all the stickers within it cost money - and “top” implies “most downloaded,” as opposed to “featured” (user 2). Most users had much trouble on task 5 (stickers) as well, and one user could not complete the task as he could not find the Rude Rider sticker package after two minutes of searching (user 4).

Feedback is communicating the results of an action. Feedback must be immediate, because if the delay is too long, people often give up. Viber’s feedback was timely and worked to reduce errors and frustration. All buttons that were pressed did their task without delay, and users immediately understood the results of their actions. Feedback must also be informative, because if the feedback is distracting, uninformative, or anxiety-provoking, users may ignore or turn off all notifications and miss the important ones. User 2 called the feedback from not having credits “sly feedback” because the app barely let him see the “you have no credits” screen before it sent him to the “buy credits” screen. Feedback must be prioritized so that important information is presented in a way that captures attention, and so that message coding is compatible with other systems. Viber prioritizes messages so that everything is quiet or hidden in the phone’s notifications tab except for calls which ring (or messages which pop up, but a feature that users can turn off). Viber also gives informative feedback, although its message coding is not fully compatible with the “back” button of multiple users’ phones. Lastly, feedback must be planned so that all actions need to be confirmed, but in an unobtrusive manner. While Viber gives a list of permissions it will take when users install the app, most users felt blindsided when Viber called in a code instead of texted. Users felt blindsided again when it send them to the app store as they were attempting to download a sticker package from within the Viber app. The poor feedback made all users think that the Rude Rider game in the play store was meant to download the Rude Rider sticker package instead. For some reason, most users also felt invaded from the way Viber showed them that it had all their contacts after users input their phone number. Users had forgotten that they had offered this permission, and wanted the feedback to have been more planned so that they had a better idea of what they were letting the app do. While most feedback was immediate, informative, and prioritized, users were disappointed in some of Viber’s planned feedback features.

Additional Problems

Additional problems in usability testing include confusion for some users in how to view only contacts that have the Viber app. The sections are displayed vertically with content in between them. A user with many favorites must scroll a lot to reach the Viber Out credit section, the You section, and the Contacts section underneath. The “All / Viber” subtabs down in the Contacts section are so low down that it was hard for most users to see that they had the option to reduce the view to only contacts who have the Viber app. This caused frustration to the user testers, as many did not figure out that only viewing contacts who have the Viber app was an option.

Additionally, some users did not like the color palette (user 2).

Lastly, it is not clear how to delete sticker packages. There is a settings option to turn a sticker package on or off, but it is not clear that there is a setting to delete them. Two sticker packages come with the app, and do not seem to have the option to be deleted. In fact, both of the sticker packages that come with the app take up valuable real estate on the sticker bar within the chat box, and when tapped send the user to the Sticker Market to download or buy them. This entire experience was intensely frustrating for the users who tried these options, and they lost trust in the app.

Recommendations

One recommendation to let users easily figure out that they can differentiate their Viber contacts from their other contacts would be to display the “All / Viber” subtabs as a priority section in the Contacts page.

A second recommendation is that the first tab in the Sticker Market should land on “Most Downloaded,” and not to “New” or “Top” (recommendation from user 2). Alternately, another user recommended to remove the stickers entirely from Viber (user 5).

Lastly, Viber Out should be featured only when contacts are unconnected to the app (either because they do not have it or because they do not have internet at the time). Otherwise, the option is redundant and makes users upset about an unnecessary feature that they might be charged for if they make an error.

Sources Referenced

Nielsen, J. (2003). Usability 101: Introduction to usability. Retrieved from <http://didattica.uniroma2.it/assets/uploads/corsi/143228/Nielsen_5_articles.doc>.

Norman, D. (2013). The design of everyday things. Basic Books: USA.