Evaluation

Is there a difference in the level of perceived system usability between the purchase logging feature of the Mint system and the StashCash system?

Null Hypothesis/H₀: The mean System Usability Scale (SUS) score of the Mint system’s purchase logging feature equals the SUS of the StashCash system’s purchase logging feature. (μ_M = μ_S).

• Alternative Hypothesis/H_a: The mean SUS score of the Mint system’s purchase logging feature does not equal the SUS of the StashCash system’s purchase logging feature. (μ_M ≠ μ_S)

Study Design: Between Subjects Study

• Conducted in two groups: one group used our StashCash interface and the other group used Mint’s interface.

Young adults:

• 31 Year Old Male

• 29 Year Old Female

• 25 Year Old Female

• 20 Year Old Male

• 17 Year Old Male

• 21 Year Old Female

We used the System Usability Scale (SUS) Survey to evaluate the score of the logging features between both applications.

The SUS Survey uses a 1 to 5 point system for each question. The final score is then Calculated in series of steps.

OddScore = (Sum of Odd Scores)-5

EvenScore = 25-(Sum of Even Scores)

SUSScore = (OddScore+EvenScore) x 2.5

The average SUS Score is 68 so if a score is above the average it is a good score and shows whether a participant found the feature useful

Between-subjects study (6 participants)

• • 3 used Mint

  • 3 used StashCash

4 tasks using the logging feature

• • Add a $20.00 transaction and categorize it as ‘Food’

  • Add a $35.00 transaction and categorize it as ‘Entertainment’

  • Add a $10.00 transaction and categorize it as ‘Coffee’

  • Add a $15.00 transaction and categorize it as ‘Food’

Post-study survey

• • System Usability Survey (SUS)

  • 2 open-ended questions:

    • What was your overall experience of the logging feature?

    • What was your favorite part about the logging feature?

From our 6 participants we used the SUS Scoring to find the value for each participant. The average score for StashCash was 65.833 which is just under the target average of 68. The average of Mint’s interface was 80 well above 68 indicating that the participant’s found the interface useful.

We then performed a t-test for two sample means between the two sets of scores. We hypothesized there was zero mean difference between the two populations. We received a P-value of 0.084750767. The test was not statistically significant at the 95% confidence interval. Therefore we fail to reject the null hypothesis.

Background:

• Participants are young, have limited budgeting experience, and use mobile apps frequently

Codes:

• Level 1 Codes: 14

• Level 2 Codes: 4

Simple and Efficient

• The logging feature allowed for users to have a quick and easy interaction with the application. It wasn’t time consuming and the feature was well constructed to be understandable to new users.

  • “It was immediate and easily accessible on the app”

  • “Nice and simple not too complicated to use”

Additional Categories

• There was discussion on needing the ability to add new categories from the logging feature. Currently users would need to add a new category in a separate tab on the app. This might cause frustration if this is a common occurrence especially for new users unfamiliar with their budgeting categories.

  • “could be nice to potentially add a category from the logging feature.”

  • “add categories within the feature”

• Make it easier to see and use a label, different color, or better icon to help the user better understand what the button does. The users seemed to struggle with finding the button. They suggested it would be better if it was placed in the menu. More testing would need to be done to see where the ideal position would be.

• Create the ability to add a new category from the dropdown menu. This ability would allow users to not have to cancel out of there log and go add another category. Now from the dropdown among the categories there would be an add new category button.

Users were happy with:

• The simplicity of the logging. It was quick easy and simple for users to enter. Entering data was immediate and easily accessible. They also liked that it didn’t require linking to a banking app like Mint’s interface does.

Users were unhappy with:

• They were unhappy with the fact they couldn’t add a new category while making a purchase. There was also difficulty recognizing where the logging button was. They found it was tough to discern that the button at the top meant it was for logging. Instead users wanted to see it apart of the menu bar at the bottom.