Goal: the overall purpose of the evaluation

• The goal is to figure out drivers to achieve safer and more efficient rides by using DriverPal.

Three key questions: different aspects of the working prototype you would like to evaluate

• Is the app easy to use and navigate around different pages?

• Are the driving instructions clear and easy to understand?

• Are the summary and guides useful in achieving safer and more efficient rides?

Describe the step-by-step test procedure in three sections: pre-test, test, post-test

• Pre-test:

  • Ask questions about their driving skills. Do they have a driver’s license? When did they get it? On a scale of 1-5, how good are their driving skills? Are they looking for ways to improve their driving? What are some approaches they are using right now to improve their driving? Are they working?

• Test:

  • The first task will advise participants to click the “Start Driving” button. There we will simulate a trip and activate the notifications manually. The participants will go through all the notifications, including abrupt acceleration, abrupt braking, weather and road condition change, etc. The participants will be asked about the usability of this feature, is it helpful in correcting behaviors, is it distracting, is it visually appealing, etc. The second task will advise participants to click the “Trip Summary” button. Participants will be given the freedom to navigate around the page and check out the reviews, guidance, and visualizations of past trips. The participants will be asked about the general effectiveness of the feature, are the visualizations and guidance helpful in improving their habits, Is the meaning of the chart clear, Is it visually appealing, is it easy to navigate, etc.

• Post-test:

  • Questions on the general usability and their interest in the app will be asked. On a scale of 1-5, how likely will they use the app? Are the guidance and notifications loud and clear? How do they like the UI design? What other features would they like to see? Etc.

Quantitative Analysis

• Since the evaluation will take place in the form of interviews and focus groups, the data collected will mostly be in the text format instead of numerical form. A few questions will be in the form of a Likert Scale. Those data will be used to perform descriptive statistics to summarize data points in a constructive way such that patterns of feedback on different features might emerge. The most effective way of measuring the impact of DrivePal on users is to run the t-test of MPG before and after the usage of DrivePal. However, given the difficulty of conducting such research, it might not be possible to implement.

Qualitative Analysis

• The log answer from the interview and focus group discussion will be grouped by different questions and features. If one specific aspect of input is mentioned by more than 5 participants, certain steps will be taken to improve the usability of such feature and fix potential problems. Keywords that are mentioned in multiple answers will also be picked out and analyzed accordingly.