Milestone 8: Design, Development, & Testing

1. Final Solution

2. Usability Testing

3. Heuristic Evaluation

1. Final Solution

The final prototype design of UniPath can be accessed via here.

Below highlights the features of UniPath:

(a) Data Security and Privacy

UniPath adopts several ways to secure users’ data. First, the AI algorithm it uses will be hosted within the server of the app, meaning that users’ data will not be uploaded to external cloud AI services. By using in-house AI, data stays within the UniPath ecosystem and is not shared with any third-party AI providers. This eliminates the risk of data breaches or unauthorized access.

Upon logging in, users are presented with a consent page which ensures they understand how their data will be used in the app. It emphasizes that all academic data, such as grades and course details, will be directly drawn from users’ university's LMS, and UniPath does not store or hold any individually identifiable data on the servers.

Also, students have full control over their data usage and can choose which data is accessed to personalize their experience. In the user profile settings, users can opt-in/out of which third-party platforms (e.g. Brightspace, Albert, LinkedIn, Handshake, etc.) and what specific data in that platform they would like Unipath to access. Users can change this anytime on the settings page.

(b) AI to Support Students

UniPath leverages AI algorithms to analyze user data and provide personalized academic, career, and mental health insights. The app uses AI-driven analytics to predict academic performance, suggest tailored study resources, and recommend suitable career paths based on user profiles and previous data. This technology can reduce students’ cognitive load by centralizing resources and simplifying decision-making processes, thus addressing the fragmented nature of existing support systems.

(c) Human Support Supplementation

While AI offers efficient support, research highlights the importance of human interaction in educational guidance (Hunt et al., 2021). To address this, UniPath integrates features like live chat with academic advisors, career coaches, and mental health counselors. This dual approach ensures that students not only benefit from automated insights but also receive personalized human assistance. This strategy aligns with Self-Determination Theory by fostering relatedness and providing a sense of belonging and support.

(d) Outreach and Engagement Strategies

UniPath will be introduced across various schools and departments within universities. University administrators will play a key role in promoting the app to incoming students through orientation programs, newsletters, and workshops. Since research shows that students often do not actively seek out help (Crişan et al., 2015), this proactive outreach is essential to encourage them to make good use of UniPath.

2. Usability Testing

I completed the usability test with 13 of my LTXD thesis classmates, who are all from NYU’s Master of Arts in Learning Technology and Experience Design (LTXD) program. They were presented with a completed prototype of UniPath and a usability test survey to complete.

In the survey, users used the app through the persona of Alex, a 2nd-year LTXD student. As users navigate through the app, they will be asked to answer questions based on Alex’s information, such as his academic progress, career goals, and mental health resources.

The survey consists of four sections: Academic, Career, Mental Health, and Privacy Settings. Each section contains two to three questions that test whether users can find the correct information in the app, along with questions seeking general usability feedback.

The aims of the survey are to:

Ensure app features function as intended and meet user needs;
Detect any challenges or pain points users encounter when completing tasks, and;
Collect feedback on the app’s design, layout, and content to make necessary refinements.

The usability test score result is shown below:

The above table shows that the overall performance of the app was satisfactory, with an overall usability score of 7.2/10. However, certain areas highlighted specific challenges that need to be addressed to improve user experience.

(a) Academic and Career Sections

These two sections scored relatively well, indicating that users could navigate main features such as course exploration, grade prediction, and job application tracking with moderate ease. Users generally found the interface intuitive but noted occasional difficulties in finding deeper features, such as job application history.

(b) Mental Health Section

This section had the lowest score at 64% (25/39). A common issue was users struggling with this question: “Which of the following mental health resources are recommended by UniPath?” The possible reason is that some resources were positioned at the bottom of the page, and users did not scroll down far enough to see them. To mitigate this, prioritizing the most popular or frequently accessed features at the top and removing less essential information can help users locate resources more easily.

(c) Privacy Settings

The results for the data privacy and notification settings were mixed. Users appreciated the detailed options available but found the structure challenging and not intuitive enough to use. Simplifying access to these settings would be helpful.

3. Heuristic Evaluation

Drawing from participants’ survey feedback and the discussion in class, below is the detailed evaluation of the app based on the heuristics developed by Nielsen (2024).

(a) Visibility of System Status

Strengths:

Users can easily view essential information like grade predictions, credit progress, and career application status directly from main tabs and dashboards.

Improvement Areas:

Some features, such as privacy settings and job recommendations, are somewhat hidden and require multiple clicks. Making these features more immediately visible and simplifying the UI design would help reduce the complexity of navigation.

Changes Made:

A welcome page (“What do you need help with today?”) is added before jumping directly to the app content after logging in. This helps prevent overwhelming users with a large amount of text immediately after the login page and simplifies navigation.

Since some users found the privacy settings difficult to locate, I moved the settings button from the top right-hand corner to the middle of the page. With keywords such as “Password” and “Data Privacy” now included under the settings button, users can find the settings they need more easily

(b) Match Between System and the Real World

Strengths:

The language used, such as “Academic Dashboard,” “Career Opportunities,” and “Mental Health Resources,” aligns well with terms familiar to university students. This makes navigation intuitive for the target audience.

Improvement Areas:

Some users misinterpreted icons and visual indicators, like the pie chart for credit progress, assuming they referred to course counts rather than credits. Clearer icons or labels would help reduce this ambiguity.

Changes Made:

Clearer labels have been added to the graphs. For example, on the Credit Progress graph, the word “credits” is now included on the chart to ensure readers do not mistake it for course numbers.

(c) Recognition Rather Than Recall

Strengths:

The main dashboard and labeled icons help reduce the need for users to remember specific paths to access information.

Improvement Areas:

Some deeper features, like job history or personalized resources in the mental health section, require additional clicks and navigation. Quick links or shortcuts for frequently accessed areas would make these resources more accessible.

Changes Made:

Some popular features, such as users' job application history, are now positioned at the top of the page for easier recognition and access.

<- Previous Milestone

Next Milestone ->

Page updated

Report abuse