Phase 4

Roles:

Project Manager: August Wang

Research Lead: Muhammad Al-Atrash

Design Lead: Lauren Hruza

Prototype Lead: Chai Zhang

Evaluation Lead: Sean McGranahan

Reporting Lead: August Wang

Table of Contents

Concept Statement

Brain Buddy is a dynamic app designed to smooth the transition to university by fostering connections among students. Entering college comes with academic challenges and the daunting task of fitting into a new social circle. Brain Buddy steps in by offering a platform for students to easily engage in study sessions, club events, or casual meet-ups. Key features include profile matching and real-time chats, enhancing meaningful bonds. The app integrates with the university's SIS, auto-placing students in course chats. It also links to club chats, boosting campus involvement. The standout feature is the buddy matching quiz, pairing students with potential friends based on common interests and schedule. Additionally, unique ice-breakers aid initial conversations, easing initial hesitations. Brain Buddy champions academic collaboration both online and offline, and fosters a community spirit, ensuring students not only thrive academically but also find their tribe on campus.

Tailoring the Scope

In tailoring the Brain Buddy app project for my semester-long HCI course, we encountered the intrinsic difficulty of assessing long-term performance and user satisfaction, a key challenge given the app's nature as a social networking platform designed for sustained user engagement. This limitation was particularly pronounced due to the project's timeframe, which restricted the ability to observe and measure user interactions and satisfaction over an extended period, a crucial aspect for social networking applications. To navigate this constraint, the project's scope was focused on initial user engagement and the immediate usability of the app, using a high-fidelity prototype in Figma. This approach emphasized the evaluation of short-term user interactions and feedback, which, while not fully representative of long-term usage, provided valuable insights into the app's design and user experience. The decision to prioritize these aspects was driven by the need to align the project with the course's educational objectives and timeframe, ensuring that the work remained feasible and relevant to the principles of HCI, despite the challenges in capturing long-term user dynamics.

UX Target 

UX Target Table

Rationale

Ease of Profile Setup and Customization:

• How it was derived: The goal is to ensure users can efficiently set up and customize their profiles. The measure is the initial user performance in completing this task.

• Baseline level: Average profile setup time for a job website; we each tried setting up an account at Indeed which prompts users to create an account using an email, input First name, Last name, City and State in the “Contact information” section; and we filled out the Education, Skills, and Languages in the “Qualifications” section to mimic Brain Buddy’s account customization process. We timed each process and took the average.

• Measuring Instruments: A user-friendly profile setup is crucial for onboarding. Lowering the time to complete this task enhances the overall user experience, making it easier for students to start using the app effectively.

Effective Buddy Matching through Quiz:

• How it was derived: The goal focuses on creating an efficient buddy matching process. The measure is the first impression of users regarding the Brain Buddy Quiz questions.

• Baseline level: Determined based on an average score (6/10, an average passing score), with no reference to an existing system, as it is a novel feature.

• Measuring Instruments: The buddy matching process is a key feature for forming meaningful connections. A user-friendly quiz with efficient questions ensures that students can trust the quiz to quickly find suitable buddies, promoting active and relevant connections.

User satisfaction with the translate feature:

• How it was derived: Focuses on the satisfaction of international students with the translation feature. The measure is the first impression, assessed through satisfaction with the translation feature.

• Baseline level: Set at a 6/10 rating.

• Measuring Instruments: Recognizes the diverse user base and addresses the need for effective cross-cultural communication. Positive first impressions with translation contribute to overall user satisfaction. The specified measuring instrument, a targeted question about satisfaction with the translation feature, ensures precise feedback. With a baseline level set at 6/10 and a target level higher than or equal to 7.5/10, Brain Buddy aims for continuous enhancement in user satisfaction, emphasizing the app's commitment to improving the translation feature for international students.

Ease of use of the ice breaker feature:

• How it was derived: Targets the ease of use for users with a little experience in initiating icebreaker interactions. The measure is the initial user performance in sending a specific icebreaker.

• Baseline level: Set at 25 seconds. The app GamePigeon works in a similar mechanical way to Brain Buddy’s Icebreaker feature (sending a fun game over message). We each tried sending a specific game (Chess) to a person in our contact. We timed the processes and took the average.

• Importance: With a baseline level of 25 seconds and a target level of 20 seconds, Brain Buddy aims to enhance the speed and simplicity with which users can send icebreakers. Ultimately encouraging engagement and interaction among students. A quick and easy icebreaker feature enhances the app's appeal and fosters connections among users.

Initial user satisfaction with app features:

• How it was derived: Aims to gauge user satisfaction from the first impression. The measure is based on the initial user performance and satisfaction with app features.

• Baseline level: Set at a 6/10 rating.

• Measuring Instruments: Recognizes the significance of a positive initial experience. First impression plays a pivotal role in retaining users and encouraging continued interaction with the app. Higher satisfaction from the start contributes to user retention and positive word-of-mouth recommendations.

Initial user satisfaction with app features (emphasis usefulness):

• How it was derived: Similar to the goal for all students, with a focus on upperclassmen. The measure includes satisfaction with app features, specifically assessing likeliness to use and recommend Brain Buddy.

• Baseline level: Set at a 6/10 rating.

• Measuring Instruments: Acknowledges potential variations in user needs among upperclassmen. Upperclassmen most likely are already in existing friend group; this evaluates if they would still use the app and/or recommend Brain Buddy to their friends based on the features that they have seen. Higher satisfaction among this user class contributes to sustained use and positive endorsements.

Fast and easy study session creation process:

• How it was derived: A slow and complicated study session creation process could discourage students from creating study sessions, and thus leading to less student engagement. Thus, having a goal of high learnability for the study session creation process is important. Learnability is evaluated through creating a study session at a specific time and location.

• Baseline level: Initial average time on task (min) during the first use.

• Measuring Instruments: Ensures that users, irrespective of experience, can consistently utilize the study session creation feature. Learnability is crucial for a seamless and intuitive user experience over time.

Initial user satisfaction with the block & report feature:

• How it was derived: Targets the satisfaction of users with a little experience in blocking a person in PM contacts. The measure is the initial user performance in blocking a contact.

• Baseline level: Set at 17 seconds. We each tried blocking an account on GroupMe. We timed each process and took the average.

• Measuring Instruments: Ensures that even users with limited experience can efficiently perform key tasks. A quick and straightforward process contributes to a positive early user experience.

Design Refinement 

Client Feedback

Instructor/TA Feedback

A significant feedback point from both TAs and the instructor emphasized the importance of adhering to Nielsen's Usability Heuristic. This involves ensuring that the wording of features incorporates familiar words, concepts, and phrases. In designing the account creation process, our focus was on providing users with ample control and freedom. By implementing "back" and "next" buttons at every step, we facilitated an easy way for users to navigate backward in the process, allowing them to maintain control and avoid frustration. This approach extends to the Buddy Matching process, where users have the flexibility to go back and modify their answers. The study session creation process similarly offers a clear exit option through the cancel button located at the top left.

In addition to user control, we prioritized establishing a good match between the system and the real world. The app is designed to "speak" the users' language, employing words, phrases, and concepts that are familiar to them. Throughout all app phases, we ensured that users can comprehend the meaning without needing to consult a dictionary, reinforcing a user-friendly and intuitive experience.

Prototype

Rationale of Decision on Type of Prototype

For our project, an interactive prototype that incorporates both horizontal and vertical aspects using Figma is a suitable choice. This decision aligns with our goal to thoroughly evaluate the key aspects of our app design, including features like Brain Buddy matching, direct messaging, group chats, creating study sessions, and profile settings.

Horizontal vs. Vertical Prototyping:

• Horizontal Approach: We initially focused on a horizontal prototype to cover a broad range of features, providing a comprehensive view of the app’s interface. This approach was crucial for ensuring that all major functionalities, like Brain Buddy matching, group chats, and study session creation, were represented and accessible.

• Vertical Approach: As development progressed, we incorporated vertical aspects into our prototype. This decision was driven by the need to demonstrate deeper functionalities of certain key features. For example, we developed the Brain Buddy matching feature more fully to simulate its real-world application and assess user interactions with it in a more detailed manner.

Fidelity Level:

• High Fidelity: We chose a high-fidelity prototype, recognizing the importance of realistic user feedback on the design’s aesthetics and usability. Given the complexity and interactive nature of our app, a high-fidelity prototype in Figma offered the most accurate representation of the final product. This choice was essential for gauging user reactions to the look, feel, and flow of the app.

Connection to Design Goals

Our design goals, established in prior deliverables, focused on creating an engaging, user-friendly platform for students to connect and collaborate. Reflecting on these goals, the following connections were made:

• Enhancing User Experience: The high-fidelity aspect of the prototype aligns with our goal of providing an engaging and intuitive user interface. By simulating the final look and feel, we could gather accurate feedback on user experience aspects.

• Testing Functionality: The combination of horizontal and vertical prototyping methods allowed us to test both the breadth and depth of our app’s features. This approach was crucial for ensuring that each feature not only functioned as intended but also worked seamlessly within the broader app context.

• Evaluating User Engagement: By developing key features like Brain Buddy matching more fully, we could assess how these features are received by users, directly addressing our goal of creating engaging content that keeps users interested and satisfied.

The decision to use a high-fidelity, mixed horizontal and vertical prototype was a strategic one, allowing for a comprehensive evaluation of both the user interface and deeper functionalities. This approach directly supported our goals of creating an engaging, intuitive, and feature-rich application for students. By realistically simulating the user experience and the app's key functionalities, we were able to gather valuable insights that will inform further development and refinements.

Process

We initiated our prototyping process by reviewing our wireframes to ensure satisfaction with the element layout, incorporating refinements suggested by our client in the previous section. For the prototype, we selected Robert Stanley, our primary persona from phase 3, as the representative profile. This decision aimed to ensure that the prototype was well-tailored for our most crucial user roles.

The choice to use Figma for building the high-fidelity prototype stemmed from two key reasons. Firstly, our phase 1 prototype and wireframe were already developed in Figma. Secondly, Figma is a widely recognized and powerful tool for constructing high-fidelity prototypes. The application was constructed with the goal of prototyping most screens and features, ensuring that all major functionalities (account creation, study session initiation, block and report, icebreaker, and translation) could be evaluated.

Concerning the color scheme, we deliberately avoided using orange due to its strong association with UVA. Since the Brain Buddy app could potentially operate in different university environments, we aimed for a broad appeal. Despite being a social networking app, we wanted to maintain a semi-professional academic vibe, so we refrained from using bold color combinations like pink or bright yellow. Instead, we settled on a blue and teal palette – blue as the base and teal for a touch of vibrancy – striking a balance between a semi-professional atmosphere and a touch of fun. Regarding typography, we opted for the Red Hat Display font. Known for its readability, it offers a versatile range of text styles, from light to semi-bold to black. This flexibility was crucial in designing the UI for the Brain Buddy app.

Interactive Prototype

Link: Brain Buddy Prototype in Figma

Video Walkthrough

Interaction Pathway

Evaluation 

Goals

The overarching goal of our evaluation was to rigorously assess and understand the user experience provided by the Brain Buddy app, ensuring it aligns with the specific needs and expectations of our target user groups. Our evaluation was designed to delve into several key aspects:

• Usability and Functionality Assessment: A primary focus was on evaluating how users interact with the app’s various features. This included assessing the ease of navigation, the intuitiveness of the interface, and the functionality of core elements like the Brain Buddy matching system, study session creation tools, and communication features like direct messaging and group chats.

• User Satisfaction and Engagement: We aimed to gauge the level of satisfaction among users with different aspects of the app. This involved understanding how features like the translation tool and the icebreaker interaction were received by users, particularly focusing on their first impressions and ongoing engagement with the app.

• Efficiency and Performance Measurement: Another crucial goal was to measure the efficiency of the app in terms of task completion times. We sought to determine whether users could perform essential actions, such as setting up their profile, creating study sessions, or using the block feature, within a reasonable and satisfactory timeframe.

• Target-Specific Insights: Since our user base includes diverse groups like lowerclassmen, upperclassmen, and international students, our evaluation aimed to gather insights specific to these groups. For instance, understanding how upperclassmen perceive the usefulness of the app and assessing the effectiveness of the translation feature for international students were key objectives.

• Identifying Areas for Improvement: Through this evaluation, we also aimed to identify any potential areas for improvement within the app. Gathering detailed user feedback and observing user interactions with the app would enable us to pinpoint aspects that could be enhanced to provide a more seamless and enjoyable user experience.

• Alignment with UX Targets: Lastly, ensuring that our findings aligned with our pre-established UX targets was essential. This meant not only measuring whether the app met these targets but also understanding how and why certain aspects of the app performed the way they did in relation to these targets.

In summary, our evaluation plan was not just about measuring user satisfaction and efficiency; it was a comprehensive effort to understand the multifaceted user experience of the Brain Buddy app, aiming to ensure that it effectively meets the needs of its diverse user base. By achieving these goals, we hoped to not only validate the current design but also gather actionable insights for future enhancements.

Evaluation Plan 

The primary goal of our evaluation was to assess the usability, functionality, and user satisfaction of the Brain Buddy app, with a focus on aligning our findings with the predefined UX targets. Our plan centered on utilizing both behavioral tasks and questionnaires to capture a broad spectrum of user interactions and perceptions.

• Behavioral Tasks (BT): We designed specific tasks to evaluate key functionalities of the app, such as account creation, joining group chats, creating study sessions, sending icebreakers, and blocking contacts. These tasks were mapped to our UX targets, such as ease of profile setup, efficient buddy matching, and ease of use of various features.

• Questionnaires (Q): To gather subjective user feedback, we incorporated questionnaires addressing confidence in buddy matching, satisfaction with the translation feature, overall app satisfaction, and the likelihood of continued usage by upperclassmen. These questionnaires aimed to measure user satisfaction and perceptions related to the UX targets.

Limitation

The evaluation and prototype of the Brain Buddy indeed present certain limitations. Firstly, the short-term nature of the evaluation does not allow for the observation of long-term user engagement and behavior changes. This limitation can obscure insights into the app's sustained usability and its impact over time. Secondly, the small sample size of 3-5 participants per benchmark or questionnaire may not fully represent the broader user base. This size limits the diversity of feedback and may lead to conclusions that are not universally applicable. However, these constraints are justified by the practicalities of conducting a focused, time-efficient evaluation and the early stage of the app's development, where initial, more controlled feedback is essential for iterative improvements before wider testing. 

Procedure

• Location 

The evaluations were conducted either in person or via Zoom in a quiet environment, ensuring that participants could focus on the app without distractions. Each session lasted approximately 10 minutes. We used a Google Form to streamline data collection, allowing for efficient tracking and analysis of responses.

• Benchmark Tasks

The procedure for the benchmark tasks was designed to be uniform across all participants to ensure reliability in our data. We used a script that guided participants through a series of tasks mimicking real-world use cases of the app. These tasks included creating an account, setting up study sessions, using the translation feature, and engaging with the Brain Buddy matching quiz. We timed each task using a phone timer and recorded any errors or difficulties encountered.

• Questionnaires

We developed four questions, each designed to assess different aspects of the app:

• Q1: Confidence in the Brain Buddy matching system.

• Q2: Satisfaction with the translation feature.

• Q3: Overall satisfaction with the app’s features.

• Q4: Likelihood of continued use and recommendation by upperclassmen.

Participants rated their responses on a scale from 1 to 10, with space for additional comments and insights. This approach aimed to capture both quantitative and qualitative feedback, providing a comprehensive understanding of user experiences and perceptions.

• Participants

We decided to recruit participants who had been involved in our initial user interviews, as they were already familiar with the concept of the app. We also reached out to our broader network of students to ensure a diverse range of user experiences. This familiarity was anticipated to create a comfortable environment for participants, which is crucial for honest and detailed feedback.

• Apparatus

• Timer: A timer on a phone or laptop to record task completion times.

• Google Form: To systematically collect questionnaire responses.

• Participant’s Phone: To access and interact with the Figma prototype simulation of Brain Buddy.

  • If they do not have the Figma app, the evaluator's phone is used instead

This evaluation plan and procedure were meticulously crafted to align with our project goals and UX targets, ensuring a thorough and consistent assessment of the Brain Buddy app’s user experience.

Results and Analysis

Benchmark Tasks

• BT 1: Account Creation

  • BT Prompt: "Create an account by entering your UVA computing ID and a password. You want to join the CS 3140 and MATH 1310 group chats. You are involved in the Japanese Student Association. Your interests and hobbies include cooking, photography, Disney channels, Disney plus, yoga and drawing."

  • Participants (Year, Student-status, Data):

    • Participant 1: Alex Chen, 1st year, in-state (2:08 min)

    • Participant 2: Shanyun Zhang, 2nd year, Out-of-state (1:56 min)

    • Participant 3: Annie Zihan Zhao, 2nd year, International (2:05 min)

  • Data Summary: 

    • Average: 2:05 min

    • Baseline: 2:30 min

    • Target: <2:30 min

  • Findings from BT 1: 

    • In general, all participants successfully completed the account creation process with ease and efficiency. However, a notable observation was that many participants encountered challenges when attempting to select checkbox items. The current prototype features small checkboxes that users can tap to select or unselect items. This design poses difficulties, particularly for individuals with larger hands, as the small checkboxes can be challenging to precisely click on. To enhance usability, a design improvement suggestion is to broaden the selection area to encompass the entire item. This means modifying the items so that clicking on the item itself is sufficient to select it, eliminating the need to tap specifically on a small checkbox. 

• BT 2: Study Session Creation

  • BT Prompt: "Create a MATH 1310 Exam 2 study session on Wednesday 12/11 from 1 pm - 3:45 pm at Rice 442. Include the note 'Looking for people to review exam 2 materials together!.' "

  • Participants (Year, Student-status, Data):

    • Participant 1: Will Hixson, 2nd year, in-state (2:00 min)

    • Participant 2: Hannah Heafner, 2nd year, in-state (1:44 min)

    • Participant 3: Neha, 2nd, in-state (59 sec)

  • Data Summary:

    • Average: 1:34 min

    • Baseline: 2:30 min

    • Target: <2:30 min

• Findings from BT 2: 

  • All participants successfully completed the task of setting up a study session, with the completion times ranging from 59 seconds to 2 minutes. This performance is notably better than the baseline target of 2:30 minutes, highlighting the task's intuitive design and ease of use. The average completion time of 1:34 minutes further reinforces this observation. Interestingly, the variation in completion times, although within a close range, suggests a degree of variability in user interaction with the app. This variability could be due to individual differences in familiarity with the app or inherent differences in user interaction styles. Moving forward, involving a larger and more diverse participant group in testing could provide additional insights into the app's usability across different user segments. This would help in identifying any subtle nuances or potential areas for improvement in the study session creation feature of the Brain Buddy app.

• BT 3: Ease of use of the ice breaker feature 

  • BT Prompt: "Muhammad is your new HW4 partner for CS 3140, to start your first conversation with him, send the “Two Truths & A Lie” icebreaker to Muhammad via PM."

  • Participants Lists: (Year, Status, Data)

    • Participant 1: Ryan Swindell (Grad Student, In-State, 20.87 sec)

    • Participant 2: Nate Gleberman (4th Year, In-State, 21.16 sec)

    • Participant 3: Ryan Wood (4th Year, Out of State, 19 sec)

  • Data Summary: 

    • User Average: 20.1 sec

    • Baseline: 25 sec

    • Target: 20 sec

  • Findings from BT 3: 

    • All users easily located and promptly sent icebreakers, indicating an intuitive design and placement of the icebreaker feature. However, we observed challenges related to the physical interaction with the prototype. Users experienced confusion as the prototype visually resembled an app but functioned as a prototype. For instance, when attempting to input text in a field, users tried to type on the prototyped keyboard without realizing that the text was already pre-filled. This limitation stems from the software used for prototyping, as Figma lacks real-time responsive interactions. Achieving the functionality of typing on a keyboard, with the textbox remembering the input and interacting with it, is currently impossible with Figma. For future app development, we may explore options such as coding the prototype or utilizing more advanced prototyping software like Adobe XD. 

• BT 4: Blocking a Contact

  • BT Prompt: "John has just said something offensive to you in PM, block him from your contact."

  • Participants Lists: (Year, Status, Data)

    • Participant 1: Jason Pham, 3rd year, In-state (13.07 sec)

    • Participant 2: Isabel Robles, 3rd year, Out-of-state  (16.54 sec)

    • Participant 3: Shanyun Zhang, 2nd year, Out-of-state (14.28 sec)

  • Data Summary: 

    • Average: 14.63 sec

    • Baseline: 17 sec

    • Target: <17 sec

  • Findings from BT 4: 

    • All participants successfully located the blocking feature within the app, indicating its intuitive nature. However, upon observation, it was noted that all participants initially attempted to access the block functionality by clicking on the profile of the person they intended to block, rather than navigating to the top right where the three-bar icon is located. This observation suggests that it might be beneficial to relocate the block and report functionalities to a more accessible placement, such as within the profile-clicking interface. 

Questionaries 

• Q1: Confidence in Buddy Matching

  • Question: "How confident are you that the Brain Buddy matching quiz will effectively match you with compatible potential buddies?"

  • Participants Lists: (Year, Status, Rating)

    • Participant 1: Robin Xiao, 3rd year, Out-of-State (6)

    • Participant 2: William Wang, 3rd year, International (8)

    • Participant 3: Elizabeth Myers, 2nd Year, In-State (5)

  • Data Summary:  

    • Average Rating: 6.3/10

    • Baseline: 6/10

    • Target: >= 7.5/10

  • Findings from Q1:  

    • A significant portion of the negative feedback we received emphasized a desire for more questions related to study habits, rather than inquiries about food preferences and favorite movie genres. Upon reflection, we identified potential confusion among two of the three participants regarding the intended goal of the Buddy Matching Quiz. Originally, they believed its purpose was to match users with potential study buddies rather than potential friends. This realization prompted us to acknowledge the need for clearer communication within the Buddy Matching Quiz interface.

    • In addition, one participant noted the variability in schedule availability each semester. The Buddy Matching Quiz was initially designed to be a semesterly activity based on a suggestion from our client during Phase 3. She recommended limiting the quiz to specific times in the semester to enhance its effectiveness in fostering new friendships. We recognized that these details about the matching quiz and the process were not well-conveyed in the current prototype. To address this, we plan to add a scene with more accurate wording to clarify the intended purpose of the matching quiz, as well as provide information on its availability timeline and notification process when a match is found.

    • Furthermore, participant feedback highlighted a potential need for a study buddy matching quiz. This consideration will be taken into account for the future development of the app.

    • From the participant who understood the intended goal of the matching quiz, we have received positive feedback on specific questions, such as the lifestyle question on preferences for going out or staying in, and exercise habits. They suggested removing some of the more filler-like questions (for example, if you could travel to any continent in the world) and replacing them with questions about the quiz taker's extrovert/introvertness and whether they prefer to be matched with an introvert/extrovert. During the evaluation, we also realized a need to ask for the quiz taker's preference on the year of their potential matches, whether they would prefer to match with people in the same school year as them, and whether they would like to match with someone of the same gender.

• Q2: Translation Feature Satisfaction

  • Question: "As an international student using our translation feature for the first time, how satisfied are you with its accuracy and ease of use? Please rate your satisfaction on a scale from 1 to 10, where 10 indicates extremely satisfied and 1 indicates not satisfied at all." 

  • Participants Lists: (Year, Status, Rating)

    • Participant 1: Diana Liu, 3rd year, International, 9

    • Participant 2: Celine Zhou, 3rdyear, International, 7

    • Participant 3: Aurora Tan, 3rd year, International, 10

    • Participant 4: William Wang, 3rd year, International, 7

    • Participant 5: Daisy Wang, 1st year, International, 9

  • Data Summary: 

    • Average Rating: 8.4 

    • Baseline: 6/10

    • Target: >=7.5/10

  • Findings from Q2:

    • Based on the data collected from international students evaluating the translation feature, the findings indicate a positive reception. The average satisfaction rating stands at 8.4, surpassing the targeted benchmark of 7.5/10. This high rating suggests that the feature generally meets the needs and expectations of this user group. Users particularly appreciated the clear design and the intuitive nature of the feature. The function of long-press a message to access various options like translation, copy, and reply was highlighted as a sensible design choice. Additionally, the convenience of the language change option within the translation feature was well-received. These positive comments suggest that the feature's usability and integration within the app are key factors contributing to user satisfaction. Overall, the translation feature appears to be a valuable tool for international students.

• Q3: Overall Feature Satisfaction

  • Question: "How satisficed are you with the various features of the Brain Buddy app?"

  • Participants Lists: (Year, Status, Student status)

    • Participant 1:  Jason Pham, 3rd year, In-state (10)

    • Participant 2: Emily Chen, 4th year, In-state (9)

    • Participant 3: Shanyun Zhang, 2nd year, Out-of-state (10)

    • Participant 4: Robin Xiao, 3rd year, Out-of-State (8)

    • Participant 5: Jasmine Senesombath, 4th year, in-state (9)

  • Data Summary: 

    • Average rating: 9.2

    • Baseline: 6/10

    • Target: >=7.5/10

  • Findings From Q3:

    • Each of the five participants expressed overall satisfaction with the diverse features of the Brain Buddy app. Despite recognizing that the prototype isn't a fully functional application, they successfully understood its functionalities by exploring the Figma prototype. The participants did not feel overwhelmed by the range of features in the Brain Buddy app; instead, they believed it struck an optimal balance to foster peer connections among students. In particular, they found features like course group chats and icebreakers to be creative and expressed interest in trying them out if the app were to be developed.

    • While participants were generally satisfied with the app features, we did receive some suggestions for improving the overall structure of the app. One participant mentioned that the separation of group chats and private messages doesn't make sense and suggested adding a tab in the group chat screen for private messages instead of having a different screen.

• Q4: Upperclassmen Usage and Recommendation

  • Question: "As an upperclassman, how likely are you to use and recommend the Brain Buddy app to your peers based on your initial impressions? Please consider the usefulness of the app's features in your response and rate their usefulness on a scale from 1 to 10, with 10 being extremely useful."

  • Participants Lists: (Year, Status, Rating)

    • Participant 1: Diana Liu, 3rd year, International, 10

    • Participant 2: Robin Xiao, 3rd year, Out-of-State, 6

    • Participant 3: William Wang, 3rd year, International, 8

    • Participant 4: Nicole C, 4th year, In-State, 5

    • Participant 5:Abby Teague, 3rd year, In-State, 8

  • Data Summary: 

    • Average Rating: 7.4 

    • Baseline: 6/10

    • Target: >=7.5/10

  • Findings From Q4:

    • Based on the collected data, the Brain Buddy app exhibits a moderate level of acceptance among upperclassmen, with an average rating of 7.4 out of 10. This rating, derived from a diverse user group including international, out-of-state, and in-state students, reflects varied experiences and perceptions. Some users perceive the app as highly beneficial, particularly introverted individuals who find it a valuable tool for initiating collaborations outside their social circle. Conversely, other users consider it less relevant, noting that as upperclassmen, they already have established social networks, diminishing the app's necessity. Critiques about the app's procedure being overly lengthy and cumbersome also emerge, suggesting a need for streamlined user experience. These insights highlight a dichotomy in user satisfaction and point towards potential areas for improvement, especially in simplifying the app’s functionality to enhance its appeal and utility for a broader range of students. 

Evaluation Summary 

Finding Summary 

The findings reveal a generally smooth account creation process in Benchmark Test 1, with a suggestion to enhance usability by broadening the selection area of checkboxes. Benchmark Test 2 indicates an intuitively designed study session creation process. Benchmark Test 3 highlights the ease of locating and sending icebreakers but notes prototype confusion due to Figma limitations. Benchmark Test 4 showcases an easily located block feature, with a suggestion to consider relocating it for better accessibility. Question 1 underscores a desire for a matching process for study buddies, as well as emphasizing the need for clearer communication in the quiz interface. Question 2 reports a high satisfaction rating for the translation feature, emphasizing its value for international students. User concerns in Question 3 suggest simplifying the app's structure for group and private messages. Finally, Question 4 highlights a moderate app acceptance level among upperclassmen, indicating varied perceptions and suggests that furthur research into needs of upperclassmen is needed.

Feedback and Recommendations

While the app currently maintains a streamlined flow, there are opportunities for improvement in enhancing clarity around different features and sections. Those opportunities are organized in the table below,

Phase 4 Evaluation Findings and Recommendations Table