This study is designed to provide a comprehensive exploration of students' experiences using the RIT wellness portal for appointment scheduling and navigating through the Student Health Center for medical treatment. Our primary objective is to gain a thorough understanding of the user journey, from accessing the wellness portal to receiving medical care. By conducting this in-depth investigation, we aim to uncover insights that will inform the design and optimization of the RIT wellness portal and enhance the overall user experience at the Student Health Center.

To gain a thorough understanding of students' experiences using the RIT wellness portal for making appointments and their journey through the Student Health Center to seek medical treatment.

The current Wellness portal at RIT (https //wellnessportal.rit.edu/) is owned and operated by the Division of Student Affairs (SA). Students may sign up and make appointments through the portal to visit the Student Health Center (SHC) or other wellness services. Students who need medical assistance take action, address some symptoms through the portal, and make appointments with SHC, as shown in Figure 1 and Figure 2. Students paying Health fees with tuition are eligible to receive service without charge. Part-time students may pay some fees to utilize the service. Visiting SHC, then an individual requiring further medical treatment may be referred to the local hospitals. The issue we are addressing with this portal is that it is not a very supportive portal to those who speak English as a second language or afterward and is not supportive to those who do not know American medical systems. In addition, through the options, it is hard to address the symptoms when they schedule a visit. The list of individuals who may be affected by this project is mostly different types of users mentioned in the participant's list and the below roles  

• On RIT main campus, students (User/ customer) 

• Staff in Student Affairs (owners of this portal)

• SHC staff, including medical professionals (administrator) 

The origin of this user research is the feedback of students who experienced a hard time utilizing the Student Health Center. Students are the first individuals who are affected by this project as they are the main users and customers. As you see in Figure 1, this portal is owned by the Division of Student Affairs, which would be one of the individuals/groups who would be affected. Lastly, the Student Health Center, as the provider of service, must be part of our research. 

Students

1. • International Students - The US medical system is quite complicated, and we may be able to see the portal usability issues the students who do not know the American medical dynamics

   • US/domestic students - American students know the American dynamics, and most of them speak English as their first language. We should be able to observe and find out the portal usability issues from the different viewpoints of international students. 

   • Students using RIT insurance - Insurance is one of the critical factors in the American Medical system. Exploring the usability issues of the portal based on the insurance provides us with further insights to come up with the new version of the portal. 

   • Non-RIT Insurance students - As discussed, insurance is the key factor, and looking into the experience of students who do not purchase health insurance from RIT may have a different experience. 

   • Frequent visitors of SHC - Recruiting through this category of students would provide us with the learnability of the portal and how it supports students at RIT. 

   • Non-frequent visitors of SHC - these students are not familiar with the portal and SHC, which provides us with insights into how first-time users would utilize the portal. 

   • Students who need to go to the hospital - There will be additional steps to take to visit the hospital. Since SHC has limited resources to provide the treatments, some of them get referred to visit hospitals. Seeking students' experiences will allow us to find out how the portal can support this case.

   • Students who don’t need to go to the hospital - Certain students don’t need to take additional steps to visit the hospital. Medical support can be done at SHC. Some common examples include general medical checks, small conditions like a common cold, or having to undergo vaccination and other general services.

Stakeholders

Medical Staff & Student Health Center

• Physician/ Nurse - They are the providers and utilize the portal to look into the symptoms students have. Working with them provides us insights into accurate tendencies of RIT students' health issues, which could be reflected in the new version of the portal. 

• Mental Health Professional - It is not always the case that students only have physical issues. It could be mental health problems. Seeking their insights will give us opportunities to look into the options for expanding the target populations to students who need assistance from counseling services. 

Administrative Staff

• Advisors - They are the supporters of RIT students, and we would like to look into the possibility of them utilizing the Wellness Portal in the future to create strong support boundaries 

• Administrative Staff at SHC - When students visit SHC, the first person to talk to is the administrative staff, and they communicate with visitors to build the profiles based on the inputs of students when students schedule an appointment.

User Interviews  

Given the need to gain comprehensive insights into the user experience, including the process of receiving medical treatment and opinions on existing services, a user interview was selected as a potentially fruitful data collection method. User interviews align with the objectives of our study by providing valuable insights and understanding the users' perspectives, needs, and behaviors. By conducting user interviews, we can gather firsthand information about how users interact with the product or service, their pain points, and the context in which they use it, as described in [1]. Our questions were open-ended to help participants provide detailed elaborations using the system.

Contextual Inquiry

A contextual inquiry will also be needed for the data collection. Contextual inquiry involves observing and interviewing users in their natural environment while they perform tasks related to the product or system being designed, as covered in [1]. In the context of designing an application, this type of inquiry is important because it can help understand users' needs and the contextual relevance of the system; it can help identify the workflows and processes, uncover pain points, build empathy, and validate design assumptions.

1. Participants Study Approach 

Planification

Week 1 (02/26 - 03/03): Finalizing Questions and User Recruitment

Weeks 2-3 (03/04 - 03/17): Data collection (interviews and contextual inquiries)

Weeks 4-5 (03/18 - 03/31): Stakeholders Recruitment

Weeks 1-5 (02/26 - 03/31): Data Analysis Report Writing

Recruitment Strategies 

Since our project focuses on the US student health system, it was important for us to recruit on-campus students from US colleges as our direct Users. Due to the project being constrained to one semester, we narrowed it down to our population of users, the RIT on-campus students.

Before officially recruiting students, we prepared supplemental questions, such as, “Have you ever used SHC before? “Are you aware of the RIT Wellness Portal?” to find out if they are eligible for our project. 

Lastly, our group members are actively working with staff at RIT which allows us to reach out to our stakeholders without disturbing the operation of SHC. - detail this more.

Furthermore, we performed a pilot study to help us make necessary adjustments to our study before deploying on a large scale.

Categorization Strategy 

The interview questions were divided into two categories to address the dual objectives of our study, which aims to gather insights into students' experiences with both the RIT Wellness Portal and the medical treatment procedures they receive.

1. The first category is designed to gather insights about participants' experiences using the RIT Wellness Portal, an online platform for accessing various wellness services.

2. The second category is dedicated to understanding students' overall experiences with receiving medical treatments at the RIT student health center.

Interviews Plan 

To understand how different factors (category-based) influence user behaviors and experiences with the RIT Wellness Portal.

Target Participants 

8 participants, divided into 4 categories.

Team Members & Types 

1. Adrienne:  US students and International students

2. Yugo:  Students with insurance and Students with non-RIT insurance

3. Fatima:  Frequent users and Non-frequent users

4. Holly : Students needing only on-campus health care and Students needing referrals for hospital treatment

Interview Format

Depending on participants' availability, interviews can be conducted in person or via video conferencing. The duration is approximately 30-45 minutes per interview. With consent, interviews will be recorded for analysis purposes.

General Interview Structure

Introduction (5 minutes)

Warm-up Questions (5 minutes)

Category-Specific Questions (20-25 minutes)

• Adrienne's Focus: Explore how cultural and educational backgrounds influence portal use and preferences between US and international students.

• Yugo's Focus: Understand how insurance status affects access to and perception of available wellness services.

• Fatima's Focus: Investigate the reasons behind varying frequencies of portal use, including barriers and motivators.

• Holly's Focus: Dive into the healthcare needs that lead students to prefer on-campus services versus external referrals, including satisfaction levels with both processes.

Closing Questions (5 minutes)

• Allow participants to share any additional thoughts or suggestions for improving the RIT Wellness Portal.

• Thank the participant for their time and participation.

Post-Interview

• Debrief:  Team members should debrief after each interview to discuss initial impressions and key takeaways.

• Analysis: Consolidate and analyze data to identify patterns, similarities, and differences across categories.

• Report: Prepare a comprehensive report detailing findings, insights, and actionable recommendations to enhance the RIT Wellness Portal experience.

Additional Considerations

• Prepare open-ended questions that encourage detailed responses.

• Be ready to ask follow-up questions to delve deeper into participants' thoughts and experiences.

• Ensure sensitivity and confidentiality throughout the process, especially when discussing personal health information.

This plan is designed to be flexible, allowing Team Amsterdam to adapt as needed based on preliminary findings or logistical considerations.

2. Materials Used 

User interviews

The below questions were asked to our users (the students) based on our devised participant study approach and plan   

Introductions and demographics

Introductions

Hi, I’m [Your Name]. Thank you again for taking part in my research study. 

How are you?

Today, we’re going to chat about a few different things. We will focus on the topic of the College Wellness Portal improvements and your experiences getting medical care as a student at RIT.

1. Do you have any questions before we start?

2. I previously mentioned I would record this, so I would like to turn on the recording now. I will delete this recording later once I have an anonymized transcript of this conversation.

Demographics  

1. International/US student

2. What is your First language? 

3. Time spent at RIT

4. RIT Insurance/ non-RIT insurance

5. Education level - is this your first degree

6. Do you pay the Student Health Service Fee? (if they are not sure, ask them if they are full-time students or not)

7. Major or field of study

8. Does your health require special care? (people with disability)

Category 1  RIT Wellness Portal

Part 1  General Website usage behavior

I want to ask some questions to get a general idea about your website usage behavior.

1. How comfortable do you feel while using technology?

2. How often do you use the RIT Wellness portal (https //pyramed01.ad.rit.edu/PyramedPortal/)?

3. What do you usually use the health portal for? 

Part 2  Experience using the portal

That’s interesting and leads on to the next part of the interview, where I want to ask about opinions about using the RIT Wellness portal.

1. Please describe your latest experience of using the RIT wellness portal. (Ask them to tell the story; ask for as many details as you can.) 

   1. (if one of their uses is for booking appointments)

   2. How is experience for making appointments through the wellness portal? 

2. How was your experience finding something particular in the portal?

3. What is your experience navigating the health portal (e.g., features and functions)? (good or bad? how?)

Part 3  Benefits of the portal 

1. Has the health portal improved your overall health-related experience in any way? Please provide examples.

Part 4  Challenges using the portal

1. Have you had any confusion or difficulty using the RIT wellness portal? (If so, please provide examples)

2. Are there any specific tasks or processes that you find confusing or inconvenient?

3. What do you think about the amount of information given during the processes? Too much/enough/not enough information? How would you rate it on a scale from one to five?

Category 2  Medical Treatment Process

Part 1  Experience getting medical care

1. Please describe your latest experience of getting medical care through the school health center. (Ask them to tell the story, in a detailed manner).

2. How is your experience in getting medical care through the school health center?

3. How is your experience of getting medical instruction from doctors?

   1. For example (just talk about this to make the participant understand)

   2. The number of pills to take

   3. Precautions eg  the wound can not touch the water

4. If they need to get a referral to go to the hospital

   1. How is your experience of getting a referral to go to the hospital

Part 2  Benefits of seeking medical care

1. Has the student health center improved your overall health-related experience in any way? Please provide examples.

Part 3  Challenges of seeking medical care

1. Have you had any confusion or difficulty seeking medical care (If so, please provide examples)

2. Are there any specific tasks or processes that you find particularly confusing or inconvenient while seeking medical care?

3. What do you think about the amount of information given during the processes? 

4. (Too much/enough/not enough information? How would you rate it on a scale from one to five?)

Suggestions for improvements

1. Are there any specific features or functionalities you wish were improved or added that the current portal cannot perform but you wish it could?

Debriefing

1. Are there any other comments you would like to make or ideas you would like to share?

2. Do you have any other questions for me?

Stakeholders interviews

Furthermore, the below questions were devised for our direct internal stakeholders, which is the student health service and portal employees 

1. Can you please tell me what your role at the Student Health Center was and how you used the portal and app in your role? 

2. How do you measure the effectiveness of the wellness center services and online portal in meeting student needs and expectations?

3. Can you provide insights into any trends or patterns observed in student usage of the wellness center services and online portal over time?

4. Can you share any success stories or best practices related to the delivery of wellness center services or the functionality of the online portal that have positively impacted student engagement and satisfaction?

5. When students check in at the Student Health Center for their appointment, do you have all the information from the portal and app to process? 

6. Can you provide insights into any demographics of students for whom it was challenging to register at the wellness center?

7. What do you wish to know from the screen that they did not have when you checked students in?  

8. Are there any features or functionalities that students frequently request but are not currently available on the online portal?

9. How do you measure the effectiveness of the wellness center services and online portal in meeting student needs and expectations?

10. Can you tell me anything you know about the portal's usability from a medical professional standpoint?

11. Is there anything else you can share with me about the portal and app?

Contextual inquiry 

It involves with observation. We have two scenarios at this moment interested in proving to participants. 

Need to go to hospital version: 

1. Schedule an appointment online

2. Wait until the appointment day comes

3. Go to the front desk and talk to the staff(check-in)

4. Get a basic check by the nurse(blood pressure, height/weight)

5. Go to the doctor to get checked

6. Get a referral from the doctor with a lot of documents

7. Check the document information and go to the hospital with the doctor’s oral instructions

Don’t need to go to the hospital version:

1. Schedule an appointment online

2. Wait until the appointment day comes

3. Go to the front desk and talk to the staff(check-in)

4. Get a basic check by the nurse(blood pressure, height/weight)

5. Go to the doctor to get checked

6. Get instructions from the doctor with paper documents 

7. Go to Wegmans for prescription pick-up

1. Objectives

The pilot study aimed to test the feasibility of our interview questions designed to explore students' experiences with the RIT Wellness Portal and their encounters at the RIT Student Health Center. Given the novelty of our research focus, it was imperative to ensure that our questions were comprehensible, relevant, and capable of eliciting rich, informative responses.

2. Methods

We recruited one participant from our fourth category of interest—students who primarily utilize on-campus health services. This choice was strategic, allowing us to focus on experiences directly related to the on-campus facilities and digital interfaces offered by RIT. The participant was asked to respond to our initial set of interview questions, which intertwined inquiries about the Wellness Portal and the Health Center. Additionally, we conducted an observation session to understand how the participant navigated and utilized specific functions on the Wellness Portal, aiming to correlate verbal feedback with actual usage patterns.

3. Key Findings

Two main issues emerged from the pilot study 

Confusion Between Different Aspects of Healthcare Experience  The participant expressed confusion when questions about the Wellness Portal and the Health Center experiences were intermixed. This indicated a potential conflation of digital and physical service perceptions, suggesting the need for clearer delineation in our questioning.

Vagueness Due to Question Comprehension Issues  Some questions were not understood as intended, leading to vague responses. This highlighted a disconnect between our assumptions of user familiarity with certain terms or functionalities and the participant's actual understanding.

4. Adjustments and Lessons Learned

In response to the findings, we made the following adjustments to our research approach 

• Recategorization of Questions  We divided our interview questions into two distinct categories 

  1. Experience Using the RIT Wellness Portal  Focused on the digital interaction and usability aspects of the portal.

  2. Experience at the Student Health Center concentrated on the physical aspects of receiving medical care on campus.

This separation aims to minimize confusion and allow participants to clearly distinguish between their digital and physical healthcare experiences at RIT.

• Question Refinement  The pilot study underscored the necessity of revising our questions for clarity and understandability. We plan to simplify the language and provide context or definitions for any technical terms or specific functionalities mentioned. This adjustment is anticipated to reduce ambiguities and foster more precise and insightful responses.

The insights from this pilot study are invaluable, not only in refining our interview protocol but also in highlighting the importance of clear communication and separation of concepts when researching intertwined digital and physical user experiences. Post-pilot adjustments will enhance the reliability and depth of our findings in the main study, contributing to a more nuanced understanding of student healthcare experiences at RIT.

[1]  Rapid Contextual Inquiry (Holtzblatt 2005)