Research Method
The research team proposed the project to the Office of Research Integrity to ensure the project followed university research protocols. Each member completed CITI training to ensure testing, research and future writing on the project follows the recommended guidelines for professionalism and proper research methods. Once the project was approved and members completed CITI training the team met to finalize the project deadlines and testing script.
The team chose a task based usability test style. Task based usability testing is popular because it enables researchers to convert goals for the design into tasks that easily show if the user was able to complete the task or not. In turn, this shows the designers if the user was able to meet their goal for visiting a website or using a service. The team went one step further and created scenario based tasks to engage students in tasks they would typically complete during their college career.
The research team created a script (figure 1) for the usability test to ensure each participant was asked to complete the same tasks, in the same way, with the same follow up questions. This consistency allowed the team to better analyze the data because we had consistent data points to review and rank on a rubric. The usability study looked at the functionality of the left hand navigation and new LibGuides template layout. The goal of the research was to answer the following research questions:
The usability test also included four post test questions to gain general feedback from the participant. The team decided to recruit five undergraduate students and five graduate students for the usability testing to ensure our typical users were tested.
The team recruited participants through a variety of means including: a flyer posted at library service points and given to library instruction attendees, emails to professors, emails to students and in person recruitment in the library. This combined team effort resulted in our goal of ten participants (five undergraduate students and five graduate students).
The usability testing was completed in an office in the library using a laptop with the software Camtasia that captures the computer screen and records audio and visual of the participant. The videos captured both the student and the screen to be reviewed by library staff who could not attend the live usability sessions. Each participant received an informed consent form (figure 2) describing the project, the purposes, the potential benefits and possible risks as well as consenting to the audio/visual recording. Each participant read and signed the informed consent form prior to starting the usability testing.
To ensure consistency in each study we used the same staff person to run each usability session. The usability study portion of the session ran 15-25 minutes long to complete the ten tasks and four post task questions. With set up, explanation of the study, reading and signing the consent form each participant was through the session in under 35 minutes.
After each test the participant was invited to choose a library branded item to thank them for their participation in the testing. The items participants were able to choose from included: water bottle, thermos, power bank or a zippered technology pencil case.
Supporting Documents
Figure 1: Usability Test Script
Hi, (participant’s name). My name is Janie, and I am going to be walking you through this usability session.
Today we are testing the Libraries’ website to see what it is like for actual people to use the site. To do this we will give you a list of questions to answer using the website. We will observe and record where you go and what you do as you try and answer the questions. I will answer questions you have as we go through the test.
Please be aware that we are not testing you, we are testing the website. You cannot do anything wrong.
We are not part of the team that designed the website, so don’t worry that you will hurt our feelings. We want to improve them, so please be honest in what you think.
As we move through the usability session, I am going to ask you to think out loud, to tell me what is going through your mind.
We will record the computer screen and what you have to say. The video will only be used to help us figure out how to improve the website, and it won’t be seen by anyone outside of the Libraries. After the test we may ask you some follow-up questions.
I am going to ask you to sign a consent form before we begin the test. It simply says that you agree to participate in the testing and that we have your permission to tape you, but that the recording will only be seen by people who work at the Libraries.
What questions do you have before we begin?
Test
We are now going to begin the test. I will be reading the questions one at a time. If at any time you need clarification or have a question please stop and ask. Due to the fact that we are testing the usability and the design of the site we may not be able to give you additional information. If we can’t, we will tell you. You are free to use any searching technique that you are comfortable with, but we ask you to stay within the Library website. When you think you have answered the question, please tell me and I will go to the next question, or ask you to keep trying because you have not quite gotten to the proper information. Again, search like you normally would and remember you are not being tested on your abilities.
Tasks
Post Test Questions
Figure 2: Informed Consent Form
INFORMED CONSENT
Department of University Library
TITLE OF STUDY: LibGuides Template Usability Study
INVESTIGATOR(S): Darcy Del Bosque, Michael Yunkin, Susie Skarl, Kristen Costello
For questions or concerns about the study, you may contact Kristen Costello at Kristen.costello@unlv.edu.
For questions regarding the rights of research subjects, any complaints or comments regarding the manner in which the study is being conducted, contact the UNLV Office of Research Integrity – Human Subjects at 702-895-2794, toll free at 877-895-2794 or via email at IRB@unlv.edu.
You are invited to participate in a research study. The purpose of these study is to gather information on the use of subjects guides by UNLV graduate and undergraduate students. The information will be used to make subject guides more useful for UNLV students in the future.
You are being asked to participate in the study because you fit this criteria: you are a UNLV graduate or undergraduate student.
If you volunteer to participate in this study, you will be asked to do the following: participate in usability testing. You will be asked to perform a series of tasks related to subject guides and may be asked follow-up questions. A moderator will be present to facilitate the conversation. It is anticipated to take about half an hour to an hour of your time.
You will receive a choice of UNLV University Libraries merchandise (valued under $10). There are few other benefits to your participation, but you will receive some information about how to do research at the library. Your participation will benefit students in the future by enabling us to construct more useful resources for UNLV students.
Risks of Participation
There are risks involved in all research studies. This study may include only minimal risks. You may become uncomfortable when answering some of the questions during the usability testing.
Cost /Compensation
There will be no financial cost to you to participate in this study. The study will take about half an hour to an hour of your time. You will not be compensated for your time, but as a token of appreciation you will receive UNLV University Libraries merchandise. The University of Nevada, Las Vegas may not provide compensation or free medical care for an unanticipated injury sustained as a result of participating in this research study.
Confidentiality
All information gathered in this study will be kept as confidential as possible. No reference will be made in written or oral materials that could link you to this study. All records will be stored in a locked facility at UNLV for 3 years after completion of the study. After the storage time the information gathered will be destroyed. Participants may opt in/out of allowing audio/video recordings of their testing to be seen when results are presented within the library or at a national conference. No names will be linked to recordings.
Voluntary Participation
Your participation in this study is voluntary. You may refuse to participate in this study or in any part of this study. You may withdraw at any time without prejudice to your relations with UNLV. You are encouraged to ask questions about this study at the beginning or any time during the research study.
Participant Consent:
I have read the above information and agree to participate in this study. I have been able to ask questions about the research study. I am at least 18 years of age. A copy of this form has been given to me.
Signature of Participant Date
Participant Name (Please Print)
Audio/Video Taping:
I agree to be audio or video taped for the purpose of this research study. I am aware that segments or clips of the video may be viewed when the study results are presented.
Signature of Participant Date
Participant Name (Please Print)