Evaluation

Alpha Testing

Student Interviews

Our alpha testing included one on one testing and interviewing where a user would test the application on our computer, and describe their UI/functionality struggles & suggestions.

Results

• We discovered users did not want the chat function to close. Many users requested that we leave chatrooms open for homework discussion, etc. after class is over (as opposed to just open question forum after)
• Users suggested that old messages show up when joining a new chat classroom, instead of having the chat start empty
• Wanted the ability to have multiple classrooms, tab between all of them

Beta Testing

Classroom Experience Sessions

We tested our full, updated application with students in two real lecture settings:

• CSC174: A small focus group of around 10 users were given the URL during lecture
• CSC171: The link was released to the full class of 180 students, although only about 40-50 were present at class time

Students were also encouraged to fill out a Post Testing response survey to fully document their experience. Data was pulled from this survey as well as observed feedback we collected as we monitored the classroom and chats during the testing sessions.

Results

• 16 total responses
• 66.7% rated the user interface experience 3/5 or higher

• 78.5% rated the functionality as 3/5 or higher

• Just over half gave the overall experience an average or above average rating
• About 4 in 5 students would consider using ChalkTalk again

• Felt the message center was too busy (date-time stamp, entering/leaving)
• Many students called for emoji integration

These extensive needfinding procedures guided us to the current development state our application is in today. We continuously refined our application throughout this evaluatory process and continue to do so now. We are very proud of our iterations and how we were able to successfully build upon past versions as we converge on the optimal user-focused product.