Overview

At Duke University in 2016, most instructors in electrical engineering and computer science departments were forced to individually run student submissions one-at-a-time and then report grades by whatever means they choose -- and this process can take weeks for each assignment. AG350 is an auto-grading web app used by students and teaching faculty in Duke’s Electrical Engineering and Computer Science departments aimed at entirely streamlining this process. The app allows users to submit assignments, evaluate code using built-in tests, and assess grades.

During the 2019 spring semester at Duke, I was the lead UI & UX designer for the AG350 team. I worked directly with the CEO to redesign aspects of the existing product, as well as carried out exploratory user research, user flow design, and wireframing for new product features.

The challenge(s)

When I joined the team, AG350 was still in early stages. The product allowed students to submit and build their assignments (in other words, run their code). But an “instructor view” of the product that allowed faculty to evaluate class assignments, and an interface for exploring test data to see what errors occurred in the submissions were still just goals in the product roadmap. Beyond designing the features to address these issues, my overall focus with this project was answering: How can we reduce confusion in the routine submission and grading experiences of students and teaching assistants? (In other words, how could I help users easily answer questions like “How did my students do overall on this assignment?” or “How can I improve my code?”)

So I was really focused on 2 challenges:

Faculty need a streamlined, transparent way to assess student progress so that they can reduce hours spent grading and have more time to teach.

Students need a simple, time-efficient way to build assignments and evaluate course progress so that they can spend more time on extracurriculars, their social life, other homework, etc.

I worked with the team to create solutions that were clean, accessible, and intuitive.

Research

I began the design process by gathering information on users, their pain points, product functionality, and so on. The foundation of knowledge I formed from this stage allowed me to give a face to the users and note some key issues to address. Both the AG350 team and I highly prioritized accessible design, and they collaborated with me to find users with unique accessibility needs.

My research methods included:

Several rounds of user interviews with students and TAs to learn about user behavior.
Open ended conversations focused on understanding the larger systems AG350 fits into with colleagues who have been students and/or TAs.
A 5 question survey focused on understanding how AG350 is used now and any shortcomings.

User personas

Alex, ECE student

Alex signs into AG350 whenever he wants to gauge how far along he is on his assignment. Alex makes sure to pass all test cases for his final submission so he can receive a good grade, and he might submit an assignment 3 or 4 times. Due to his visual impairment, it's important that the software communicates data simply and has contrasting colors.

Shelby, TA

As a TA busy with her own homework, Shelby would like to view all student assignments seamlessly. Shelby wants to help students discover the bugs in their code as quickly as possible. Outside of office hours, she logs on to AG350 every now and then to check if assignments are submitted.

Key findings

TA’s want to view aggregate grade data and stats. Having an overview can help summarize grade trends.

TA’s cannot view assignments and grades for all students from a central page. The current process requires several steps and visiting each student’s Github page.
TA’s can run thousands of tests and can’t tell students which tests they failed. There is no way to parse data.

Students are confused about what defines an “unsuccessful” build. There is no feedback explaining if the build failed to run versus if it ran and had errors.

When a build is in progress, a user would have to refresh the page to see an update. It’s a guessing game to see how long it takes.

Defining goals for success

My conversations with the AG350 team allowed me to establish goals that were prioritized for the product’s success. And based on the learnings from my research, I added more goals to give my process a meaningful direction and further define what success would look like:

The design is accessible and removes points of exclusion to reach a diverse audience → there’s a focus on being high in contrast, RG color blind friendly, & screen reader friendly.
Grade data is visually displayed to students in non-stressful and clear way.
Instructors can understand meaningful information concerning student grade data in seconds.
Instructors are able to search through assignment data and find sources of error.

Ideation and initial wireframes

My initial wireframes were pretty exploratory. At first I designed an assignment overview page that displayed test results with either green bars and “Passed” or red bars and “Failed”. But discussion with the team led me to learn that test cases can be complex and are better accompanied by numeric scores. Additionally, with the goal to display data in a way that both did not stress out students and was accessible, we decided to scrap red & green visuals. These would emphasize shortcomings and would be poorly suited for RG color blind folks.

Journey map

I mapped out a user journey to illustrate how users would likely engage with AG350. This helped give vision to the project and pinpoint common user tasks.

The solution

AG350 has a whitespace-heavy look accompanied by a few colors to give a clean and professional look for this educational tool. The contrast ratios produced by the colors in its various text and background combinations passes WCAG 2.1, AA compliance standards for text and color contrast.

Student course assignments & assignment overview

I discovered that most folks currently do not use AG350 for more than one course, and that on average, a single course doesn’t assign a large amount of assignments through AG350. Therefore, I removed the course drop downs on the course assignments page, and now at first glance a student can view upcoming tasks.

The assignment overview page communicates essential submission info (like test case results) and holds links to the Github repository. The overall score is given visual dominance to quickly communicate grade outcome. In early iterations I opted for “Help” and “Build” buttons for the code commit section that were styled as links, however not only can these be generally misleading to everyone, but individuals using screen readers trigger links and buttons with different actions. To avoid unnecessary friction for this group of users and overall confusion about interaction behavior, the buttons are clearly designed as such.

Code commits section

Potential misunderstandings are diminished by the progress bar that would actively signal completion status and explain why a build would not be successful if that occurred. Visual emphasis is given to commit descriptions to remind users what the added code’s purpose was without forcing users to visit their Github repository directly.

Instructor view

Often, teaching faculty like to report score statistics to their class, and on this page they can break down this data. The box and whisker plot accompanies the histogram here to visually supplement the data and display the range of quartile values. TAs can also view individual submissions and grades in the centralized roster when students come to office hours for help.

Build details page

Within the build details page, users can search through test cases and view which “assertions” were passed within a test, thereby helping TAs discern shortcomings in the code. And test data which can be complex and include unspecified data types is displayed in list format.

Results

This version of AG350 was released to hundreds of Duke students in the 2019 Fall semester as it expanded to various Computer Science and Electrical Engineering classes (about 30% of the student body).

Previously, if teaching faculty wanted to evaluate their students’ progress in their assignments or point individual students to issues in their code, they needed to do it manually. Along with designing the interfaces to improve these processes, I enhanced existing features in AG350 to improve student and instructor understanding and reduce friction.

Learnings

Design decisions can be effectively defended when they’re done to advance inclusivity and accessibility.
Accessibility can act as an anchor for design to reduce complexity (“Which colors do we need and want to use that have high enough contrast?”).