Model Metrics
Model Metrics
Embedded Files
Measuring the goodness of your design
Measuring the goodness of your design
How do we help modelers assess their models with respect to size, architecture, readability, and compliance to standards?
Research and Planning
Capturing the current pains
Capturing the current pains
I was paired with a developer to start this project. We established some high-level assumptions about the pains our users were experiencing.
I coordinated interviews with internal employees and external users to check our assumptions.
We discovered that users had too many disjointed tools to measure their model, but did not have a single pane of glass view. They didn’t understand the cause and effect of changes because the information was disjointed and spread across the system.
Design Sprint
Design Sprint
I facilitated my very first Design Sprint while working on Model Metrics. Gathering developers and cross-functional partners into a small team, we followed the Google Sprint workshop. The 5-day sprint created a space to explore the challenge and potential solutions. At the end of the week, we produced a live prototype.
The following week, I facilitated a round of usability tests with customers at our annual customer advisory board conference. Our findings showed that we were on the right path. A simple dashboard aggregating data to tell the user when their model was "good" was well received. Guidance for fixing their model when it was "not good" was key.
The Design Sprint also served as a valuable team-building activity.
Design Process
Book Club
The developer and I had never designed a dashboard before.
We started a book club to get a better understanding of dashboard design.
Stakeholders could be wowed by vibrant colors and unintelligible graphs, but we wanted to make sure our dashboard displayed useful and insightful information, no more and no less.
We set our focus on he most critical user workflows and adopted the following design principle:
Make the dashboard simple, clean, understandable and actionable. Users will have a only a minute or two to review and react
Low fidelity mockups
Low fidelity mockups
The developer and I worked collaboratively to wireframe several ideas to display the data. We codified datatypes: single number, part of a whole, percentage complete, histogram data, complexity, problematic areas, and number of inputs/outputs to select the appropriate visualizations for each.
We remained vigilant not to introduce color or unnecessary details into the early designs because we wanted to focus on content.
Low-ish Fidelity Mockups
For certain visualizations, we introduced just enough color to codify when needed
Heatmap
Heatmap
Red, yellow, green to convey severity levels
Complexity Histogram
Complexity Histogram
Shading to imply threshold breached.
High Fidelity Mockups
Because the developer and I collaborated so closely and actively evaluated javascript graphing packages together, there was no need to produce high fidelity mock-ups. The developer shared code as it was being built and we tackled design problems and technical constraints as they arose.
Research
Usability Testing
Usability Testing
I planned a usability study with participants across the automotive sector. The UX study was conducted during an advisory board conference, where we had access to many users. The results helped us sell the feature to key stakeholders.
My favorite parts
My favorite parts
My favorite part of this project was the high level of collaboration that I had with the developer. Even though he was located in Germany and I was on the East Coast of the US, we made the best use of overlap time and utilized videos to share ideas asynchronously.
Another part of the project I liked was using a visualization developed by a colleague at MathWorks, the Complexity Histogram, which conserved much-needed UI real estate.. As a bonus, it tested well in usability studies.
Complexity Histogram
Complexity Histogram
Imaging a birds-eye view looking down on the top of a histogram. The more opaque the color, the higher the bucket in the historgram.
Lessons Learned
If you decide to usability test a feature at a customer conference or advisory board, try to do it early before they have a chance to imbibe in a few too many drinks
Page updated
Google Sites
Report abuse