Pugh matrices are an effective tool in measuring and weighing the successes and failures of multiple designs
In order to create the best comparison and matching criteria, you may not want to wait until a design is finished. Pugh matrices can also be useful for testing elements of designs as well as the greater whole. This can save time that would usually be wasted creating new designs and help guarantee "innovations" actually improve from previous iterations.
Pure Baseline Method:
It is important that when you establish criteria that you choose something that is measurable and can easily be compared between the designs. More criteria will give you a better understanding on the successes and failures of your design
Next, establish baseline results for the designs. These are your expectations of success in the criteria. The baseline can be established from a central or basic design or your expectation of minimum success.
Additionally, make sure your baseline expectations are not too high or too low. If none of your designs are able to meet expectations for a criteria, you need to consider whether your base expectations are too high or if your designs are not yet up to par.
In a table, use 0 to represent base results for the criteria
Then denote if a design is the same as the baseline for that criteria (0), better than the baseline (+1), or worse than the baseline (-1)
For more precision, you could incorporate much better than (+2) and much worse than (-2) scores
Then, use those results to calculate a total score to decide the most successful design
If more than one Design has the best score like Design A and Design C, you may want to decide to use the weighted scale (see below) or include more criteria.
Weighted Baseline Method:
You can weight the criteria to prioritize those that are more important.
For example, if I considered Criteria 3 important, Criteria 1 twice as important, and Criteria 2 in between, my weighting system could look like this.
Then, you multiply the successes, failures, and meets expectations by the weight, and add your new total scores.
Now because of the weighting, I am able to see that Design A better fits my needs that Design C.
Pugh Matrices With Multiple Tests:
Depending on the criteria, you may want to do multiple tests. The raw data can be important to have, and the other Pugh matrices don't incorporate multiple tests.
One way you could format the matrix is below (with only two criteria and two designs for ease):
If your baseline for Criteria 1 is 2 cm (desire more than) and the baseline for Criteria 2 is 30 seconds (desire less than), the data could look like this:
In this matrix, it can be difficult to include a final score, so you could use more than one table.