Handling Relative Values (Rankings)

I am including a special section on this topic, because there is a lot of confusion about the merits of using relative values, also known as rankings or ranks.

The relative values matrix in SquidMat’s predecessor embraced one of the most hideous ideas in all of decision-science folklore:

If one of your measures can only be expressed as relative values, then you must convert ALL of your measures into relative values.

There is a lot of bickering among those working in the decision sciences about how best to do things, but few (certainly no one I know) would offer up that curious piece of advice. It represents the kind of logic a doomed cult leader would employ.

· "I can’t be taken alive, so everyone has to drink the poisoned Kool-Aid."

· "This measure has to be expressed as relative values, so I must, therefore, convert every measure into relative values."


The ironic part is that the programs—SquidMat and its predecessor—don’t even have a separate procedure for handling relative values. In fact, they don’t even know you’re handing them relative values. They, and every other such program, believe that any number you give them is a "real" number (i.e., not a relative value/ranking). You haven’t caused the program to do anything different. All you’ve done is lie to it. "Here you go, Mr. Program. Have some nice, juicy ‘real’ numbers that just happen to have equal intervals between them." (Insert diabolical chuckle here.)

So let’s just knock it off.

OK, So We Have To Use Ranks…

Yes, there are times when the best we can do is use relative values. What we do in that situation is resign ourselves to mixing "real" number with ranks. As I said, these programs are going to treat the numbers as if they are "real." So if we absolutely have to include a measure that can only be expressed in relative values—fine. We accept the fact that we have to toss the analysis an inferior measure and move on.

But, for crying out loud, let’s not make it worse by converting everything else to inferior measures, too. It’s better to mix "real" and relative values.

A Better Plan

The problem with relative values is that they have no interval information. They merely show the order in which things are ranked, not how far apart they are. So I recommend that, when you can, use a scale to try and put some interval information back into the measure.

For example, I’m standing on a hill in Village A, which is quite far away from the ocean. And I can see from here that Village C is right next to the ocean and Village B is somewhere in between A and C. I don’t know how many miles apart things are, so my first instinct is to rank the three villages in order of nearness to the ocean:

· Village A: 3rd place = 3

· Village B: 2nd place = 2

· Village C: 1st place = 1

But I can also tell that Village B is much closer to C than it is to A. So I’m not comfortable with using relative values, which essentially tells the program that A is as far from B as B is from C. So I devise a scale of 1 to 5. I set C equal to 1 and A equal to 5. I then give Village B a value of 2:

· Village A: 5

· Village B: 2

· Village C: 1

Even though I have no idea what the actual miles are, which would be the best measure to use, I have improved on the rankings by reporting that Village B is much closer to C than it is to A. It’s a rough, subjective estimate, to be sure, but it’s a darned sight better than telling the program that the distance between villages is the same.

(It’s better because in the mathematical routines designed to mix apples and oranges— such as weighted sums of Z values used by SquidMat and the multiplication matrix used by SquidMat’s predecessor—ignore the absolute values given them and key instead on the intervals between those values.)

Notice that the range of the scale you use doesn’t matter. It can be 1 to 5, 1 to 10, or -437,989 to +1,382,993,753.44983. Everything gets rescaled to Z scores, anyway. So don’t sweat the scale itself.

Set the lowest COA value equal to the lowest value scale value and the highest COA value to the highest scale value. Then place the remaining COA values according to how you think they fall in between.

It’s as simple as that. Now you don’t have to lose any sleep over mixing relative values with "real" numbers.

For information on how SquidMat conducts an analysis compared with its predecessor, go here.