Research, evaluation, analysis


Questionnaires often ask people to rank items or topics by importance or interest or some other characteristic. For example a questionnaire might ask you to rank the importance of several social issues to you.

There are, however, more effective ways to assess attitudes or opinions. For one thing, when you assess them with ranking you end up with a lot more missing data than with other methods. That problem can be dealt with, but the variability in the data will still be reduced. That's undesirable because it reduces the chance of finding significant differences or relationships.

Ranks also have a non-normal distribution, which limits the number of statistical methods you can use to analyze them. Perhaps most importantly, ranks are only ordinal measures. For example, if someone ranks the importance of three items, we don't know that the difference in importance between the first-ranked and the second-ranked item is the same as the difference between the second-ranked and the third-ranked. The rater may actually consider the first two to be similar in importance, and the third trivial, or he may consider the first important and the second and third trivial.

A more effective way to find out how people rank the importance of items is simply to give them a list and ask them to pick the single item they consider most important. That way you end up with percentage estimates, which are ratio data and give you a good idea of the relative importance of the items.

This approach can be made even more powerful by using forced- choice items, which we will look at next week.

Ranks © 2000, John FitzGerald
Home page | Decisionmakers' index | E-mail