Research, analysis, evaluation

Getting Control

One of the fundamental ideas of scientific research is the making of controlled comparisons. A controlled comparison is, simply put, a fair one. For example, if you were comparing the aptitude of men and women for a particular field, you would want to compare samples of men and women who had similar training and experience. If one group had more training and experience relevant to the field than the other group the comparison would not be fair, and we would say that training and experience had not been controlled.

Some experimental designs incorporate what are known as experimental and control groups. These designs are used, for example, when you are investigating the effect of treatment or training. In a pharmaceutical experiment, for example, the experimental group might take a drug in pill form, while the control group would take an inert substance in pill form. The only known difference between the groups would be the contents of the pills (these groups are often matched for other characteristics known to affect the results).

I occasionally meet people who think that every experiment must have a control group. However, many experiments do not logically require one. If you want to compare the opinions of Canadian and Americans, for example, there is no need to assemble a control group of Mexicans. You would devote your effort instead to ensuring that the groups of Canadians and Americans do not differ in ways which are unrelated to nationality but which are related to the opinions you are investigating. You would not usually, for example, collect opinions primarily from young people in one country and a more representative sample of age groups in the other.

Of course, people do perform uncontrolled studies like that. Furthermore, with the number of journals there are around these days, they often manage to get these studies published. The adequacy of experimental control is something that you must be concerned with when reading reports of research. In fact, in fields like pharmaceutical research the comparability of results often is given intense consideration by reviewers and by other researchers in the same field, because in many experiments it is difficult to arrange.

The best way to find out how fair or controlled a study was is to read the method section (often known these days as the methodology section) of the original article or report. There is usually more than enough information there to permit an evaluation of comparability. Furthermore, the authors themselves will often provide full and frank assessments of the comparability of their results. If you are not given enough information to assess the comparability of results, then the study must be taken with a grain of salt. You are best to keep looking for a properly controlled study which got the same results.

Getting Control © 2000, John FitzGerald
Home page | Decisionmakers' index | E-mail