Why We Experiment

The concept of the variable is central to research. A variable is simply a property which differs among members of a group. Some Torontonians are men, some are women, so sex is a variable in Toronto.

Experimental analysis compares variation in two or more variables. The variation in one of the variables is controlled by the experimenter. For example, a researcher investigating the effects of age on learning might compare only ten-year-olds and twelve-year-olds.

In experimental analysis the idea is to predict the uncontrolled variable with the controlled variable. Because it is the variable being predicted, the uncontrolled variable is called the dependent variable. The variable on which the prediction is based is the independent variable.

By controlling the independent variable a researcher can make comparisons more precise. For example, I recently drew a sample for a survey a client conducted of people throughout Ontario. One of the goals of the survey was to compare the opinions of people from six regions of the province. If I had just drawn a random sample from the entire province, some regions would have ended up providing far more members of the sample than others (40% of the population of Ontario, which has an area equal to the combined area of France and Spain, lives in Greater Toronto). The accuracy of any comparison between regions would depend on the number of people in each region; that means, for example, that comparisons between two regions with many members in the sample would detect far smaller diferences than comparisons between two regions with few members in the sample.

By oversampling in some regions and undersampling in others – that is, by drawing higher percentages of the population in some regions than in others – the samples from each region could be made roughly the same. Consequently, differences between regions would not be missed because of natural variation in the region in which respondents lived.

Calibration of the accuracy of estimate is the object of experimental control. Research is an attempt to estimate mathematical characteristics of a population, such as the percentage who hold certain opinions or the average age (the mathematical characteristics of populations are known as parameters). The accuracy of these estimates may be fairly accurately determined with mathematical formulas which assess characteristics of the sample and of the data. Failure to use these formulas often means that the research will either be too inaccurate to be useful, or so accurate that even tiny differences between groups are found to be statistically significant.

Why We Experiment © 1999, John FitzGerald