As Karl Popper has observed, researchers in the natural sciences who are investigating the same subject are able to conduct experiments with the same initial conditions, while researchers in the social sciences usually are not. In the social sciences you are investigating how people behave, and people differ in so many ways that affect their behaviour that it's unlikely that two researchers can ever draw equivalent samples.

This is a particular problem in testing. Tests which have been shown to be reliable and valid elsewhere may not be reliable and valid for the group you are testing. Early in the century the low scores of immigrants to the United States on intelligence tests were often interpreted as a sign of the general inferiority of immigrants. In fact they were probably due to immigrants' lack of familiarity with both the language of the tests and the subjects of the questions (identifying a city's professional baseball team, for example).

So if you are testing you should always assess the reliability of your test instead of assuming that it's reliable. If you are making a decision based on research findings you should incorporate safeguards in the implementation of your decision to ensure that the relationships observed in previous research are true of your group as well.

For example, if you are basing a decision about how to motivate engineers on previous research with physicians and truck drivers you will want to be able to check that the engineers are responding to your attempts to motivate them in the way the physicians and truck drivers did to other people's attempts elsewhere. Even if you're basing your decision on research with engineers at another company, there are probably plenty of differences between that company and yours, and you will still want to verify that your decision is having the desired effects. In the next article we'll look further at how you can do that.