Research, evaluation, analysis

Estimating Value

One of the functions of decisions is to create value, and in this article we'll look at how value can be conceived of statistically.

In investment decisions value is easily defined. We get value when we turn a profit. A simple example of value can be provided by looking at a simple form of investment a bank account. When you deposit money in a bank account you hope to accumulate interest at a faster rate than inflation depletes the value of your money.

For example, if you deposit $1,000 for a year at 5% uncompounded, you expect (or at least hope) that inflation will be less than 5%. If inflation is 4%, you will end up with $1,050 at the end of the year, but those $1,050 will be worth only about 1,010 of the dollars you originally deposited. Nevertheless, you come out ahead, so the investment created value. Of course, if inflation is greater then 5%, the investment would have lost value.

We can extend this concept of value into other areas. For example, let's suppose you have instituted a training program which is intended to improve your employees' job performance. You compare the job performance of the first 100 employees to be trained with the performance of 100 employees who have not yet been trained. You find that 80% of the employees who had received training also received satisfactory job evaluations, while only 70% of the employees who had not been trained received satisfactory evaluations.

On the face of it, the training program would appear to produce value of 14% in job performance. Would we be justified in assuming this? To assume this, we must exclude the possibility that the differences between the two groups of employees are simply due to the luck of the draw. For example, if you flipped a coin ten times and got six heads, you probably wouldn't assume there was something wrong with the coin. Similarly, in evaluation you have to be able to estimate the probability that your results are simply a lucky accident.

You do that by applying a statistical test. Statistical tests simply estimate the probability of obtaining a result through the luck of the draw. If we apply a statistical test called the chi-square test to these results, we find that the difference is not quite large enough to conclude, by the most lenient common standard, that the difference between the two groups reflects anything other than the luck of the draw. The difference is big enough that you'd probably want to collect some more data before reaching a final decision, but the statistical analysis has shown you that the program is not demonstrably the success it might appear to be.

The investment example underscores the importance of accurate estimates of the factors likely to affect the success of your decisions. Investments are often justified by long-run returns, but the problem is that we live in the short run. For example, the average rate of return in American stocks since 1926 is strongly positive. If you had started investing in American stocks in 1928, though, you might well have been wiped out in 1929.

The same considerations apply to the training example. Even if the statistical test had been significant, continued monitoring of the program would be necessary. The factors affecting any program can be expected to change, so repeated evaluation of the success of the program is required. For example, the initial success of the program may be due to the ability of the particular staff providing the training rather than to the content of the program itself. If the original staff move on to other projects, the success of the program may be affected.

Once you have an accurate estimate of the value produced by a decision, you can perform a cost-benefit analysis to see how much that value is costing you.

Estimating Value © 2000, John FitzGerald
Home page | Decisionmakers' index | E-mail