Function Follows Form
Program evaluation is often treated as if it were purely functional. The only question it is intended to answer is "Did the program work?" The answer to this question, however, may not even be informative if questions about form are not asked as well.
A few years ago a major American study of the effects of class size reported that reductions in the number of pupils per class had been associated with various academic improvements. However, the articles I read said nothing about just how those improvements were achieved.
We know from the research literature that reducing class size does not dependably produce improvements. If benefits are achieved, they must be due to some practice facilitated by reduction in class size, a practice which is not always used when class sizes are reduced. Formal analysis of what was going on in some of those classes could have established just how the improvement was produced. Reducing class size without this information cannot be expected to produce improvement dependably.
The formal analysis must be relevant to function, of course. I recently reviewed articles in a rehabilitation field which compared the success of different so-called models of rehabilitation. Some of these models were distinguished only by such characteristics as where the rehabilitation was provided (for example, in-patient versus out-patient rehabilitation). The site of rehabilitation, however, is likely to be far less important to the success of treatment than characteristics like type of rehabilitative technique and various characteristics of the disability (severity, for instance). If one site had been found to be superior to others (none was), the reason probably would have been a difference between sites in the type of rehabilitation offered or the type of disability treated.
At its best, formal analysis becomes theory, and that is the best foundation for any evaluation. A sound theoretical analysis provides clear definitions of success and clear specifications of the variables which it would be most appropriate to monitor in the evaluation.