Uncontrolled Variables (MR)

One factor that could cause the after measurement to differ from the before measurement is history. The amount of history in the example above, in which only 30 minutes separated the two measurements, is so small as to be an unlikely factor of importance. If, however, the two measurements were several days, or even months, apart, the effect of history might become major factor. For example, if a period of several months were involved advertising activities of other brands or changes in hair styles, such as a shift from curly to straight and sleek, could make major changes in the after measurement. Thus the longer the time period between the two measurements the greater is the danger that history will confound the results. All happenings outside the sphere of the experiment may be considered a part of history.

In the television commercial example, it is possible that the entire situation would become boring to some of the subjects. By the time of the after measurement they would react differently than at the before measurement regardless of what took place in the interim. Such biological or psychological changes that take place with the passage of time are called maturation. Experiments that require elaborate contrivances of an artificial nature and that require the willing cooperation of test subjects over a significant period of time are the most subject to maturation effect.

Pretest effect is a third factor that may endanger the validity of the before after design. In the television example, when the consumers were asked at the start to indicate the brand of shampoo they would prefer, they immediately knew even if they did not particularly think about it that the researcher was interested in shampoo. This could easily influence their responses to the after measurement. For example, some consumers might be stimulated to think about shampoos, to notice the hair of other consumers, or to discuss shampoos with other consumers. All of these might influence their response at the second request for brand preference. Simply remembering the brand preferred at the first drawing a consumer might automatically report the same brand or just to be different, report a different one to change one’s luck. Pre-testing can influence later measures in many ways. It is difficult, if not impossible, to know what the new effect of these influences might be, or even their direction. The more artificial and the more obvious the measurement process, the more effect it may be expected to have on later results. On the other hand, if the measurement process is not apparent to the experimental subjects (for example, of the measurement is the adding up of sales at the end of a day) then it will probably not influence results.

Measurement variation is another actor that may cause variations in the before and after measurements that may be confused with the effect of the experimental variable. Questionnaires are the most common measuring devices used in marketing research and they are notably inexact. In the shampoo commercial example the measurement variation might be small because respondents are in a controlled situation that doesn’t change between the before and after measurements, and no interviews are used. But in projects where interviewers are used and respondents may be in different circumstances in the before and after measurements, the possibilities of variation in the two measurements become much larger. Where measurements are made mechanically, however – when an electronic device records the times when a TV set is turned on – the measurements should have little variation.

The four factors that raise doubt as to the validity of conclusions from a before after design are history, maturation, pre-testing and measurement variation. These factors suggest the need of a control group against which to compare the results in the experimental group.

Comments are closed.