Problems in Marketing Experimentation

Experimental studies were rare in marketing before 1950. Since then the number of experiments has grown steadily and at an increasing rate. While the number of experiments is still small as a percentage of all marketing research projects, the advantages of experiments in determining cause and effect are so strong that one is safe in predicting continued expansion in their use. Some of the factors that limit the use of experiments have been brought out in the above discussion; however, there are additional practical problems that deserve comment.

Lack of Theoretical Base: True experimental cannot place without hypotheses to test. Such hypotheses are usually developed from underlying theory which in turn develops from extensive observation and description. Marketing research of the past been primarily descriptive; as a result; extensive information on Markets and marketing methods is now available. Some theory is now available, and conditions are ripe for ore extensive development in the future. This situation suggests that the researcher in the future will be able to develop more useful hypotheses and, will be able to conduct more experiments.

Time factors: Individual markets vary from one time to another; and the same is true, of course with people. Preferences and motivations change from year to year or even day to day. This dynamic nature of the consumer is particularly important because of the time factor involved in many marketing experiments. Sales tests must be given a considerable time period if complete results are to be obtained. How long after an advertisement is run its impact on sale is felt? No one knows for sure, but it is likely that some effect may carry over a period of several weeks or even months. If various tests are to be rotated among the same markets as in Latin Square designs, the time required for the entire experiment may run into many months, or even years. By the time such a test is finished, consumer opinions may have changed. As experiments are continued over longer periods, morality becomes a greater threat to validity.

Brief experiments measure only short term effects. Most experiments assume that the immediate results (e.g. sales) measure the effect of the experimental variable. But what are the cumulative effects of advertising that may build goodwill and consumer acceptance over a long period of time? Innovations in product design may be immediate successes, but they lose popularity in the long run. Other products may meet initial resistance, only to go on at a later date to become successes. Experiments should cover long enough periods to enable the after measurements to include more, or all of the effects of the variables. This is simply not feasible with many business problems, for decisions often cannot await the outcome of lengthy experiments.

Cost of Experimentation: Experimental research is often expensive. In most cases, at least and control group will be required in addition to the experimental group. In some instances, these groups will have to be measured twice or data will have to be collected continuously over a period. Thus the cost of an experiment tends to be greater than the cost of a descriptive study.

In some experiments control of the experimental variable is relatively simple (e.g. the number of shelf facings in the display of the product in the store); but in other instances control of the variable can be extremely expensive. Preliminary investigation of a possible experimental involving the substitution of local variables in a nationally televised commercial indicated that the cost of controlling the experimental variable in this manner would be $180,000.

Test marketing of a new product is another form of experimentation that may become very expensive. As the costs of introducing a new product into even a relatively small market may be quite high. The tendency is to test new products in only open or a few markets. This may keep costs within feasible limits, but it raises serious questions. Variation among markets is probably greater in many cases than variation in preferences among different test products.