Selected Experimental Designs

To begin the discussion of specific experimental designs it is useful to visualize an experiment in an over simplified form. The researcher has an hypothesis that if an experimental variable (e.g. advertising, shelf display, training) is applied to an experimental unit (e.g. a group of consumers, a store, some sales representatives), it will have a measurable effect (e.g. the number remembering the brand names, units sold, calls made). A plan is developed for controlling conditions pertinent to the experiment so that some experimental units can be exposed to the experimental variable and the results measured. The following are some of the most common designs for marketing experiments.

After Only Design: This is the simplest of all experimental designs; in fact, it should not be called an experiment, but by starting with this design, it will be easier to see the need for the more complex designs. As the after only name suggest, this design consists of applying the experimental variable (e.g. advertising) to an experimental group (e.g. consumers) and measuring the dependent variable (e.g. recall of brand name) after and only after; the application of the experimental variables. In the notation that will be used hereafter, this can be shown as follows:

Experimental Group

Experimental variables introduced Yes
After measurement Yes [x1]
Effect of experimental variable = x1 – implicit before measure

An example will illustrate. Roe herring was a traditional breakfast dish in Virginia, but it is no longer so widely eaten. In an attempt to widen the market, the firm owning the Tidewater brand ran an advertisement in the Sunday morning Richmond newspaper and in both the morning and evening newspaper during the following six days. This ad carried a coupon that could be exchanged at a grocery store for one free can of Tidewater roe herring. A total of 46,486 free cans were so claimed. The conclusion drawn from this study was that no other advertising medium could produce such immediate action at such low cost.

Presumably, the conclusion from the roe herring experiment is arrived at in the following manner. The 46,486 coupons redeemed is a large number undoubtedly much larger than the number of cans that would have been consumed during the same period without the advertising. If the advertising had been run in other media, it is unlikely that the results obtained would have been so large as those obtained with newspaper advertising.

Problems of validity are apparent in the above interpretation. The conclusion that the number of cans of Tidewater roe herring obtained by consumers through coupon redemption was much larger than the number that would otherwise have been purchased is based on an implicit comparison of the 46,486 figure with some idea of the number of cans that consumers otherwise would have obtained. Executives of the company have, on the basis of experience, a general idea of what sales of Tidewater roe herring would normally be during a period to that in which the advertising ran. It would be more scientific, however to make an explicit measurement of this normal sales volume rather than to rely on a vague implicit estimate. A further question is entirely ignored in the interpretation presented: namely, did the large number of free cans go to consumers who otherwise would have brought these cans although perhaps over a much longer period of time? A comparison of actual sales following the advertising period with normal sales for the same period would help to answer this question.

The after only design is even weaker with respect to the conclusion that newspaper advertising brought larger results than a similar amount of advertising in other media would have achieved. This conclusion is based on comparison of the results obtained from newspaper advertising with an implicit estimate of what would have been obtained from advertising in other media. The latter estimate is based on past experience with other media in other places and with other products and perhaps on general advertising philosophy. How much more scientific it would have been to divide the newspaper advertising money among several media in such a way as to permit measurement of the results actually achieved with each.