In most marketing research experiments, the subjects (individuals, dealers, and so on) from whom information is to be obtained are selected by some sampling procedure. After the information required by the project is obtained, these subjects are not used again. In some instances, however, a sample is recruited and information is obtained from the members continuously or at intervals over a period of time. A permanent or fixed sample of this type is called a panel. Panels are used for both exploratory and conclusive studies. The procedure in using them is basically the same in each case; however, when used in an experiment, the panel must be viewed as having a design that can be depicted in the following manner:
Measurements are taken at intervals e.g. weekly reports by consumers on food products purchased, and experimental variables (e.g. a new package size) are introduced when desired. The result is a design similar to a series of before and after experiments.
Any of the measurements can be considered before measurements for the introduction of experimental variables thereafter. Similarly any measurement can be used as an after measurement for preceding variables. When used in this simple manner, panel data become essentially the same as the before and after design. As such panel data are subject to criticism with respect to the lack of control of history. Such use panel data is weak at best.
Better experimental design is achieved with panel data if the data are looked at as a time series – numerous measurements are made both before and after the introduction of the experimental variable. Trends can then be established as a base from which to measure the effect of the experimental variable.
To use panel data as an effective experiment it is important that experimenters predict in advance the trend that they would expect if the experimental variable were not introduced. This prediction becomes, in effect, a control measurement against which the measurement obtained after the introduction of the experimental variable are compared. Even so, it is clear that any of innumerable factors of history could influence the subject under study (consumer purchases) and, thus confuse the interpretation of the effect of the change in channels. Intimate knowledge and observation of the subject under study will enable the experimenter and executives using the results to make judgments as to the probability of uncontrolled variables affecting the outcome.
The practice of hunting through past panel data to find a time when sales (or another factor) changed and then attempting to find some casual factor that changed at the same time cannot be considered experimentation. Such practice may be good exploratory research, but hypothesis drawn in this manner must be subjected to more controlled study.
Factorial Designs: In the experimental designs that have been discussed, a single experimental variable with usually only one level was considered. It is possible to test several levels of the experimental variables for example, several ads could be tested, each with a separate experimental group. All but one group alternately could be considered as control groups against which to compare the experimental group, or an additional control group not exposed to any advertising could be used to protect against possible negative effects of all ads.
Factorial designs permit the experimenter to test two or more variables at the same time and not only to determine the main effects of the each of the variables, but also to measure the interaction effects of the variables. Consider the problem of determining the proper concentration of sugar and flavor in a soft drink. A simple approach would be to make up a batch of the optimum mixture as judged by the producer and to have a sample of consumers taste it and competing products and indicate an order of preference. The consumers might even be asked to comment on the degree of sugar and flavor in a soft drink. Another approach would be to make up several batches with differing levels of sugar content, but with the flavor held constant. Consumers could then taste a sample of each and indicate a preference. Sugar could then be held constant and flavor varied.