Market Research Non response Errors

In almost every study, no response is obtained from a certain part of the sample – that is, from those refuse cooperation; those who cannot be located; and those who are unsuitable to interview, such as the ill, deaf, and senile. In most studies, it is assumed that the replies from those who are interviewed are also representative of the non-response group. In some cases this may be so, but in many instances the non-response group markedly from the group that cooperates. If this non response group is large, if may easily bias the results of the study.

As indicated above, non-response problems are increasing; in the 1960s many research firms obtained 80 to 85 percent completion of surveys with three or four call backs, but completion rates dropped to 55 to 65 percent in the 1970s, and even further in the 1980s. The non-response group usually includes a significant number of both not-at-homes and refusals. The results from two large, very carefully done surveys – one done by telephone and one by personal interview, both with probability samples. Seventeen or more calls were made in the telephone study to try to reach each respondent in the sample and eight or more visits were made to households to try to complete the personal interviews. This is a much larger number call backs than is usually attempted, but significant portions of both samples were still non-respondents. The components of the non-response group were roughly similar in the two surveys except for the no answer or not–at-home group, which included 20.7 percent of the telephone sample and only 3.3 percent of the personal interview sample.

Not-at-homes: The percentage of not-at-homes varies by city size, day to the week, time of day, seasons of the year, age, and the sex of the respondents, as well as with the provisions made to control not at homes in individual studies. It is however, almost always surprisingly large. The percentage of sample homes reached on each call for both the telephone and personal interview surveys mentioned above.

Only one fourth of the calls by either telephone or door-to-door were completed on the first attempt. More than 75 percent of the households ultimately reached for the sample had been reached by the fourth personal call and the fifth telephone call. This suggests why researchers have now concluded that three or four call backs are about the optimum number in most surveys.

Failure to obtain data from not-at-homes may bias survey results because population groups vary in the probability of being at home. Young males particularly are not likely to be home; but other groups that are also more likely than the average to be not-at-home include families with no children employed women, high income families, young people of both sexes, and those who live in large cities.

With quota samples on door-to-door surveys, the survey directors seldom have any knowledge of the not-at-home problem. In quota samples for shopping center interviews, of course, there are no not-at-home problems. When probability samples are used, the director is aware of the not-at-homes and must decide how much effort is desirable to get more of them into the sample. Because of the differences between the at-homes and not-at-homes, it is usually wise to make at least three call backs to interview the original not-at-homes.

George Gallup, Jr., in reporting on a survey among senior marketing information executives said ‘A key concern of all is the refusal rate in surveys’. Based on their own experience half of the survey respondents (51%) say the refusal rate is increasing, 37% report it is staying the same, while only 6% say it is decreasing. Another 6% do not express an opinion.

The refusals rate is climbing for many reasons, among them an overload of surveys, poor interviewing a distrust of strangers, time pressure, fund raising disguised as surveys, long questionnaires and impersonal computer interviews.