Stating the question in collection of Market Data

In asking questions, interviewers are usually expected to follow certain rules. A typical set of such rules run as follows:

1. Each question must be asked exactly as worded.
2. The interviewer must not comment on the question meaning or indicate in any way what kinds of answers might be acceptable.
3. Every question must be asked unless the interviewer is instructed to do otherwise.
4. Questions must be asked in the same sequence as given on the questionnaire.

There is considerable evidence that even with a relatively simple straight forward structured questionnaire interviewers do not always follow the above rules.

If a question is not easily understood by respondents interviewers will rephrase it in ways which will cause less confusion. Since no two interviewers are apt to rephrase a question in the same way, different respondents get different stimuli that produce variations in replies. A variation of this problem occurs when interviewers are instructed to probe with additional questions to secure more complete answers. In either case, the addition of a comment by the interviewers raises the possibility that the respondent will be influenced in a different way than other respondents. Failure to probe adequately, when probing is necessary to obtain a meaningful answers, is one of the most serious sources of interviewers error. The average umber of errors of various types made by interviewers per question asked in one carefully controlled personal interview survey.

Average Number of Errors in Stating Question by Error Type in personal Interview Survey

Type of Error Number of Errors per

Reading 0.293
Speech variations 0.116
Probes 0.140
Feedback to respondents 0.161

Questions that state the alternative answers, such as multiple-choice questions, are particularly subject to interviewer bias. This bias occurs because the interviewer puts too much emphasis on one alternative in stating the question. The interviewer’s method of asking questions will also influence results on other, more subtle ways. For example, slight variations in tone of voice will change the entire meaning of some questions.

Some field-interviewing organizations require the interviewer to indicate in the questionnaire what additional comments were made in order to elicit satisfactory answers. Presumably, these are evaluated later to determine whether they based the respondent sufficiently to warrant elimination of that questionnaire from tabulation.

In some interviewing situations, such as in group interviews a tape recorder may prove helpful in reducing bias by recording both questions and answers verbatim. Experiments have shown that, contrary to general expectations, respondents will accept the use of tape recorders. The most critical element in gaining acceptance of the recorder is the interviewers; own attitudes toward it and resulting behavior. The recorder, however, may introduce bias. One study found that tape recording increased the accuracy of reported responses of lower class respondents but reduced the reported accuracy of middle and upper class respondents.

Bias from the interviewer method of stating the question is lower in telephone surveys than in personal interview surveys, but there are more subtle factors involved than just the method of communication. Fewer interviewers are needed in telephones studies, but this means that “averaging out” if such interviewer bias is less apt to take place in telephone surveys. On the other hand, the smaller number of interviewers and their concentration at one location makes more training feasible and permits supervisors to listen in to a sample of each interviewer’s work and, thus to identify interviewers who are deviating from established procedures and to take corrective action. Computerized telephone surveys appear to be reducing this type of interview bias even further, but more experience is needed before definite conclusions can be drawn. —