Respondent Selection Errors

Telephone Surveys: In telephone surveys the interviewer is typically given a list of numbers to call, or numbers are dialed on the basis of random digit dialing. If the interviewer in the former case is also given the name of the individual at each number to whom she is speaking, there is no problem, as the interviewer simply asks for that individual by name. Unfortunately, names are seldom available, and the interviewer must select the individual to be interviewed at the number called – usually a household. If a sample is to be obtained that is representative of the population, some random or systematic method of selecting the person to be interviewed is required. All procedures for selecting the individual to be interviewed pose a problem; the more sophisticated they are, the more time they take, and the more likely they are to lead to refusal by the respondent. This is discussed further below in connection with the non-response problem.

Mall Intercept Interviews: In the case of shopping center interviews, respondents often are selected by convenience. It cannot be said that errors are made in respondent selection when this is the case, but the procedure is often biased because interviewers are likely to select those individuals who look friendly and appear easy to interview. To introduce more objectivity into the process, specific times and locations within the shopping center can be selected randomly and it can be specified that the first person passing a given point after each interview or attempted interview will be sought as the next respondent.

Door-to-door Surveys: When quota samples are used in door-to-door surveys, interviewers select the individuals to be interviewed subject only to quotas for various population groups such as sex, age, and income. This interviewer control of the selection of respondents is unlikely to result in the equivalent of a random sample. Interviewers tend to follow the paths of least resistance and of greatest convenience. One study showed that, when interviewers were given economic level quotas, they tended to under select in both the high and low income classes. Errors or falsification also occur in classification; interviewers who classify the same respondents on the basis of income may differ in as much as 30 percent of the cases. Interviewers tend to select households they ‘like’ and they think higher incomes, even when controls are used to discourage this.

An effort is often made to overcome this respondent selection bias by setting up more elaborate controls or quotas. This can create biases of another type. When the number of controls gets beyond three or four, the filed worker finds difficulty in locating respondents who meet all the characteristics prescribed. As a result, interviewers tend to ‘push’ a 35 year old into the 40 and older age group or to ‘force’ their quotas in other ways. Realistic quotas, adequate training on procedure and direct supervision in the field are the best ways of preventing such errors.

One of the common reasons for the use of probability samples is that they eliminate the bias that comes from interviewer selection of respondents as described above. For telephone interviews this is largely true, but for door-to-door interviews the use of a probability sample provides no guarantee per se that respondent selection bias will be eliminated. The use of such a design may cause different biases, such as errors in listing dwelling units, in selecting dwelling units and in selecting individuals within dwelling units.

Evidence indicates that in listing dwelling units interviewers tend to under list in low income blocks. One study found a 13 percent under listing in such blocks. Once dwelling units have been listed the specific dwelling units to be included in the sample must be selected. When field workers make dwelling selections from listings in the field, they tend to come up with fewer low income homes than when dwellings are selected randomly in the office.

Interviewers also tend to select the more accessible individuals in the household in both telephone and personal interviews. Despite instructions that indicate the random procedures to follow in selecting respondents within the household interviewers will ‘rig’ the selection system to prevent making callbacks for the not-at-homes.