At first, one is inclined to think that the construction of a questionnaire is simple. All that is necessary is to write down the questions to be asked and have them printed on sheets of paper. Unfortunately, the problem is deceptive. It is easy to write down questions but are they the right questions? Will a question mean the same thing to Mrs Longstreet in Fort Collins, Colorado, as it does to Mrs Vanderbilt in New York City? Will it mean the same to either of them as it means to the researchers? Slight variations in question wording may make a considerable change in the answers obtained. Which is the correct alternative? If one question is asked before another, it might get a different answer than if it is asked after the other. If some questions are asked, respondents may refuse to answer other questions incorrectly.
Questionnaires are the measuring instruments used in marketing research. If they are to produce measurements, they must be both valid and reliable. In one public opinion survey, the following question was asked with the results shown:
Which of the following statements most closely coincides with your opinion of the Metallic Metals Act?
Thirty percent of the respondents had no opinion; the other 70 percent distributed their answers as follows:
It would be a good move on the part of the United States (21.4 percent).
It would be a good thing but should be left to the individual states (58.6 percent)
It is all right for foreign countries but should not be required here (15.7 percent)
It is of no value at all (4.3 percent)
If these data were included with other information and presented in a leather bound report, they would be impressive evidence in favor of the second alternative. Of course, the point is that there is no such thing as a Metallic Metals Act. Questions will get answers, but do the answers mean anything?
Political differences over American involvement in Vietnam in the late 1960s and early 1970s led many politicians to conduct polls among their constituents to determine the latter’s attitudes on the question. Two such polls, both conducted in New York City at about the same time, asked the following questions, with the results shown.
Do you approve of the recent decision to extend bombing raids in North Vietnam aimed at oil reserves and other strategic supply depots around Hanoi and Haiphong?
No or don’t know 34
Do you believe the United States should bomb Hanoi and Haiphong?
No or don’t know 86
While the wording of these questioning differs slightly, they seem to be getting at the same issue. They leave one less than sure what the “true” sentiments of the public are on this question. The questioning is not reliable as it produces different results the same thing is asked in different ways.
In May 1975 the British government was faced with a decision to remain in the European Common market or to with draw. The prime minister decided to have a public referendum on the issue. He stated that he wanted a single straightforward question that the public could answer yes or no. One can hardly imagine a more crucial question- wording problem. Many different versions were proposed, but the one finally selected was the following:
Do you think that the United Kingdom should stay in the European Community (the Common Market)?
The authors of this text do not believe this was the best possible wording. After studying, students may want to try their hands at improving the question.
As the above discussion indicates, the questionnaire can be the source of many errors in a survey, probably more errors than other aspects of survey research. These errors can be summarized under the following headings:
Memory – the respondent can’t remember the incident or remembers incorrectly
Motivation – the respondent may be motivated to report incorrectly to provide a better image of him self.
Communication — the respondent may not understand accurately what he is asked.
Knowledge – the respondent simply may not know the answer