skip to content »

Validating a survey instrument

validating a survey instrument-14

A low response rate can be devastating to the reliability of a study (Benson, 1946; Phillips, 1941; Robinson, 1952). "Factors related to survey response rates." Journal of Applied Psychology 9-251. Fortunately, "low response rates are not an inherent shortcoming of mail surveys", and the researcher must do everything possible to maximize response (Berdie, Anderson, and Neibuhr, 1986, p. Much of the research in questionnaire methodology has centered around techniques to maximize response. "Response rate to mail questionnaires with a return deadline." Public Opinion Quarterly 4-375.

validating a survey instrument-47validating a survey instrument-15

When respondents receive a questionnaire in the mail, they are free to complete it on their own time-table (Cahalan, 1951; Jahoda, et al., 1962). 58) The most successful follow-ups have been achieved by phone calls (Roscoe, Lang, and Sheth, 1975; Sheth and Roscoe, 1975; Speer and Zold, 1971). Many researchers have examined whether postcard follow-ups are effective in increasing response (Cox, Anderson. "Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature." American Sociological Review 7-462. On the other hand, the lack of an interviewer limits the researcher's ability to probe responses. "The negative effects of personalization on response patterns in mail surveys." Journal of Marketing Research 4-117. Structured questionnaires often lose the "flavor of the response", because respondents often want to qualify their answers (Walonick, 1993). However, Jones and Lang (1980) point out that increasing the response rate does not necessarily improve the precision of survey results. "Effectiveness of ingratiation tactics in a cover letter on mail questionnaire response." Psychonomic Science 9-351.

Following up on Nonrespondents One of the most powerful tool for increasing response is to use follow-ups or reminders (Scott, 1961; Toops, 1924).

Low response is the curse of statistical analysis, and it can dramatically lower confidence in the results.

While response rates vary widely from one questionnaire to another, well-designed studies consistently produce high response rates. "Initial returns on mail questionnaires: a literature review and research note." Research in Higher Education 1-367.

Many investigators have reported that interviewer voice inflections and mannerisms can bias responses (Barath and Cannell, 1976; Benson, 1946; Boyd and Westfall, 1965, 1970; Cahalan, 1951; Collins, 1970; Dohrenwend, Colombotos, and Dohrenwend, 1968; Franzen and Lazersfeld, 1945). "The effects of source and appeal on mail survey response patterns." Journal of Marketing Research 4-378. Minneapolis: University of Minnesota Agricultural Extension Service.

Written surveys are not subject to this bias because there is no interviewer.

Unlike other research methods, the respondent is not interrupted by the research instrument.