Horizontal or Vertical? The Effects of Visual Orientation of Categorical Response Options on Survey Responses in Web Surveys

TitleHorizontal or Vertical? The Effects of Visual Orientation of Categorical Response Options on Survey Responses in Web Surveys
Publication TypeJournal Article
Year of Publication2019
AuthorsHu, J
JournalSocial Science Computer ReviewSocial Science Computer Review
Volume38
Issue6
Pagination779 - 792
Date Published2020/12/01
ISBN Number0894-4393
Keywordscategorical response options, horizontal orientation, vertical orientation, web surveys
Abstract

Prior research has studied the impact of horizontal versus vertical orientation of rating scales on survey responses, but not the effects of orientation of categorical options. The present study attempts to fill this gap by investigating the orientation effects of categorical options on response burden and response quality and by studying how the effects of orientation of response categories interact with cognition capacity. The data are drawn from an experiment embedded in the 2013 Internet Survey of the Health and Retirement Study (HRS), in which respondents were randomly assigned to a question with either horizontally or vertically aligned response categories. Response burden is measured by response time, and response quality is measured by reliability, primacy effect, and item missing on the question. Horizontal orientation is found to be more burdensome than vertical orientation on the web for HRS respondents, but there is no significant difference between the two orientations on primacy effect and item missing, and the two orientations are equally reliable. The effects of cognition capacity are found to interact with orientation on both response burden and response quality on the web. Compared to vertical orientation, horizontal orientation leads to longer response time but less primacy effect for people with lower cognition capacity versus people with higher cognition capacity. However, the effects of the orientation and cognition capacity do not interact on item missing. Vertical orientation for categorical response options in web surveys is recommended in general, but more caution should be exercised if respondents are elderly and primacy effect is a main concern.

DOI10.1177/0894439319834296
Citation Key11139