A Difference-in-Differences Approach for Estimating Survey Mode Effects

TitleA Difference-in-Differences Approach for Estimating Survey Mode Effects
Publication TypeThesis
Year of Publication2023
AuthorsOllerenshaw, T
Academic DepartmentPolitical Science
Number of Pages39
Date Published6/24/2023
UniversityDuke University
CityDurham, North Carolina
Keywordsmode, survey responses

: As surveys increasingly rely on new modes, it is important that researchers understand
how mode influences survey responses. Two common designs for identifying mode effects are
cross-sectional approaches and experiments. But cross-sectional designs risk a combination of
omitted variable bias and post-treatment bias when conditioned on respondent characteristics that
are themselves mode sensitive. In theory, experiments can obviate these biases, but only when
the experiment occurs in tightly-controlled settings that avoid differential uptake across modes.
Considering the costliness and paucity of such experiments, in this paper, I propose a differencein-differences approach for estimating mode effects. Leveraging mixed-mode panel surveys,
mode effects can be identified by comparing changes in responses for panelists who switch
modes across waves to those who remain in the same modes. Difference-in-differences offers a
cost-free alternative to experiments and potentially large bias reduction gains vis-à-vis widelyutilized cross-sectional designs. I apply the difference-in-differences approach by estimating the
effects of completing live interviews vs. web surveys on racial attitudes and political knowledge
in the 2016-2020 ANES and on cognitive functioning measures in the 1992-2020 Health and
Retirement Study.

Citation Key13406