Improving Inferences Based on Survey Data Collected Using Mixed-mode Designs

Mixed-mode designs have become increasingly common in survey data collection. However, different modes may have different measurement properties, which need to be accounted for when analyzing mixed-mode data. This dissertation investigates the presence of mode effects in means and interviewer variances in both cross-sectional and longitudinal studies, and it develops methods to incorporate mode effects when making inferences. Specifically, Study 1 proposes three approaches to detect and address potential mode effects in cross-sectional data collected with randomized mixed-mode designs. We applied this work to assess face-toface (FTF) versus telephone (TEL) mode effects in a randomized mixed-mode experiment conducted in Wave 6 of the Arab Barometer Study (ABS). The methods developed in this study can offer tools for data collection agencies and researchers to analyze mixed-mode data. Study 2 examines whether interviewer variances remain consistent across different modes (e.g., FTF versus TEL) in two mixed-mode studies (the ABS and the Health and Retirement Study [HRS] 2016), representing different interviewer assignment schemes. The results can help inform interviewer training strategies and mixed-mode designs. Study 3 investigates mode effects in a longitudinal study when different mixed-mode designs are used across waves. Here, we considered the 2016 and 2018 waves of the HRS, since the HRS 2018 first introduced a sequential WEB-TEL mixed-mode design, alongside the typical FTF and TEL modes. Given that not all respondents would participate in the survey regardless of the mode used, we turned to the causal inference literature—specifically, principal stratification—to account for mode choice as a post-treatment observed variable. This study illustrates the application of principal stratification for mixed-mode inference, with the findings potentially guiding future mode assignment strategies. We examine mode effects cross-sectionally among respondents estimated to be able to complete the study in any of the compared modes; and we consider time effects within modes, again among respondents estimated to be capable of completing the survey in a given mode across both waves.