Optimizing Survey Errors and Survey Costs: A Risk-Conscious Stopping Rule and Timing Analysis

We introduce a risk-conscious stopping rule that considers uncertainty in predictions of survey costs and errors. The rule is particularly useful for repeated surveys with a few key statistics and tight budgets. It helps meet cost constraints by stopping a subset of cases early, aiming to minimize the negative impact on the quality of key statistics without adding new design features. To implement a decision rule that stops a subset of cases in the data collection process, a survey manager not only needs to choose which set of cases to stop, but also when to stop them. Implementing the stopping rule early may help to maximize cost savings, while decisions with reduced uncertainty can be made later as more data are collected. Dynamically identifying the “optimal” timing for implementing a stopping rule that relies on predictions can be difficult since future outcomes are unknown during data collection. We use data from the Health and Retirement Study to analyze how the timing affects the performance of the risk-conscious stopping rule. The Monte Carlo method is used to quantify uncertainty in stopping decisions. Our study illustrates several scenarios in which implementing the stopping rule after 8–10 call attempts has the lowest number of call attempts per interview while maintaining the same level of data quality.