Innovative design and analysis approaches for addressing increasing rates of attrition

All network activities in this second thematic area will focus on: 1) innovative design approaches for maintaining or increasing response rates in longitudinal studies of aging, and 2) state-of-the-art analytic approaches to adjusting survey estimates for attrition that may be non-ignorable in nature, which could lead to biased population estimates of trajectories in health outcomes that are of substantive interest. From a design perspective, methodological innovations in this area will focus on mixed-mode study designs (e.g., optimizing models for assigning participants to different data collection modes and contact protocols); methods to minimize respondent burden (e.g., questionnaire design features such as modular survey design; online schedulers); incentive strategies (e.g., cost-efficient approaches such as two-phase sampling; panel management; targeted incentives); respondent contact protocols (e.g., messaging; periodicity of contact attempts and types); and more efficient screening techniques for aging populations that are designed to improve screener response rates (e.g., use of commercial data to over-sample households with age-eligible individuals). From an analytic perspective, methodological innovations in this area will focus on appropriate adjustment of baseline and time-varying sampling weights to account for differential rates of attrition along key population subgroups; weighting and imputation methods for adjusting estimates of trajectories to allow for both ignorable and non-ignorable attrition; and estimation techniques for non-probability samples of small minority subgroups followed over time.

Bibliography

Biemer, P.P., Harris, K.M., Burke, B.J., Liao, D. & Halpern, C.T. (2022) Transitioning a panel survey from in-person to predominantly web data collection: Results and lessons learned. Journal of the Royal Statistical Society: Series A (Statistics in Society), 185, 798– 821. https://doi.org/10.1111/rssa.12750

Summary: In recent years, costs associated with in-person interviews have increased, while the data quality advantages typically associated with this data collection method have lessened. Therefore, many longitudinal surveys have begun to transition from in-person to web data collection, which has prompted the proposal of a multi-sample, multi-phase responsive design in an effort to minimize data quality risks associated with the transition from in-person to web-based data collection.

Goodman, A., Brown, M., Silverwood, R.J., Sakshaug, J.W., Calderwood, L., Williams, J., et al. (2022) The impact of using the Web in a mixed-mode follow up of a longitudinal birth cohort study: Evidence from the National Child Development Study. J R Stat Soc Series A, 822– 850. Available from: https://doi.org/10.1111/rssa.12786

Summary: A mixed-mode design utilizing both telephone and web participation was investigated in the National Child Development Study in 2013. Compared to using telephone-only data collection methods, the offer of a web survey increased overall participation rates by 5.0 percentage points.

Maslovskaya, O., Struminskaya, B. and Durrant, G. (2022), The future of online data collection in social surveys: Challenges, developments and applications. J R Stat Soc Series A, 185: 768-772. https://doi.org/10.1111/rssa.12895

Summary: Online data collection has become increasingly popular as widespread use of technology has increased, especially during the COVID-19 pandemic, as face-to-face data collection was not an option. Online data collection has some unique challenges, such as coverage issues, data quality, and measurement problems. 

Ofstedal, M.B., Kézdi, G. and Couper, M.P. (2022) Data quality and response distributions in a mixed-mode survey. Longitudinal and Life Course Studies, -1(1-26)

Ofstedal, M.B., Kézdi, G. and Couper, M.P. (2022) Data Quality and Response Consistency in a Mixed-Mode Survey (PDF). Presented at the Panel Survey Methods Workshop, Lausanne, Switzerland.

Sastry, N. & McGonagle, K.A.(2022) Switching from telephone to web-first mixed-mode data collection: Results from the Transition into Adulthood Supplement to the US Panel Study of Income Dynamics. J R Stat Soc Series A, 933– 954. Available from: https://doi.org/10.1111/rssa.12840

Summary: In an effort to reduce costs and increase respondent cooperation, web interviews were conducted in the 2019 wave of the Transition into Adulthood Supplement (TAS) to the US Panel Study of Income Dynamics (PSID). Compared to using telephone-only interviews, it was found that there were higher response rates, interviews were conducted faster, and data quality was higher and with lower associated costs when using the web-first approach.

Schwarz, H., Revilla, M. & Struminskaya, B.(2022) . J R Stat Soc Series A, 981– 1003. Available from: https://doi.org/10.1111/rssa.12857

Summary: As more surveys are conducted, respondents are likely to have previous survey experience, which can make the use of incentives important for continued participation. Quality of survey responses does not seem to be associated with previous web survey experience, or participating only to receive an incentive.