Innovative Design and Analysis Approaches for Addressing Increasing Rates of Attrition
All network activities in this second thematic area will focus on: 1) innovative design approaches for maintaining or increasing response rates in longitudinal studies of aging, and 2) state-of-the-art analytic approaches to adjusting survey estimates for attrition that may be non-ignorable in nature, which could lead to biased population estimates of trajectories in health outcomes that are of substantive interest. From a design perspective, methodological innovations in this area will focus on mixed-mode study designs (e.g., optimizing models for assigning participants to different data collection modes and contact protocols); methods to minimize respondent burden (e.g., questionnaire design features such as modular survey design; online schedulers); incentive strategies (e.g., cost-efficient approaches such as two-phase sampling; panel management; targeted incentives); respondent contact protocols (e.g., messaging; periodicity of contact attempts and types); and more efficient screening techniques for aging populations that are designed to improve screener response rates (e.g., use of commercial data to over-sample households with age-eligible individuals). From an analytic perspective, methodological innovations in this area will focus on appropriate adjustment of baseline and time-varying sampling weights to account for differential rates of attrition along key population subgroups; weighting and imputation methods for adjusting estimates of trajectories to allow for both ignorable and non-ignorable attrition; and estimation techniques for non-probability samples of small minority subgroups followed over time.
Current Critical Directions for Future Research on Innovative Design and Analysis Approaches for Addressing Increasing Rates of Attrition:
- Differential Attrition: What are the different factors driving attrition for different subgroups of aging populations?
- Imputation / Weighting: What are the best statistical methods for imputation of missing data and weighting adjustment due to wave nonresponse in longitudinal studies of aging?
- Incentives: How can we optimally deploy incentives to secure engagement and participation? How are incentives impacting various sources of survey error, and what are reasonable / effective incentives for using new technologies?
- Institutional Trust: How does mistrust in governments / institutions impact consent decisions and retention?
- Interviewer Labor Models: Interviewers play an important role in motivating participants in longitudinal studies of aging to remain active and engaged in the studies, and also in the recruitment of new participants. Unfortunately, it is becoming harder to hire, train, and retain high-quality interviewers for these studies. What labor models for interviewers are best at reducing interviewer attrition (and in turn attrition for participants in interviewer-administered longitudinal studies, due to unstaffed areas, insufficient interviewer hours, etc.)?
- Interviewer Trust: What are the effects of interviewer trust on survey response rates?
- Recapture: How can we best recapture individuals who have temporarily not responded to a study? Which approach works best (e.g., low vs. high burden, mode, outreach method, incentives) to try and bring them back in?
- Respondent Engagement: Which methods to keep panelists engaged in our studies work best (e.g., newsletters, return of results) to improve retention and benefit participants? Is there heterogeneity in what matters most for different subpopulations?
- Survey Burden: How exactly does respondent burden / survey length affect response rates and rates of attrition? Do longer or more frequent surveys actually keep panelists more engaged in longitudinal studies of aging? Do shorter interviews / instruments used to recruit individuals into a panel study, with longer instruments reserved for once they are included in the study, result in improved recruitment across populations? Does this vary by survey mode?
Bibliography
All bibliography entries below are tagged with colored shapes corresponding to the major thematic research areas of NIMLAS. Specific critical topics for future research that the particular product within each area is addressing are provided in text next to the colored shapes.
Data collection methods for improving representation
Addressing increasing attrition rates
New measurement technologies
Consent to additional data collection
Improving measurement in longitudinal studies of aging
| Author | Title | Source | Summary | Critical Topics Tag |
|---|---|---|---|---|
| West, B.T. and Schoeder, H.M. (2026) | A Complete Example of the SRC Data Quality Profile from the 2022 Wave of the Health and Retirement Study (HRS) | SRC Seminar Series presentation, April 21, 2026, Ann Arbor, MI. |
This presentation provides a holistic evaluation of the quality of the data collected in the 2022 wave of the Health and Retirement Study, applying the total survey error perspective for assessing data quality outlined in West and Wagner (2025). Overall, the results suggest high quality of the HRS data along all dimensions of total survey error. |
Differential Attrition |
| Wagner, J. and West, B.T. (2025) | Developing a Modern and Standardized Data Quality Profile for SRC Surveys | SRC Seminar Series presentation, September 30, 2025, Ann Arbor, MI |
This presentation outlines the core elements of a proposed data quality profile, assessing the overall quality of a given survey based on multiple quantitative and qualitative components of the total survey error framework. The presentation considers past efforts to develop these types of profiles and provides suggested guidance for development in the future. Please reach out to Brady West for a copy. |
Differential Attrition |
| Schroeder, H., Zhao, C., Martinez, S., and West, B.T. (2026) | SRC Data Quality Profile: HRS 2022 Panel | Technical Report of the Health and Retirement Study (hrs.isr.umich.edu). |
This report provides a holistic evaluation of the quality of the data collected in the 2022 wave of the Health and Retirement Study, applying the total survey error perspective for assessing data quality outlined in Wagner and West (2025). Overall, the report suggests high quality of the HRS data along all dimensions of total survey error. |
Differential Attrition |
| Zhang, X., Wagner, J., Elliott, M.R., West, B.T. and Coffey, S. (2025) | A Multivariate Stopping Rule for Survey Data Collection: Empirical Evaluation from a Panel Study. | Journal of Official Statistics: 0282423X241301479. |
Surveys are experiencing declining response rates. With more and more effort expended to combat these declining response rates, the cost of large-scale surveys has continued to rise. Stopping rules are one of the interventions used to improve the efficiency of data collection. However, most previously proposed stopping rules have focused only on the quality of a single estimate or assumed sufficient funds for nonresponse follow-up. In multipurpose surveys, there may be data quality objectives that must be met for multiple estimates with a constraint on costs. In this paper, we introduce a multivariate stopping rule that aims to maximize the tradeoff between the cost of data collection and the quality of multiple estimates. The multivariate stopping rule uses predicted costs and mean squared errors of different estimates to evaluate alternative sets of cases for stopping. |
Differential Attrition |
| Yan, T. (2025) | The Combined Effects of Prior-Wave Item Nonresponse and Perceived Burden on Subsequent-Wave Nonresponse in a Longitudinal Survey. | Paper presented at the 2025 Joint Statistical Meetings, Nashville, TN, August 5, 2025. |
The study demonstrates that higher perceived burden of panel survey participants, especially in earlier waves, is a much stronger predictor of attrition in subsequent waves than item nonresponse, providing support for existing theories of respondent burden in the longitudinal survey context. |
Survey Burden |
| West, B.T., Si, Y., Hu, Y., McCabe, S.E., and Veliz, P. (2024) | The Role of Weighting Adjustment for Attrition in Longitudinal Trajectory Modeling: A Simulation Study. | Communications in Statistics: Simulation and Computation. DOI: https://doi.org/10.1080/03610918.2024.2362923. |
This simulation study suggests that analysts of longitudinal survey data should regularly use all available observations from participants in longitudinal surveys when fitting trajectory models to longitudinal survey data (and not only observations from those participants with complete data across the waves). In addition, careful specification of the trajectory model is one of the most important factors. Further, the use of time-varying weights for the available observations and accounting for within-person correlations can be important when conducting the analyses. |
Imputation / Weighting |
| Watson, N., and Cernat, A. (2022, January 11) | Simulating the Consequences of Adaptive Survey Design in Two Household Panel Studies. | Retrieved December 20, 2022, from https://academic.oup.com/jssam/advance-article-abstract/doi/10.1093/jssam/smab050/6503715?redirectedFrom=fulltext. |
Adaptive survey design techniques have typically been used in effort to help improve response rates and lower overall survey costs. Much of the existing research surrounding these techniques have focused largely on cross-sectional surveys, whereas this study seeks to explore the potential efficacy of this approach within longitudinal surveys. Implementing this approach appeared to reduce the number of follow-up calls to participants, but reduced the overall response rate across time. |
Differential Attrition |
| Wagner, J., Zhang, X., Elliott, M.R. , West, B.T. and Coffey, S.M. (2023) | An Experimental Evaluation of a Stopping Rule Aimed at Maximizing Cost-Quality Trade-Offs in Surveys. | Journal of the Royal Statistical Society Series A: Statistics in Society. 186(4): 788-810. PMC10746548. |
Surveys face difficult choices in managing cost-error trade-offs. Stopping rules for surveys have been proposed as a method for managing these trade-offs. A stopping rule will limit effort on a select subset of cases to reduce costs with minimal harm to quality. Previously proposed stopping rules have focused on quality with an implicit assumption that all cases have the same cost. This assumption is unlikely to be true, particularly when some cases will require more effort and, therefore, more costs than others. We propose a new rule that looks at both predicted costs and quality. This rule is tested experimentally against another rule that focuses on stopping cases that are expected to be difficult to recruit. The experiment was conducted on the 2020 data collection of the Health and Retirement Study (HRS). We test both Bayesian and non-Bayesian (maximum-likelihood or ML) versions of the rule. |
Differential Attrition |
| Schwarz, H., Revilla, M. and Struminskaya, B.(2022) | Do Previous Survey Experience and Participating Due to an Incentive affect Response Quality? Evidence from the CRONOS Panel. | J R Stat Soc Series A, 981– 1003. Available from: https://doi.org/10.1111/rssa.12857. |
As more surveys are conducted, respondents are likely to have previous survey experience, which can make the use of incentives important for continued participation. Quality of survey responses does not seem to be associated with previous web survey experience, or participating only to receive an incentive. |
Incentives |
| Sastry, N., and McGonagle, K.A.(2022) | Switching from Telephone to Web-first Mixed-mode Data Collection: Results From the Transition into Adulthood Supplement to the US Panel Study of Income Dynamics. | J R Stat Soc Series A, 933– 954. Available from: https://doi.org/10.1111/rssa.12840. |
In an effort to reduce costs and increase respondent cooperation, web interviews were conducted in the 2019 wave of the Transition into Adulthood Supplement (TAS) to the US Panel Study of Income Dynamics (PSID). Compared to using telephone-only interviews, it was found that there were higher response rates, interviews were conducted faster, and data quality was higher and with lower associated costs when using the web-first approach. |
Survey Burden |
| McGonagle, K.A., Sastry, N., and Freedman, V.A. (2022) | The Effects of a Targeted “Early Bird” Incentive Strategy on Response Rates, Fieldwork Effort, and Costs in a National Panel Study. | Journal of Survey Statistics & Methodology, 00, 1-22. |
Adaptive survey design is a powerful tool that can be used to combat declining response rates in household surveys. One way to implement adaptive survey design is to utilize incentives. The use of early-bird incentive (EBI) has been found to increase response rates as well as lower study costs in a sample of high-effort respondents from the 2019 wave of the US Panel Study of Income Dynamics. |
Incentives |
| Maslovskaya, O., Struminskaya, B. and Durrant, G. (2022) | The Future of Online Data Collection in Social Surveys: Challenges, Developments and Applications. | J R Stat Soc Series A, 185: 768-772. https://doi.org/10.1111/rssa.12895. |
Online data collection has become increasingly popular as widespread use of technology has increased, especially during the COVID-19 pandemic, as face-to-face data collection was not an option. Online data collection has some unique challenges, such as coverage issues, data quality, and measurement problems. |
Survey Burden |
| Lynn, P. (2022) | Non-response on Longitudinal Surveys [PowerPoint slides]. | Economic and Social Research Council, University of Essex. Peter Lynn.pdf. |
Nonresponse in longitudinal studies can be impacted by many factors, such as increased burden on respondents and a failure to locate participants due to longer study duration. Various survey adjustments can be implemented to help combat nonresponse in longitudinal studies, including utilizing survey weights and auxiliary data. |
Differential Attrition |
| Kraemer, F., Silber, H., Struminskaya, B., Sand, M., Bosnjak, M., Koßmann, J., and Weiß, B. (2023) | Panel Conditioning in a Probability-based Longitudinal Study: A Comparison of Respondents with Different Levels of Survey Experience. | Journal of Survey Statistics and Methodology. doi:10.1093/jssam/smad004. |
Panel studies can be prone to learning effects, which causes respondents to provide lower-quality responses on subsequent waves of a panel survey due and is followed by a decrease overall data quality as a result. In addition, learning effects and conditioning of respondents can increase study attrition rates. In an effort to better understand the mechanisms impacting potential panel conditioning effects, the effects of conditioning frequency were explored using data from the German GESIS Panel study. Panel conditioning was found to have both negative and positive effects on response quality, suggesting that learning effects have a nuanced impact on response quality and attrition and therefore must be closely examined. |
Differential Attrition |
| Kocar, S., and Biddle, N. (2022) | The Power of Online Panel Paradata to Predict Unit Nonresponse and Voluntary Attrition in a Longitudinal Design. | Quality & Quantity, 57, 1055-1078. doi:10.1007/s11135-022-01385-x. |
There can be many factors that impact survey participation rates, as well as attrition and overall response rates. Using paradata combined with socio-demographic and socio-psychological participant characteristics collected from the Life In Australia study data, it was found that individual response rate was positively associated with higher levels of educational attainment as a socio-demographic predictor of response. |
Differential Attrition |
| Goodman, A., Brown, M., Silverwood, R.J., Sakshaug, J.W., Calderwood, L., Williams, J., et al. (2022) | The Impact of Using the Web in a Mixed-mode Follow up of a Longitudinal Birth Cohort Study: Evidence From the National Child Development Study. | J R Stat Soc Series A, 822– 850. Available from: https://doi.org/10.1111/rssa.12786. |
A mixed-mode design utilizing both telephone and web participation was investigated in the National Child Development Study in 2013. Compared to using telephone-only data collection methods, the offer of a web survey increased overall participation rates by 5.0 percentage points. |
Differential Attrition |
| Duvoisin, A., Refle J.-E., Burton-Jeangros, C., Consoli, L., Fakhoury, J., and Jackson, Y. (2023) | Recruitment and Attrition for Panel Surveys of Hard-to-reach Populations: Some Lessons from a Longitudinal Study on Undocumented Migrants. | Sage Journals, Volume 36, Issue 4, November 2023. https://doi.org/10.1177/1525822X231210415. |
Conducting research among hard-to-reach populations is a difficult endeavor because some of their characteristics are known to be associated with survey nonresponse and panel attrition. In the case of the Parchemins study, which followed undocumented migrants over their process of regularization and during the first years of regularized life in Geneva, we underscore the difficulties in recruiting and keeping respondents who come from such a hard-to-reach population. Factors hindering their participation include the fear of being denounced as undocumented, missing time due to high workload, health issues, or language problems. Using unique data from the recruitment and the follow-up processes, we demonstrate that investing high resources and time is particularly beneficial to reach such a population and to reduce attrition over successive data collection waves. In addition, we present the strategies adopted to draw a convenient sample from our targeted population, which mainly relies on generating trust. |
Differential Attrition |
| September 11, 2023 | CNSTAT Report: Examining the Effect of Interviewers on Longitudinal Survey Response Rates and Approaches to Improve the Hiring and Retention of High-Quality Interviewers | Importance of Measuring and Accounting for Interviewer EffectsInterviewers can vary in demographic attributes (e.g., age, gender), level of perceived experience and professionalism, approach to recruitment and survey administration (e.g., adherence to verbatim question wording), and behavior (e.g., reactions to reluctant respondents). Variability among interviewers can affect response rates, answers to survey questions, and data quality. In addition, interviewer effects can vary among groups of respondents (e.g., larger effects among older respondents compared to younger respondents). Thus, measuring, modeling, and accounting for interviewer effects is a crucial component of large-scale survey-driven studies. Rising interviewer turnover and difficulties recruiting new interviewers are significantly increasing study costs and threatening the face-to-face interview paradigm used by many large-scale health and aging studies. Multiple organizations and federal agencies have responded with new strategies to improve hiring and retention of interviewers, including shortened hiring processes (i.e., reduced time between initial application and starting work), increased pay, bonuses and incentives, and restructuring of interviewer positions to regular, full-time positions with benefits. The effectiveness of these strategies has been mixed, in part because interviewers vary in their needs (e.g., some prefer full-time employment while others prefer short-term, flexible employment). Moreover, the impact of these strategies likely varies based on labor markets. |
Interviewer Labor Models |
|
| Christmann, P., Gummer, T., Häring, A., Kunz, T., Oehrlein, A.S., Ruland, M., Schmid, L. (2024) | Concurrent, Web-First, or Web-Only? How Different Mode Sequences Perform in Recruiting Participants for a Self-Administered Mixed-mode Panel Study. | Journal of Survey Statistics and Methodology, 2024, smae008, https://doi.org/10.1093/jssam/smae008. |
This experimental study of recruitment strategies in the mixed mode context of a German panel study suggests that web-first invitations including a paper questionnaire and web-only recruitment approaches obtain similar response rates and data quality at reduced cost, relative to using paper as a nonresponse follow-up reminder. |
Survey Burden |
| Charman, C., Mesplie-Cowan, S., Collins, D. (March 2024) | The Post-pandemic Role of Face-to-face Fieldworkers. (PDF) | Interviewer Labor Models |
||
| Cabrera-Álvarez, P. and Lynn, P. (2024) | Text Messages to Facilitate the Transition to Web-First Sequential Mixed-Mode Designs in Longitudinal Surveys. | Journal of Survey Statistics and Methodology, 2024, smae003, https://doi.org/10.1093/jssam/smae008. |
This randomized experiment from the Understanding Society panel showed that there were not significant positive effects on response rates of using text messages to encourage web-first participation in a panel survey transitioning from a single mode approach (CAPI) to a web-first mixed-mode approach (web then CATI). The approach did increase web participation among younger members. |
Survey Burden |
| Boye, K., Mood, C., Tahiln, M. (2022) | Impact of Incentives and Response Mode on Non-response Rates and Non-response Bias in a Swedish General Population Survey [PowerPoint slides]. | Sweedish Institute for Social Research, Stockholm University Sigtuna 26-27 Sept Boye Mood Tahlin_participants.pdf. |
Utilizing more than one response mode and providing respondents with an economic incentive to participate in a survey are shown to help improve response rates while not increasing non-response bias. |
Incentives |
| Biemer, P.P., Harris, K.M., Burke, B.J., Liao, D., and Halpern, C.T. (2022) | Transitioning a Panel Survey From In-person to Predominantly Web Data Collection: Results and Lessons Learned. | Journal of the Royal Statistical Society: Series A (Statistics in Society), 185, 798– 821. https://doi.org/10.1111/rssa.12750. |
In recent years, costs associated with in-person interviews have increased, while the data quality advantages typically associated with this data collection method have lessened. Therefore, many longitudinal surveys have begun to transition from in-person to web data collection, which has prompted the proposal of a multi-sample, multi-phase responsive design in an effort to minimize data quality risks associated with the transition from in-person to web-based data collection. |
Survey Burden |
| Bergmann, M., Bethmann, A., Hunsicker, C., Schumacher, A., Connolly, F.F., Malmberg, G., and Olofsson, J. (2025) | The SHARE Respondent Driven Contact Experiment: Testing Prepaid Incentives and Contact Strategies | Paper presented at the 2025 conference of the European Survey Research Association, Utrecht, Netherlands, July 17, 2025. |
In the SHARE Wave 10 refreshment sample (target number of interviewers: 2000), a randomized 2×2 experiment testing the effectiveness of a 5 euro prepaid incentive and the inclusion of a contact card for the interviewer found that the prepaid incentive was quite effective at increasing response rates (about 6.9 percentage points), while the inclusion of the interviewer-specific contact card was not. The contact card also had no influence on the effect of the incentive. |
Incentives |