Innovative applications of new technologies for longitudinal measurement

Use of live video interviewing as a method to conduct surveys became more common during the Covid-19 pandemic and in the UK, this mode of data collection was implemented in major social surveys for the first time. This paper investigates the use of LVI, focussing on opportunities and barriers, and collating evidence and experiences from seven major social surveys in the UK, with an emphasis on longitudinal surveys. The specific aims are the investigation of: uptake and response rates to LVI, the characteristics of those that responded via LVI, and the feasibility of collecting complex elements via LVI, such as consent, cognitive assessments and sensitive questions. One of the main findings is that LVI in the UK surveys analysed was used in different ways: either as the only/primary survey mode when in-person/face-to-face data collection was not possible, or as a complementary mode in mixed-mode designs. The results suggest that, if LVI were the only or primary data collection mode, response rates would be notably lower than in alternative modes – for both cross-sectional and longitudinal studies. There is also some evidence that lower response rates in LVI could potentially lead to an increase in representation bias. On the other hand, there are encouraging f indings, including that once respondents agree to participate via LVI, this mode proves to be a suitable approach for collecting complex elements. This is a key finding since previous research has identified limitations of other remote methods for collecting this kind of data, which is an important component of many studies, especially longitudinal studies. Overall, the evidence from this study suggests that LVI, under certain conditions, can be a suitable complementary data collection mode in a mixed-mode survey design, offering potentially more cost-efficient fieldwork. We identify particular feasibility advantages for longitudinal surveys. An increase in LVI uptake in both longitudinal and cross1 sectional surveys may be expected over time as online working becomes increasingly common and some barriers to LVI can be addressed. Further LVI development of fieldwork procedures are required.

Live video interviewing emerged as a method for collectng survey data during the COVID-19 pandemic, having rarely been used for survey data collecton prior to this. There is now a need to assess experiences and outcomes from studies that utlised video interviewing, partly with a view to informing the future feasibility of the method in different contexts. This paper reports on the experience of the European Social Survey (ESS) with video interviewing, having used this approach as a complementary method to in-person interviewing at its 10th round (2020-2022). The ESS can provide a unique perspectve, being the first cross-natonal survey to use video interviews. In total, 17 countries offered video interviewing alongside in-person interviewing at ESS Round 10. In this paper, we present a range of results based on ESS Round 10 in two main categories. We first look at the effectveness of the implementaton of video interviewing and then compare quality between video interviews and in-person interviews across various indicators, including interviewer effects. The results show that the prevalence of video interviews varied widely between countries, likely relatng to natonal contextual factors. However, in countries where a large share of video interviews was carried out, we found that the interview experience was rated positvely, and quality indicators were closely comparable with in-person interviews. These results suggest that future use of video interviewing may be more feasible in some countries that others, but in certain contexts it has the potental to offer an effectve complementary opton to in-person interviewing.

This presentation discusses important design considerations when asking panel survey respondents to participate in additional data collection using smartphone apps, and provides empirical evidence in support of best practices (along with directions for future work).

This paper examined data donation from smartphone apps as a way to collect passive data. The focus of this presentation was the willingness to donate data, the drivers behind data donation and the biases in the donated data.

This study involved a subsample of respondents from 10 countries wearing an accelerometer on their upper thigh for 8 days. While some evidence supported the “healthy volunteer” hypothesis, indicating overrepresentation of healthier individuals in the fully-adherent group, biases and patterns were not necessarily consistent. Addressing the challenge of obtaining consent and minimizing delays between consent and task onset were identified as important factors in improving participation rates.

Understanding Society, the UK Household Longitudinal Study has a programme of research and development that underpins innovations in data collection methods. One of our current focuses is on using mobile applications to collect additional data that supplement data collected in annual interviews. To date, we have used mobile apps to collect data on consumer expenditure, well-being, anthropometrics and cognition. In this paper, we review the potential barriers to data collection using mobile apps and experimental evidence collected with the Understanding Society Innovation Panel, on what can be done to reduce these barriers.

Smartphone apps are increasingly being used for a variety of studies involving intensive measurement. Many of these studies are conducted among small groups of volunteers where selection bias is of less concern and ways to increase participation have been largely ignored. As these app-based studies are being included in large population-based surveys to permit broader inference and extend measurement across a broad range of topics, concerns over participation (or otherwise) in such studies and the implications for selection bias are becoming more salient. Given this, we conducted an experiment to test the effect of various features of an app study protocol on uptake or participation. Respondents in a large nationally-representative longitudinal survey in Great Britain were asked to download an app and use it every evening for 14 days to answer questions about their experiences and wellbeing that day. We experimentally varied three features of the study protocols: Inviting participants to the app study earlier in the annual interview increased participation rates, while a shorter daily questionnaire and bonus incentives for completing all 14 days had no significant effect. We also examine the effects of study protocols on non-participation bias, as well as the problems respondents had with installing the app and reasons for not participating.

In this paper, we compare the quality of expenditure data collected from a mobile receipt scanning app in the Understanding Society Innovation Panel with benchmark data from the Living Costs and Food Survey, the national budget survey in the United Kingdom. The results suggest that expenditure reported with scanned receipts plus direct entry aligns closely with the benchmark whereas app data from scanned receipts only clearly underestimate expenditure. In addition, the app data align more closely with the benchmark for respondents who are older, male, have an above-median income, and live in rural areas. The implications of measurement differences vary: comparisons of estimated budget shares are closer to the benchmark for some categories than others.

Mobile apps are an attractive and versatile method of collecting data in the social and behavioral sciences. In samples of the general population, however, participation in app-based data collection is still rather low. In this article, we examine two potential ways of increasing participation and potentially reducing participation bias in app-based data collection: (1) inviting sample members to a mobile app study within an interview rather than by post and (2) offering a browser-based follow-up to the mobile app. We use experimental data from Spending Study 2, collected on the Understanding Society Innovation Panel and on the Lightspeed UK online access panel. Sample members were invited to download a spending diary app on their smartphone or use a browser-based online diary to report all their purchases for one month. The results suggest that inviting sample members to an app study within a face-to-face interview increases participation rates but does not bring in different types of participants. In contrast, the browser-based alternative can both increase participation rates and reduce biases in who participates if offered immediately once the app had been declined. We find that the success of using mobile apps for data collection hinges on the protocols used to implement the app.

Smartphones can provide valuable insight into human behavior patterns that cannot be measured through surveys. Although this data can be very useful and convenient to collect, there is some question around the quality of smartphone data, as errors can arise through geolocation measurement, or different individuals sharing a smartphone.

The study found that 29.2% of the participants experienced moderate psychological distress, with 12.1% reporting severe distress. Factors such as difficulty accessing medical care, worries about COVID-19, concerns about interactions with police during lockdown measures, and spending more time at home were associated with higher levels of distress. The findings highlight the urgent need to address the mental health consequences of the pandemic and consider mental health in policy responses, especially in low- and middle-income countries (LMICs).

The National Health and Aging Trends Study (NHATS) has developed a protocol to objectively measure visual and auditory function in Medicare beneficiaries aged 65 and older. The protocol includes vision tests for distance and near acuity, contrast sensitivity, and a hearing test using a tablet platform. The study found that objective measures were more effective in identifying visual and auditory impairments and had stronger associations with demographic factors. Overall, the study demonstrated that objective visual and auditory functioning can be successfully measured through an interviewer-administered home-based protocol.

Cognitive impairment is a major health issue impacting many older adults, making it imperative to have indicators of cognitive functioning. Web and phone surveys have shown to be useful in measuring cognitive functioning, allowing for the development of a cognitive impairment score.

COVID-19 has created an immediate need for frequently updated information surrounding the societal impacts of the pandemic. The Mannheim Corona Study (MCS) was designed to collect this data, based on the existing probability-based online panel infrastructure of the German Internet Panel (GIP).

The activPAL device is typically used to collect data on step count and walking, sit-to-stand transitions, and sedentary time in older adults. Older adults tend to be more sedentary than other age groups, making data collected from the activPAL device especially useful for current and future research on physical activity in older adults.

Smartphones make it possible to collect both self-reported data as well as passively measured behavior data simultaneously. Utilizing these data sources in conjunction with one another can be especially useful in measuring and contextualizing data, but can be difficult to implement due to concerns around privacy and individual’s willingness to share their smartphone data.

Smartphones can be extremely useful in collecting data about daily activities (sleep, physical activity, etc.), as they are typically in close proximity to their users. The feasibility of using smartphones to collect this sort of information depends largely on the ability to translate raw smartphone sensor data to activity outcomes, which can be influenced by both sociodemographic and smartphone-related characteristics.

The activPAL Technologies’ CREA algorithm can be used to track bedtime, wake time, and sleep time, but there are marked differences between the measured sleep and wake times using the activPAL and ActiGraph devices. The activPAL algorithm appeared to overestimated sleep time by measuring earlier bedtimes and later wake times.

Given the increase in utilization of smartphones to conduct web surveys, it has become increasingly more important to consider the challenges that older adults may face when taking mobile surveys. Using a within-subjects experimental design (the same participants tested each survey design), this study determined that a below-content design promotes higher data quality and participant satisfaction, and is therefore recommended for use in mobile surveys for older adults.

Linking trace data to existing panel survey data can improve the analysis potential of the data, but can be sometimes difficult to implement due to requiring additional consent and completion of tasks from survey participants. In order to address these concerns, a group of participants were invited to install a smartphone app to share additional data, but this request was found to decrease panel retention in the following wave.

Utilizing smartphone applications can be an extremely useful tool for administering surveys and collecting behavioral data. The likelihood of using a smartphone application was found to be lower for women and older adults, as well as for those with higher levels of educational attainment.

Offering participants in mobile app studies personalized feedback on the data they report seems an obvious thing to do: Participants might expect an app to provide feedback given their experiences with commercial apps, feedback might motivate more people to participate in the study, and participants might be more motivated to provide accurate data so that the feedback is more useful to them. However, personalized feedback might lead participants to change the behavior that is being measured with the app, is costly to implement, and also constrains other design decisions for the data collection. In this article, we report on an experimental study that tested the effects of providing personalized feedback in a 1-month mobile app–based spending study. Based on the app paradata and responses to a debrief survey, it seems that participants reacted positively to the feedback. The feedback did not have the potential negative effect of altering the spending participants reported in the app. However, the feedback also did not have the intended effect of increasing initial participation or ongoing adherence to the study protocol.

The COVID-19 pandemic had a significant impact on physical activity. Using the BetterPoints smartphone application, physical activity was measured throughout the COVID-19 lockdown, and found that participants who were most active prior to the pandemic demonstrated the most significant decreases in physical activity at the onset of the pandemic. Additionally, older adult participants were found to have a less significant decrease in physical activity at the beginning of the pandemic, but became more sedentary as the pandemic progressed.

The results of the study showed that the average step counts recorded by different devices were fairly close, ranging from 11,734 to 11,922 steps. The consumer monitors tested in the study generally provided accurate estimates of step counts, and the Charge2 and Apple2 devices performed reasonably well in heart rate estimation. However, all devices underestimated MVPA in real-life situations, which suggests that caution should be exercised when using these devices to track higher-intensity physical activities.

GPS technology can be used to track activity and location, and determine activity patterns between different populations. Those with lower educational attainment levels and socieoeconomic statuses were found to spend more time outside of their residential areas, likely due to lesser resources. Additionally, poverty rates in nonresidential areas tended to be lower.

Android cellphone devices can be used to track daily activities, as well as record sound that is detected near the device. Through this mechanism, in a sample of largely Black and Hispanic persons, it was found that participants were more likely to watch television while alone, and also reported increased feelings of loneliness while watching television.

This paper examines non-response in a mobile app study designed to collect expenditure data. We invited 2,383 members of the nationally representative Understanding Society Innovation Panel in Great Britain to download an app to record their spending on goods and services: participants were asked to scan receipts or report spending directly in the app every day for a month. We examine participation at different stages of the process. We further use data from the prior wave of the panel to examine the prevalence of potential barriers to participation, including access, ability and willingness to use different mobile technologies, and biases in the types of people who participate, considering socio-demographic characteristics, financial position and financial behaviours. While the participation rate was low, drop out was also low: over 80% of participants remained in the study for the full month. The main barriers to participation were access to, and frequency of use of mobile devices, willingness to download an app for a survey, and general cooperativeness with the survey. While there were strong biases in who participated in terms of socio-demographic characteristics (with women, younger, and more educated sample members being more likely to participate), and in terms of financial behaviours (with respondents who already use mobile devices to monitor their finances being more likely to participate), we found no biases in correlates of spending.

We asked members of the Understanding Society Innovation Panel about their willingness to participate in various data collection tasks on their mobile devices. We find that stated willingness varies considerably depending on the type of activity involved: respondents are less willing to participate in tasks that involve downloading and installing an app, or where data are collected passively. Stated willingness also varies between smartphones and tablets, and between types of respondents: respondents who report higher concerns about the security of data collected with mobile technologies and those who use their devices less intensively are less willing to participate in mobile data collection tasks.

Smartphone technology can be utilized to measure a variety of daily activities, including the sleep-wake cycle. The data collected by smartphones can be further enhanced and verified by participant sleep diaries and actigraphy (a small device that is worn like a watch and monitors sleep activity).

Smartphones an be very useful in survey research, but it is important the surveys are properly optimized to be completed on a smartphone. This can include selecting questions that are easier to complete on a smartphone, ensure that questions are fit to the width of smartphone screen to prevent horizontal scrolling, and choose simpler question types over more complex ones.

Smartphones can be very useful in administering web surveys, but special survey design considerations must be taken into account when creating web surveys for the older adult population. It is important to clearly label the survey and its navigation tools to ensure the highest level of usability within this population.

Lack of physical activity is related to reduced health outcomes, therefore making it extremely important to understand the mechanisms by which health outcomes are influenced by physical activity. Wearable devices such as the activPAL can be very useful in collecting data in field-based research settings on physical activity.

Bluetooth and Wi-Fi networks can be very useful in collecting information about social networks within a specific location, and can be utilized to make connections within aging populations residing within assisted living facilities. This social network data can also be connected to relevant health data.