Development of a Questionnaire to Evaluate Out-of-Home Participation for People With Dementia

Older adults with dementia and their significant others value participation in activities outside the home, such as grocery shopping, pharmacy visits, and dog walking (Egan et al., 2006; Teitelman et al., 2010). Participation in outside activities offers numerous benefits, including maintaining cognitive functioning and autonomy, keeping in touch with others, and exercising in nature (Rao et al., 2014; Sugiyama & Ward Thompson, 2007; Winchester et al., 2013). It also involves an endless variety of places in combination with activities in which older adults want and need to engage.

Being immersed in activity in places—being in place—is an ongoing life process that generates important meaning (Rowles, 2008). It is closely related to the process of place integration—the continual adjustment of person–place relationships to enhance well-being (Cutchin, 2004)—and is consistent with the transactional perspective on occupation: Participation in occupation is seen as fundamental to the functional coordination of person and place in response to problematic situations (Cutchin & Dickie, 2013; Dickie et al., 2006).

Person–place relationships may change in relation to the development of dementia as the relationships that were once central to one’s life are disrupted by loss of familiarity, loss of ability to navigate and access places, or fear of embarrassment. These changes can have a cascading effect on participation and amplify risk of further disengagement. Thus, having regular access to outside places and activities is critical for maintaining familiarity for people with dementia (Brorsson et al., 2011). The interdependency of places and activities can facilitate or hinder participation, for example, through distances and layout of the physical environment, availability of transportation and support, meaning of activities, risk perception, and familiarity.

Maintaining a level of out-of-home participation can be a complex endeavor for people with dementia, who face challenges such as getting lost in previously familiar environments, needing help and support to orient themselves, and experiencing a “shrinking world” (Duggan et al., 2008). Familiarity is often underscored as a key to maintained out-of-home participation, understood as being experienced by an individual embedded in the use of the environment through a repeated activity (Brorsson et al., 2013). People with dementia may attempt to anchor themselves with activities in familiar environments and places that continue to give meaning to life and aid in maintaining identity (Van Steenwinkel et al., 2014). The net loss of participation in places and activities for people and the associated losses of meaning and well-being are as yet not well explored. One reason may be the lack of a systematic approach to collecting data on these issues, taking into consideration the various places and activities in which older people may engage.

Assessments addressing participation of people with dementia tend to focus on activities of daily living within the home, are often linked to the cognitive level, and typically use caregiver or professional judgments as responses; examples are the Kitchen Tasks Assessment (Baum & Edwards, 1993), the Disability Assessment for Dementia (DAD; Gélinas et al., 1999), and the more recent Functional Capacity Card Sort (Piersol et al., 2016). However, although the DAD was designed specifically for community-dwelling people, no item directly addresses out-of-home activities.

Recent research has developed and evaluated assessments for screening risks outside the home (Bantry White & Montgomery, 2015) and for designing dementia-friendly environments (Fleming et al., 2016), but these tools do not provide specific information about places where older adults with dementia go to perform out-of-home activities. Moreover, to our knowledge, no assessment tool yet developed aims to capture detailed information on places and activities in combination, specifically identifying participation restrictions and pointing out barriers and facilitators in different contexts. This article describes a systematic cross-cultural process of developing a questionnaire, Participation in Activities and Places Outside Home (ACT–OUT), for older adults with cognitive impairment and aligning ACT–OUT to people living with mild- to moderate-stage dementia so they can understand and answer the questions as intended.

ACT–OUT was developed using a multiphase, cross-cultural approach in combination with cognitive interviews performed in a Swiss context. Current literature on questionnaire and survey development underscores the importance of initially defining questions and items carefully, having distinct test phases, and obtaining continuous committee reviews throughout the process (Gillham, 2007). In this study, three phases were implemented (Figure 1): (1) the construction phase, including preparation and design of the questionnaire; (2) the revision phase, including cognitive interviews with older adults; and (3) the alignment phase, in which the questionnaire was further modified after use with people with mild- to moderate-stage dementia. Ethical authorization (Protocol 469/13) was obtained from the Commission Cantonale d’Éthique de la Recherche sur l’Être Humain in Switzerland, where empirical data were gathered.¹

The development of ACT–OUT was based on a collaboration among research teams in Sweden, Switzerland, and the United States using an overarching process and structure across all phases of instrument development (construction, revision, and alignment). We used a dual-panel method developed by Hagell and McKenna (2003) to systematically communicate ideas, suggestions, and modifications among the teams, including ongoing discussions, translations, and meetings throughout the study; some communications were audio recorded, and all decisions were documented through minutes. Various viewpoints, literature reviews, and different clinical expertise offered multiple views during the questionnaire’s construction, revision, and alignment phases (Gillham, 2007). Within this cross-cultural process, we used recommendations based on similar multilanguage research projects (Haak et al., 2013; van Nes et al., 2010). English was used as the common language among the teams, and changes to ACT–OUT were done systematically in parallel processes for all languages in the different phases of the development process to ensure cultural as well as language translation.

The first draft of ACT–OUT began as a list of activities older adults do outside their home, inspired by existing research and coming from the long tradition of focusing on functioning and activity performance in occupational therapy (Versions 0.1 and 0.2; see Figure 1). Because people tend to do more than one activity in the same place, we consolidated the assessment approach by moving from activities that people might do at particular places to focusing on the places themselves in the construction of the items, allowing for places to be multiactivity based. This change of the primary focus of the tool, directed by the need to combine places and activities in our operationalization of out-of-home participation, created two interlinked parts in ACT–OUT (Version 0.3; see Figure 1). Part 1 includes a list of places and items, aiming to determine the specific and unique constellation of places where the person goes or does not go. Part 2 poses questions related to a selected number of places from Part 1, such as activities performed, transportation means, accompanying persons, risk perception, and familiarity (see Figure 2). This construction phase involved all three cross-cultural teams and revolved around item and question generation with a continuous check with the key theoretical constructs underlying out-of-home participation as a coordination of place and activity (Cutchin & Dickie, 2013; Rattray & Jones, 2007).

ACT–OUT Version 0.3 was reviewed by experts to ensure evidence of test content validity of the items for people with dementia (Rattray & Jones, 2007). The expert group included two senior researchers in geography, two representatives of the Swiss Alzheimer’s Association, one physician with extended clinical and research experience in dementia, one professor of social work, and three occupational therapists.

In Phase B, ACT–OUT Versions 0.4 to 0.7 (see Figure 1) were revised in successive cognitive interview rounds to increase content validity and limit misunderstandings and ambiguity of questions.

The participants (n = 26) were recruited in three rounds using purposive sampling through senior associations in two cities (Lausanne and Fribourg) and a nearby rural area in Switzerland. The criteria for inclusion were age over 65; living in own home in urban, suburban, or rural settings in the French-speaking region of Switzerland; and having no known cognitive impairment. We excluded people who used a wheelchair for outside mobility because this was expected to show a different type of participation restrictions. Sociodemographic data collected in this phase included data on additional issues potentially influencing participants’ level of participation outside the home (e.g., health problems, driving; Wettstein et al., 2013). Participants lived in various urban and suburban areas (n = 13), in villages or small towns (n = 11), and in the mountains (n = 2; Table 1).

The initial criteria for recruitment were applied in Round 1. In Round 2, more older participants were selected to validate the questionnaire for an older and potentially less independent population. In Round 3, participants were selected from more rural settings, including remote areas (e.g., the mountains), where home support was less available.

Three interviewers (including the first author) conducted one-on-one cognitive interviews with all participants in Phase B to address their general understanding of the concepts used and the specific wording of ACT–OUT questions, supporting evidence of test content validity (Willis, 2005), and to increase trustworthiness through pooling and confronting interviewers’ experiences (Ridolfo & Schoua-Glusberg, 2011). Concurrent think-aloud suggestions and probing techniques were used in the interviews, followed by debriefing questions and observations of respondents’ reactions (Hak et al., 2008). Each round was followed by constant comparison analysis of the responses (Ridolfo & Schoua-Glusberg, 2011), resulting in further adaptation of the questionnaire involving all three cross-cultural teams. Although there was a progression in the focus from Round 1 to Round 3, the aims of these rounds were complementary and did not result in disparate findings.

Round 1 focused on confirming a general understanding of the questionnaire (e.g., “What do you think this questionnaire is about?”), validating the list of places by asking whether a place the participant visits is missing, organizing items in clusters (e.g., “How do the clusters refer to the places you visit?”), and validating the format (e.g., “What do you think about the length of the questionnaire and the time it takes to answer all questions?”). Attention also was paid to response categories and scales.

Round 2 focused on question phrasing, such as checking participants’ interpretation of specific words by asking the same question in different ways, and on items in Part 1 to identify places as different from one another to avoid overlap. Participants were asked about meanings they attached to places they visited and the types of activities they performed there. Round 3 focused on sequencing of items and questions because the position of questions may influence the way informants respond (Rattray & Jones, 2007) and on phrasing of questions in Part 2, especially the question about risk perception.

In Phase C, ACT–OUT Version 0.8, then an adapted Version 0.9 (see Figure 1) were aligned to fit people with dementia. Five participants living with dementia were recruited through the Swiss Alzheimer’s Association. Using the Functional Assessment Staging Test (Auer & Reisberg, 1997), the participants were at functional Levels 4 (n = 4) and 5 (n = 1). They lived alone (n = 2) or in couples (n = 3) and in suburban (n = 2) or rural areas (n = 3). Two interviewers (including the first author) conducted face-to-face interviews to further align ACT–OUT questions to the target respondents with dementia (Presser et al., 2004) to ensure that participants understood and interpreted the questions as intended and that they demonstrated safety and confidence when responding.

The final version, ACT–OUT Version 1.0, has three parts and is available in English, French, and Swedish. Part 1 identifies places people go and places where change has happened compared with the past or in anticipation of the future. It is presented as a list of places in four clusters: (1) seven items in the cluster “consumer, administrative, and self-care places” (e.g., grocery store, hairdresser, bank, post office); (2) five items for “places for medical care” (e.g., hospital, dentist, day care); (3) six items for “social, cultural, and spiritual places” (e.g., restaurant or café, cemetery, entertainment or cultural places); and (4) seven items for “places for recreational and physical activities” (e.g., garden; forest, mountain, lake, or seaside; sports facility).

The more detailed questions of Part 2 are asked in relation to two items per cluster chosen by the participant. Part 3 consists of four questions about people’ views of themselves regarding (1) life satisfaction; (2) participation; (3) attitude toward risk taking; and (4) level of concern about getting lost, falling, being stressed, and being embarrassed (see Figure 2 for examples of items).

ACT–OUT is administered in a face-to-face interview with the person with dementia. The interviewer may rephrase the questions if needed to facilitate understanding. If a significant other is present, he or she can support the person with dementia.

In Round 1, all participants validated the content of the questionnaire and reported that it covered their life outside the home. The version of ACT–OUT that was used (0.4; see Figure 1) had five clusters (commercial, health care, administration, social–spiritual–cultural, and recreational places). Round 1 interviews indicated that this initial clustering of items had to be revised because the meanings of some places were mixed for the participants. One participant said, “I don’t understand why the hairdresser is in the health care category. I buy a service when I go there. It’s not like when I go to my doctor.” The commercial, health care, and administration clusters were regrouped, creating a new commercial and administration cluster, on the basis of how participants understood these places. The participants also reported that the questionnaire was indeed relatively long, and we therefore decided to ask questions about only two places per cluster in Part 2, adding up to eight places reported in more detail.

In Round 2, the items in Part 1 were reorganized on the basis of whether participants identified them as being different or similar places. For example, participants identified “bank” and “post office” as representing similar places because they usually got money at either place, so they were combined into one item. We therefore reorganized the sequencing of items in Part 1 on the basis of what places participants perceived were related or not.

In Part 2, the response alternatives were rephrased. For example, regarding familiarity of places, only one of three alternatives was used (“very familiar”), indicating that the response scale needed specificity. Thus, we created the response alternatives “completely,” “very,” and “somewhat” familiar. The question about risk perception was particularly difficult to formulate. In Round 2 we tried it out in different ways, deciding that Round 3 would focus more particularly on that question.

On the basis of participants’ responses in Round 3, we adapted the sequencing in the interview to have the Part 2 questions for two items per cluster asked directly after each cluster. We decided to check this with participants with dementia.

At the end of Round 3, the question about risk perception was settled, with the outcome being a double-barreled question formulation: “Picture yourself in that place. What do you have to be careful about or pay extra attention to?” An open-ended response alternative was added to allow the exploration of more examples because each individual may have unique perceptions of risks and challenges; interviewers could then rephrase that question freely.

The participants in Round 3 responded to questions about risk perceptions related to specific items by referring to general perceptions of who they were. On the basis of these responses, we added a few general questions at the end of the questionnaire, creating Part 3. Participants reported that this sequencing allowed them to be well immersed in the subject and to reflect on their own perceptions regarding life outside the home.

Participants with dementia overall reported that Part 1 of ACT–OUT was easy to answer; stories and explanations about various places emerged spontaneously. Having Part 2 questions after each cluster of places (Part 1) also helped participants keep track of places and activities discussed. A higher degree of flexibility in how to ask questions about risk perception, using either the idea of picturing oneself or describing what one needs to be attentive to, allowed participants with dementia to tell stories about places they visited and problems they encountered there.

An additional important change in Part 1 of ACT–OUT was that an item related to travel was reintroduced because participants with dementia identified traveling as important, conferring a feeling of freedom and competence. Traveling had earlier been removed as being inconsistent with ACT–OUT items defined as places. We therefore included a stationary place related to traveling, such as a train station or airport, where one actually boards a vehicle.

We have described a systematic process of developing a questionnaire about participation in activities and places outside the home and testing it with people living with dementia. The overarching development process and structure used in this study highlight the complexity of assessing out-of-home participation in this population by taking into account places and activities in combination, acknowledging that they cannot be distinctly separated (Cutchin & Dickie, 2013). The targeted construct of ACT–OUT is still difficult to clearly define and operationalize because of the many linked dimensions in the concept of participation: activities, places, and influences of cross-cultural diversity.

The cross-cultural collaboration both challenged and aided the research team by reminding us to identify types of places that allow cultural diversity and variation in activities undertaken there, rather than limiting items to culture-specific places. Moreover, it also gave opportunities to discuss the phrasing of questions, as exemplified in the question about risk perceptions. The cross-cultural team contributed to the phrasing of questions and response alternatives on the basis of extensive experience in interviewing older adults with and without dementia. However, because all empirical data are from Swiss participants, the ecological validity of ACT–OUT when used in other contexts remains to be investigated.

The results show that the process of developing a tool such as ACT–OUT is cyclical rather than linear. There were back-and-forth amendments in the process, as well as discussions of details and modifications by an interdisciplinary and multicultural research team, requiring time and flexibility. Recording of decisions was crucial for keeping track of amendments in the versions of ACT–OUT. Because cognitive interviews can also introduce errors (either by identifying spurious problems or not identifying real problems; Conrad & Blair, 2009), the use of three interviewers helped alleviate this problem by offering various viewpoints. Having successive rounds of cognitive interviews also allowed for trying out changes in the phrasing of questions and provided empirical arguments for keeping earlier modifications (or not). The use of various techniques in the cognitive interviews (e.g., think-aloud and probing questions) helped obtain rich verbal reports from participants. The dual-panel method supported the cooperation of the research team. These methods in combination are essential, and all contribute to the development of an assessment tool (Drennan, 2003), stressing the need to have a research team able to manage various methodological approaches and viewpoints on which to base decisions about change.

The development process of ACT–OUT also indicated the importance that older adults give to places they go to in relation to activities they perform there. The reintroduction of an item related to traveling in the final phase was based on its recurrent appearance in the participants’ discourse, showing how important traveling might be to older adults’ participation (Bai et al., 2016). This item is also indicative of the challenge of choosing places as items for ACT–OUT in relation to activities being performed outside the home. Whereas many activities are easily recognized by place, such as grocery shopping or getting a haircut, other activities can be performed in diverse places, such as meeting friends or reading a newspaper.

The use of places as items for investigating out-of-home participation postulates that places are made not only of delineated spaces but also of specific activities people perform there, which together hold meaning. In fact, human experiences are complex; people are able to do multiple activities with multiple meanings in multilevel places at the same time (Cutchin & Dickie, 2013). Models of aging in place and of transforming spaces into places show how the environment and the occupation are inseparable and together create meaning (Shank & Cutchin, 2011).

Furthermore, investigating out-of-home participation involves understanding the intertwining of perceived risks and loss of familiarity as dementia progresses. In Switzerland, questioning the risks a person perceives when going outdoors has underpinned a resistance to being labeled as a person “at risk” (Kuhne et al., 2011), which could lead to undesired consequences such as being subjected to various professional preventive interventions or threats of institutionalization. To avoid responses influenced by such misunderstandings, we decided to try a double-barreled question with an open reply: “What do you have to be careful about or pay extra attention to?” Although double questions usually are to be avoided, this one seemed to facilitate more elaborated and narrative responses in Round 3, but the quality of the responses to this question needs to be explored in further empirical studies with ACT–OUT.

One limitation in developing ACT–OUT as a potential cross-cultural questionnaire is that all empirical data used in this study were only from Switzerland. Data collected in other languages and other contexts are needed to develop and evaluate ACT–OUT further as a valid cross-cultural questionnaire. Still, the research team represented various backgrounds and philosophical paradigms, offering a cross-cultural nature to the consistency of the recursive process and consensus-reaching discussions in a rigorous instrument development procedure.

Because this study can be viewed as generating evidence of test content validity in relation to the target population, it is important to highlight that the psychometric properties of ACT–OUT are still unknown, and empirical data from different contexts and samples are needed to explore and evaluate aspects of validity (American Educational Research Association et al., 2014). A clear limitation is also the small number of participants with dementia involved in Phase C. They can be seen as individual cases, and they did not provide enough rich and diverse data to clearly reach saturation or identify the impact of cognitive impairments of respondents in relation to the validity of ACT–OUT responses. Therefore, a validation study of ACT–OUT will be performed after this study with a more diverse sample of people with dementia from Switzerland, Sweden, and the United States. Determination of a scoring system for out-of-home participation and evaluation of clinical applicability and psychometric properties of ACT–OUT both within and across contexts will then be explored and evaluated more in-depth. Because ACT–OUT is based on a structure in which participation in places will vary across individuals, a Rasch model approach will potentially be needed for psychometric testing, because it allows people to score only on items that are relevant to them and still allows for comparisons across people (Bond & Fox, 2007).

Eventually the use of ACT–OUT will allow health care professionals, including occupational therapy practitioners, access to the complexity of out-of-home participation for people living with or without dementia. Professionals will also be able to monitor changes in relation to time and access to more targeted interventions (e.g., environmental adaptations) to support participation and a more dementia-friendly society.

The development of ACT–OUT is a first step toward a more systematic evaluation of out-of-home participation as perceived by older adults living with dementia, combining places and activities, for research and clinical purposes. Occupational therapy practitioners and researchers may use this tool to

• Assess clients’ views on out-of-home participation to identify restrictions, facilitators, and obstacles;

• Monitor change in out-of-home participation as related to places and activities; and

• Identify target activities and places for intervention outside the home.

The information provided by ACT–OUT is information is needed to identify needs, plan intervention, and provide support for people with dementia.

This study was funded by the Réseau d’Études aux Confins de la Santé et du Social (SAGE-X 34373), a peer-reviewed fund of the University of Applied Sciences and Arts of Western Switzerland. Special thanks go to Delphine Dumoulin and Raphaël Voélin for contributing to the data collection and Eva Hammar for helping with language issues in Swedish, French, and English. We also thank the CACTUS research group at Karolinska Institutet for critical discussion on ACT–OUT.