Discriminant Validity of the Electronic Activity Card Sort (ACS3) Among Adults With Traumatic Brain Injury and Multiple Sclerosis Open Access

Participation, or involvement in life situations (World Health Organization, 2001), is an essential outcome in rehabilitation, and it has garnered more attention in recent years, including heightened health care reimbursement (Centers for Medicare & Medicaid Services, 2023; Goverover & Chiaravalloti, 2023). Participation is associated with higher quality of life (QOL), decreased depression, and better overall well-being (Santini et al., 2020). Changes in health status can disrupt established activity patterns, often leading to reduced participation in meaningful life activities, negatively affecting independence and life satisfaction (Marzo et al., 2023).

The Activity Card Sort (ACS; Baum & Edwards, 2008) measures participation across multiple domains of daily living by comparing current activity levels with preinjury or prediagnosis baselines. This patient-centered approach to measurement eliminates reliance on normative data to detect meaningful participation changes (Tyminski et al., 2020). A recent virtual adaptation, the Electronic Activity Card Sort (ACS3; Boone et al., 2022), features 100 activities across diverse participation domains. Although the ACS3 has been shown to have high validity with the original ACS among healthy control participants (Boone et al., 2022), its application across clinical populations remains unexplored.

Measuring participation across health conditions that entail distinct trajectories provides unique insights into a particular instrument’s sensitivity and validity. Traumatic brain injury (TBI) and multiple sclerosis (MS) represent two distinct neurological conditions for such a comparison (Warren et al., 2013). MS is a progressive neurodegeneration condition that leads to a gradual decline in functional abilities and participation over time (Goverover et al., 2020; Johansson et al., 2020). In contrast, TBI typically involves an acute onset followed by a recovery period and chronic stable participation outcomes (Erler et al., 2018; Goverover & Chiaravalloti, 2023). Studies using the ACS have shown significant participation reduction in both people with MS (Goverover et al., 2020) and those with TBI (Goverover et al., 2017) compared with healthy control participants, with those with MS showing more pronounced decline in high-demand leisure and social activities then those with TBI. However, direct comparisons have been limited by methodological differences.

In this exploratory cross-sectional study, we compared the performance of the ACS3 between TBI and MS groups and within each group. On the basis of previous research showing distinct patterns in these conditions, we hypothesized that the ACS3 will detect different participation profiles between MS and TBI. Understanding these distinctions will advance the utility of the ACS3 in capturing nuanced participation changes across diverse populations, informing both clinical practice and rehabilitation outcomes research.

Participants (n = 22; 11 with TBI, 11 with MS) were recruited through community flyers, support groups, social media, and word of mouth between May and November 2023. Participants were eligible if they were an adult (≥18 yr), community dwelling, fluent in English, had adequate vision to view study materials, had access to internet-connected devices, and could provide informed consent. Participants with MS (Thompson et al., 2018) had no recent (past-month) exacerbation and expressed concerns about memory. Persons with moderate to severe TBI who were at least 6 mo postinjury were matched by age to the MS cohort. Data were collected via Zoom, with participants completing a demographic questionnaire and the ACS3. The virtual format was selected to align with the ACS3 online delivery and trends toward telehealth (Wosik et al., 2020). The study was approved by the New York University Institutional Review Board.

The ACS3, a measure of life participation, was adopted from the paper form of the ACS (Baum & Edwards, 2008) and is intended to be used electronically. It includes 100 activities across four domains: instrumental activities of daily living (IADLs), social activities, leisure activities, and fitness or exercise activities (previously called “high-demand leisure activities”). Individuals view photographs of adults performing these activities and rate their own involvement on the basis of five categories: (1) “I have never done [this],” (2) “I continue to do [this] (since illness or injury onset),” (3) “I do [this] less since my injury/illness,” (4) “I have given up [doing this],” or (5) “I would like to start [doing this].” The retained activity score from before onset to current engagement is calculated by dividing the sum total of current activities by the sum total of previous activities. The ACS3 demonstrates concurrent validity with the ACS (r = .86) with respect to total scores, with no significant difference between the ACS and ACS3 in retained activity scores in community-dwelling adults (p = .39; Boone et al., 2022).

We used descriptive statistics to summarize ACS3 total and domain scores in each group. Independent t tests were used to compare the ACS3 total and domain scores between the TBI and MS groups. Within each group, paired t tests were used to compare preinjury or preillness and current activity scores. Associations between demographic variables (e.g., age, time since injury or illness onset, marital status, education, and gender) and ACS3 total and retained activity scores were analyzed for the whole sample. Pearson correlations were used for continuous demographic variables, and Spearman correlations were applied for categorical variables. The statistical significance was set at p < .05. All analyses were completed using R with the tidyverse, gtsummary, and ggplot2 packages (Sjoberg et al., 2021; Wickham, 2016; Wickham et al., 2019).

Demographic characteristics for the total sample and each group are presented in Table 1. The TBI group consisted of five individuals with moderate TBI and six with severe TBI. For the MS group, three individuals had relapsing–remitting MS, three had primary progressive MS, and five had secondary progressive courses of MS. The groups were matched for comparison, with no significant difference in any of the demographic variables, except for time since onset, race/ethnicity, and current employment status.

The means and standard deviations for the ACS3 are provided in Table 2. Participants in the MS group significantly retained fewer activities compared with those in the TBI group for the total ACS3 score, t(19.34) = −2.64, 95% confidence interval (CI) [−28.33, −3.30]; IADL retained activity scores, t(18.54) = −3.52, 95% CI [−31.9, −8.10]; leisure retained activity scores, t(19.8) = −2.3, 95% CI [−33.3, −1.66]; and current fitness scores, t(12.8) = −2.14, 95% CI [−66.4, 0.36]. The MS group’s before-illness leisure ACS3 scores were significantly higher than those of the TBI group.

Comparisons between preinjury or preillness and current ACS3 scores revealed significant changes (p < .001) in both the total sample (N = 22) and within each group (MS and TBI) across all domains (IADLs, leisure activities, fitness activities, social activities) and total scores (see Table 2).

Last, correlations between demographic characteristics (age, time since injury or illness onset, marital status, education, and gender) with ACS3 retained activity scores are presented in Table 3. For total retained activity scores, single or married participants had higher activity retainment than divorced participants, and women demonstrated higher retained activity scores than men, regardless of their diagnosis. For the specific domains, younger individuals demonstrated higher leisure retainment, and those with more education demonstrated higher fitness retainment.

The ACS3 demonstrates preliminary discriminant validity between people with MS and those with TBI, with participants with MS demonstrating decreased participation in life activities compared with those with TBI. Both groups experienced a significant participation decline from preinjury or illness baselines, with the MS group showing lower scores in total ACS3 retained activity scores, IADL current and retained activity scores, and current fitness scores.

The findings from this study are consistent with those of previous research that have used the in-person ACS. The MS group’s mean total ACS3 scores (M = 47.1, SD = 10.8) closely matched those of prior studies (M = 48.5, SD = 10.7; Goverover et al., 2020). Participants in the TBI group, although their current total scores were similar to those of community-dwelling adults (Boone et al., 2022), and higher than those found in a previous in-person ACS study (Goverover et al., 2017), still demonstrated significant postinjury participation decreases.

The magnitude of participation decline is noteworthy. Previous studies have found that healthy control adults retained more than 90% of their participation levels over 10 yr (Goverover et al., 2017, 2020), whereas in the current study the MS and TBI groups retained only 63% and 79%, respectively. These findings not only validate the observed participation declines but also demonstrate the ACS3’s ability to detect distinct patterns between conditions. These distinct patterns suggest condition-specific impacts rather than measurement artifacts from retrospective reporting.

The MS group’s lower current fitness participation aligns with the results of previous studies showing less physical activity in persons with MS compared with those with stroke or TBI (Hale et al., 2008; Nosek et al., 2006), which is particularly significant given the known benefits of physical activity for MS (Proschinger et al., 2022). Moreover, the social participation scores were similar between the MS and TBI groups, both of which demonstrated significant changes in retained activity scores. This is crucial given that social participation correlates with QOL, life satisfaction, and overall well-being in both groups (Erler et al., 2020; Pokryszko- Dragan et al., 2020; Sparling et al., 2017; Vos et al., 2019).

Limitations of this study include its modest sample size, demographic skew (predominantly female with above-average education), and reliance on retrospective self-reports. The significant difference in time since diagnosis or injury between groups (MS = 19 yr, TBI = 12 yr) may have influenced participation patterns and recall accuracy. Although the ACS3 was validated in an in-person format (Boone et al., 2022), its virtual administration in the current study lacks validation, potentially affecting the findings. Last, recalling preinjury or preillness activities over extended periods may reflect natural life transitions rather than purely condition-related changes. However, the comparable age between cohorts helps control for aging-related effects on participation outcomes.

Emphasizing participation as a rehabilitation outcome is essential for meaningful patient engagement. The ACS3 differentiated participation patterns between adults with MS and those with TBI, revealing significant declines in participation from preinjury or preillness to current status. It is valuable for capturing changes in participation by considering both current and previous participation levels. With respect to clinical work, it can guide personalized rehabilitation strategies, thereby improving patient outcomes. Future research should explore the ACS3’s validity and reliability in larger and more diverse populations to enhance generalizability.