Abstract
Importance: The efficacy of telehealth (TH) interventions needs to be studied.
Objective: To compare the efficacy, fidelity, acceptability, and feasibility of face-to-face (F2F) versus TH seated play (SP) interventions among children with autism spectrum disorder (ASD).
Design: As part of a larger randomized controlled trial, children were assigned to the SP group and received TH and F2F interventions over 8 wk using a pretest–posttest study design.
Setting: A research lab or through videoconferencing.
Participants: Fifteen children with ASD (ages 5–14 yr) were randomly assigned to the SP group and received the intervention F2F or through TH.
Intervention: Children received 16 SP intervention sessions (2 sessions per week for 8 wk).
Outcomes and Measures: Pretests and posttests included standardized fine motor assessments. Video coding compared socially directed verbalization during training sessions. Parents and trainers provided feedback on their experiences.
Results: Seven children received the intervention F2F, whereas 8 received TH intervention. Children in both subgroups showed similar training improvements in fine motor skills and socially directed verbalizations (ps > .01). Parents rated both interventions as acceptable and feasible; however, they reported longer preparation time and effort during TH interventions (ps < .01). Trainers reported greater parental involvement but more communication and technological issues during TH interventions. Fidelity checks indicated fewer reinforcements during TH versus F2F sessions.
Conclusions and Relevance: TH intervention is feasible and effective in improving fine motor and social communication performance. Clinicians should reduce parental burden and overcome technological issues.
What This Article Adds: This study confirmed the efficacy, fidelity, acceptability, and feasibility of delivering seated play, standard of care interventions for children with autism spectrum disorder via telehealth. However, clinicians should work on reducing parental burden and overcoming communication and technological issues related to telehealth.
Telehealth (TH) is a therapy delivery method that allows remote access to interventions through the use of information and communication technologies (American Occupational Therapy Association, 2018). The need and demand for TH rose sharply after the outbreak of the coronavirus disease 2019 (COVID-19) pandemic. TH provides a safe alternative to access therapies while limiting the risk of human-to-human virus transmission (Terrell et al., 2021). Although initial parent surveys collected during the pandemic suggested high satisfaction while receiving online physical, occupational, and speech-language therapies (93.7%–99.0% satisfaction; Tenforde et al., 2020), many families and health care providers have concerns about incorporating TH into their regular clinical practice beyond the pandemic (Aranki et al., 2022; Gabellone et al., 2022). For example, parents have concerns about the increased burden of supervision needed during TH sessions, whereas therapists have concerns about technological challenges and children’s ability to sustain attention during remote sessions (Gabellone et al., 2022). Taken together, these concerns regarding TH indicate an urgent need to compare the efficacy, fidelity, acceptability, and feasibility of TH versus face-to-face (F2F) delivery of standard of care interventions offered to children with autism spectrum disorder (ASD).
The current standard of care for children with ASD involves occupational, speech, and behavioral therapies targeting children’s fine motor, social communication, behavioral, and functional skills through applied behavioral analysis (ABA; Lovaas, 1987), Picture Exchange Communication System® (PECS; Benson et al., 2022; Bondy & Frost, 2003), Treatment and Education of Autistic and related Communications Handicapped Children (TEACCH; Mesibov et al., 2004), sensory integration (Case-Smith et al., 2015; Schoen et al., 2019), and motor learning and creative play principles (Bernier et al., 2022; Bhat et al., 2011) to provide session structure, practice and repetition, appropriate prompting, reinforcements, and trial and error learning. Using similar therapeutic principles, researchers have reported improvements in fine motor skills, sensory processing, and social functioning as well as cognitive functioning among children with ASD after sensory integration (Iwanaga et al., 2014,; Pfeiffer et al., 2011) or goal-directed, fine motor interventions (Bernier et al., 2022).
Previously, our research group reported positive effects on fine motor and communication skills after an F2F seated play (SP) intervention that was modeled with many of the aforementioned standard intervention approaches appropriate for children with ASD (Srinivasan & Bhat, 2013; Srinivasan et al., 2015, 2016). During the pandemic, we collected data on SP, standard of care interventions that were provided in both TH and F2F formats as part of a larger randomized controlled trial (RCT) involving school-age children with ASD. Although our overall RCT aims to compare the effects of various movement interventions with the standard of care, SP intervention, this article focuses on the differences in TH versus F2F method of delivery among children receiving the SP intervention on the basis of stakeholder input and measured child outcomes. Specifically, in this study, we compared the efficacy, fidelity, feasibility, and acceptability of an SP, standard of care intervention delivered via TH versus the F2F method among children with ASD using a pretest–posttest design. The standard of care, SP interventions used in this study involved tabletop play activities often practiced by children in occupational, behavioral, or speech-language therapy sessions to promote their social communication and fine motor skills.
The impact of the COVID-19 pandemic on health care access was devastating for children with developmental disabilities (Jeste et al., 2020). About 74% of parents of children with developmental disabilities reported that their children had lost access to at least one therapeutic or educational service during the pandemic (Jeste et al., 2020). Children with ASD and their families were particularly affected by the restrictions imposed by the pandemic; this impact can be attributed to several factors, including greater reliance on daily support and behavioral therapy services, increased symptom exacerbation because of routine disruptions, and overall increased parental burden and psychological stresses (Bhat, 2021; Bhat et al., 2021; Srinivasan et al., 2021).
A past parental survey of the Simons Foundation Powering Autism Research for Knowledge (SPARK) families showed negative effects of the pandemic on children’s ASD-related behaviors and parents’ mental health, with greater negative effects evident among children with greater ASD severity as well as greater cognitive, language, and functional delays (Bhat, 2021). Additionally, parents of children with greater cognitive, functional, and language delays felt that their child would benefit less from online services (Bhat, 2021). Given the lack of definitive evidence in support of TH-based interventions, it is important to compare stakeholder perceptions and objective, standardized treatment outcomes associated with F2F versus TH delivery of SP interventions among children with ASD.
A comprehensive systematic review of TH services among children with ASD and other childhood disabilities showed comparable effects of therapeutic interventions conducted via TH or F2F methods but also concerns about its efficacy in certain populations needing greater support (Ellison et al., 2021; Feldhacker et al., 2022). The majority of the studies included in the systematic review focused on parent training and education, with only a few studies providing direct therapy (with or without parent involvement) to children with ASD (Ellison et al., 2021). For example, using TH-based, cognitive–behavioral therapy, parents reported reduced anxiety and insomnia among children with ASD after completion of the intervention (Hepburn et al., 2016; McCrae et al., 2021). Similarly, using a TH-based social skill intervention, Cihon et al. (2021) also reported improved social skills postintervention in a small pilot study involving 3 children with ASD.
Given the scarcity of studies comparing the effects of direct therapy using F2F and TH delivery methods, it is not clear whether the previous findings can be generalized to standard therapeutic activities for children with ASD (e.g., speech or fine motor therapies). In this study, we compared the efficacy, fidelity, acceptability, and feasibility of an SP intervention provided using F2F or TH methods among school-age children with ASD between ages 5 and 14 yr. We hypothesized that subgroups of children with ASD receiving SP interventions via F2F and TH methods would demonstrate comparable improvements on measured outcomes related to fine motor skills and socially directed verbalization. Despite similar improvements in outcomes, parents and trainers may report challenges (related to technological issues, caregiver involvement, etc.) in the delivery of TH interventions.
Method
Participants
Fifteen families of children with ASD between ages 5 and 14 yr (M = 8.4 yr, SE = 0.6; 13 male children, 2 female children) participated in the study. Participants were recruited through phone calls, online announcements, and fliers distributed to schools, ASD services, and advocacy groups. We also recruited participants through the SPARK research match service (https://www.sfari.org/resource/spark/). Screening interviews were conducted before participation to confirm the child’s eligibility through birth and developmental history, collect demographic information (i.e., age, sex, race–ethnicity), and explain the study procedures. Parents completed the Social Communication Questionnaire (SCQ; Rutter et al., 2003) to explain their child’s ASD-related symptoms, which served as an initial check during the screening period (Table 1). SCQ has high sensitivity and specificity when used to screen for social communication delay among children with ASD (Marvin et al., 2017).
Postscreening parents were required to provide a copy of their child’s diagnostic record of ASD. Children were included in the study if they held (1) a professionally confirmed ASD diagnosis supported by a school record (e.g., a school psychologist record confirming an ASD diagnosis), (2) an Individualized Education Plan for ASD-related services, or (3) a medical or neuropsychological record from a psychiatrist or clinical psychologist using the Autism Diagnostic Observation Schedule (Lord et al., 1989) or Autism Diagnostic Interview–Revised (Lord et al., 1994) measures. Children were excluded if they had significant vision or hearing challenges, behavioral problems, medical or orthopedic conditions preventing participation, or a history of seizures. Each family’s assignment to the F2F or TH subgroup was based on the social restrictions at the time during the pandemic and family preferences. Seven children received the training sessions F2F, whereas the remaining 8 children received the training via TH. No significant differences were found in age, sex, or race–ethnicity between the F2F and the TH subgroups (ps > .05; Table 1).
Parents of all participating children completed the Vineland Adaptive Behavior Scales–Second Edition (VABS–II; Sparrow et al., 2005), the Social Responsiveness Scale (SRS; Constantino et al., 2000), and the Developmental Coordination Disorder Questionnaire (DCDQ; Wilson et al., 2009) to provide information on their child’s baseline social communication skills and motor performance (Table 1). No differences were found in VABS–II, SRS, and DCDQ scores between the F2F and TH subgroups (ps > .05; Table 1). Written informed consent and written–verbal assent was obtained from parents and children with ASD, respectively, before study participation. All study procedures were carried out in accordance with the protocol approved by the University of Delaware institutional review board (IRB). Note that this was a two-site study between the University of Delaware and the University of Connecticut with a single IRB protocol registered with the University of Delaware IRB. We have obtained written parental and experimenter permission–consent to use pictures for this publication.
Study Design and Sample Size Estimation
This study is part of a larger RCT that seeks to compare the effects of whole-body movement-based interventions with the current standard-of-care, SP intervention for children with ASD. Because of the pandemic, we incorporated a hybrid intervention model that allowed families to choose between F2F or TH intervention delivery methods on the basis of their preferences. In this analysis, we used a pretest–posttest design to examine the efficacy, fidelity, acceptability, and feasibility of delivering SP activities either F2F or via TH. We wanted to examine the SP and movement groups separately for this analysis given the differences in the nature of training activities (sedentary vs. whole body) and unique challenges associated with each of these training protocols when delivered via TH (e.g., potential technological or communication issues).
We determined the sample sizes using G power on the basis of effect sizes reported in our previous publications for motor and social effects of SP, standard of care interventions (i.e., standard mean difference for fine manual control [FMC] composite score = 0.33; standard mean difference social verbalization percentage = 0.78%; for variable definitions, see Srinivasan et al., 2015, 2016). For the primary fine motor and social variable, with a significance criterion of α = .05, statistical power = .80, and aforementioned effect sizes, a minimum sample size required per group would be 10. In this study, we recruited 15 participants for the full group. We were able to ensure fairly equal number of participants across the two subgroups (F2F subgroup, n = 7; TH subgroup, n = 8).
Procedures
Children received standard of care, SP interventions over 8 wk, with pretest and posttest administered in the week before and after the intervention period. Children were offered 16 SP intervention sessions for 60 to 90 min per session, two sessions per week that included reading (e.g., reading picture books or Story Online stories), building (e.g., building Lego™ and Play-Doh™ creations; see Figure A.1 in the Supplemental Material, available online with this article at https://research.aota.org/ajot), and art–craft activities (e.g., making art or crafts that require fine motor skills such as cutting, coloring, pasting, folding paper, decorating; Supplemental Tables A.1 and A.2 and Figures A.1–A.4). Each training session included interactions between the child, the trainer (i.e., physical therapist, trained graduate student), and a trained undergraduate student who served as a buddy and model to the child. The graduate and undergraduate students were trained by the third and fourth authors, both pediatric physical therapists with behavioral training and substantial experience working with children with ASD.
The training included an initial and weekly review of ASD-related behaviors, treatment strategies to reinforce learning and address behavioral challenges (e.g., ABA and motor learning principles), details on weekly training activities, and actual rehearsal of training instructions with mock participants. Caregivers were able to participate and provide oversight of the TH sessions on the basis of their child’s needs. For TH subgroup families, parents would meet with the trainers on the conference call about half-hour before the intervention started to ensure appropriate camera placement, audio levels, appropriate access to intervention supplies, preparing the workspace, and instructions on how to support their child during the activity—after which the child was invited to join the session. The participating families chose between the F2F or TH training delivery method on the basis of their preference and ongoing social restrictions associated with the pandemic.
Child Outcome Measures
We tested each child’s fine motor performance during a pre- and posttest using the Fine Motor Precision and Fine Motor Integration subtests of the Bruininks–Oseretsky Test of Motor Proficiency–Second Edition (BOT–2; Bruininks & Bruininks, 2005). Note that the BOT–2 has been approved for telepractice by the test publishers (Pearson Clinical Publishing). During the Fine Motor Precision subtest, children engaged in fine motor activities involving coloring shapes, drawing lines through paths, connecting dots to form a diamond, as well as folding and cutting paper. During the Fine Motor Integration subtest, children copied different shapes such as a circle, triangle, diamond, star, wavy line, and so forth. An FMC composite score was calculated on the basis of the point, scaled, and standard scores obtained for each subtest. We also developed a training-specific task using Lego blocks to build a standard Lego house creation (Supplemental Figure A.1). The time to task completion was calculated to measure the child’s building performance. Children in the F2F subgroup completed all assessments F2F in the tester’s presence, whereas children in the TH subgroup received the test via TH.
To ensure consistency of test administration across delivery methods, we dropped off a webcam, tripod, and two fine motor test booklets (one completed exemplar and an empty booklet) with the TH family. Next, we met with the parent to explain the Zoom set up and each of the fine motor tasks; we then asked them to show the child the pages from the exemplar booklet. We also encouraged the parent to pin the tester’s Zoom view to receive a closeup of the tester’s actions and workspace. During the testing session, the trainer reminded the parent and the child of the task instructions, the parent used the exemplar booklet (for BOT–2) and an instruction sheet (for Lego test; Supplemental Figure A.1) to show the task and repeated the instructions, and then the child received an opportunity to complete the task. The parent immediately showed the child’s drawing to the tester via Zoom, and the image of the child’s drawing was captured. After testing, the parents returned the test kit and the hard copy of the child’s completed BOT–2 test booklet to the researchers.
Training-Specific Measure of Socially Directed Verbalization
Socially directed verbalization was examined during an early (Session 1 or 2) and a late training (Session 15 or 16) session. Datavyu software was used to code the child’s socially directed verbalization, and the coded files were processed in MATLAB. Specifically, each child’s socially directed verbalization was coded for the duration of verbalization (e.g., words, sentences, phrases) or vocalizations (e.g., word approximations, laughs) directed toward a trainer (physical therapy graduate student), model (the student who was the buddy to the child), or parent.
Parent and Trainer Surveys for Stakeholder Feedback Postintervention
Postintervention, we collected feedback from parents and trainers about their experiences on F2F and TH methods of intervention delivery using Google forms. Parents were asked to complete the forms within a week’s time, and an additional reminder was sent out after 1 wk. Parent questionnaires assessed training feasibility through questions about preparation time (e.g., “How demanding was the preparation time for the training sessions [travel, setup, etc.]”?), parent effort (e.g., “How much effort did you put in to keep your child engaged during the training sessions?”), and the child’s attention during the training sessions (e.g., “How attentive was your child during the training sessions?”).
Acceptability of training was evaluated through questions on parent satisfaction (e.g., “Overall, how satisfied were you with the intervention delivery method [TH–F2F] your child received?”), appropriateness (e.g., “Was the intervention delivery method [TH–F2F] appropriate for your child?”), and benefit (e.g., “Did your child benefit from the intervention delivery method [TH–F2F] he/she received?”). Parents used a 5-point Likert scale to provide responses. For the purposes of statistical analyses, these responses were converted to a 3-point scale (1 = feasible/satisfied, 2 = in between/neutral, 3 = not feasible/not satisfied). Lastly, parents were also asked about their preference for method of training delivery (F2F vs. TH) if the study were to be repeated in the future (i.e., “If we were to repeat this study, which delivery method would you like your child to receive?”).
On the trainer exit questionnaires, we assessed feasibility using questions about communication with the family (e.g., “On average, how easy was it for you to communicate with the child and parent during the training sessions?”) and parent involvement during training (e.g., “On average, what percentage of time was the parent involved in the training sessions?”). The trainers responded on a 5-point Likert score, which was later converted to a 3-point scale (1 = feasible, 2 = in between/neutral, 3 = not feasible). Trainers were also asked about the challenges of applying TH delivery methods compared with the traditional F2F method. Specifically, we asked about the impact of the TH method on session length (e.g., “On average, how did telehealth affect the length of the training sessions?”) and technological issues faced during TH training sessions (e.g., “On average, to what degree did parent familiarity with technology affect your ability to deliver the intervention sessions?”). The trainers answered the questions binarily (i.e., “TH longer/more difficult than F2F” or “TH not different than F2F”).
Training Protocol for Seated Play Intervention
Our SP intervention resembled treatment approaches used within the current standard of care for children with ASD, which often targets children’s social communication and fine motor skills. During the training sessions, trainers used evidence-based approaches grounded in ABA, PECS, and TEACCH; they also used (1) motor learning principles such as task breakdown, repetition, modeling, graded prompting, and reinforcements (i.e., ABA and motor learning); (2) visual picture schedules showing step-by-step instructions for the building and art and craft tasks (i.e., PECS; Supplemental Figures A.2–A.4); (3) consistent session structure using similar and small spaces and familiar trainers and routines (i.e., TEACCH); and (4) trial-and-error learning and periods–prompts of free play–exploration (i.e., motor learning; Supplemental Tables A.1 and A.2). Specifically, trainers provided visual and verbal cues–prompts and manual assistance as needed; in addition, they provided positive reinforcements throughout the session.
A picture schedule was shown to each child to provide structure to the session and to explain the different training conditions and activities. Trainers also encouraged children to make choices (e.g., choice of books, types of supplies used, color of crayons, choice of decorations) and to be creative during the fine motor activities, including a free build and art periods or prompts. Note that all of these elements were provided in both F2F and TH subgroups. For the TH subgroup, parents partnered with the virtual trainer and assumed an active role in providing manual assistance, directing the child’s attention to the visual instructions and providing additional help, as needed. All participating children received more than 85% of the planned intervention sessions (number of sessions received: F2F, M = 15.0, SE = 0.2; TH, M = 16.0, SE = 0.0; p > .05). No significant group differences were found in session duration in minutes (F2F: M = 71.3, SE = 5.2; TH: M = 71.4, SE = 4.5; p > .05).
Children engaged in reading and fine motor activities across the following conditions: (1) hello: children greeted the trainers, played ice breaker games, and engaged in natural conversations with adult partners; (2) reading: children read or listened to a picture book, answered questions, and engaged in back-and-forth conversations with trainers; (3) warm-up: children engaged in hand dexterity tasks (e.g., peg board activities, string art, beading) to warm up finger and hand muscles; (4) building: children built Lego or Playdoh creations based on visual instruction sheets (Supplemental Figures A.1 and A.2); (5) free build: children built creations of their choice or modified the creation made during the building activity; (6) art–craft: children made art–craft creations by coloring, cutting, gluing, and folding materials (Supplemental Figures A.3 and A.4); and (7) farewell: children reflected on session activities, said goodbye, and cleaned up the supplies.
During F2F training sessions, the trainer and model (who acted as a buddy to the child and emulated the training activities) sat on either sides of the child and provided demonstrations and visual, verbal, or manual assistance as needed (Figure 1A). For the TH intervention, we delivered the training materials to the child’s home, along with training packets containing supplies and visual instruction sheets for each day’s activities and a general art–craft supply kit, including stationery, crayons, markers, scissors, glue, glitter, mini-stapler, and so forth. Later, parents met with the trainers on Zoom and were provided step-by-step instructions on ways to support their child during the intervention sessions. Parents also debriefed with the trainers at the end of the previous day’s activity and in the minutes before starting a new TH session to be clear about their role.
During TH training sessions, the parent and child, the trainer, and the model joined the Zoom call and could view each other within the gallery view (Figure 1B). The trainer recorded a pinned view of the parent and the child because it provides a more closeup view of the child’s actions, and the model recorded a gallery view of everyone involved. Both Zoom views provide a careful picture of the child’s motor and social abilities within a videoconferencing context. Parents were asked to provide oversight of their child’s therapy session with the goal of redirecting the child’s attention to the training activities and providing visual, verbal, or manual prompts, as needed (Supplemental Tables A.1 and A.2 and Figures A.1–A.4).
Training Fidelity Measure
We developed a training fidelity measure to examine the trainer’s quality of providing instructions, prompts, and reinforcement bids as needed during the training sessions (Supplemental Table A.2). A single fidelity coder coded a randomly selected early and late training session and coded the trainer’s and model’s–parent’s behaviors during the session (97 behaviors and 134 points in total). Within each condition, we rated the trainer’s and model’s–parent’s instructions and actions to provide task-related instructions–prompts and reinforcements. For example, the coder assigned points to the trainer for providing session structure or transitions by saying “What are we doing first or next?” while showing a picture schedule of the different intervention conditions. The trainer and model–parent were rated for offering verbal or gestural reinforcements for each activity, such as “You did it, great job!” with or without a F2F or virtual high-five, fist bump, and so forth. Instructions–prompts and reinforcements for each condition were associated with point scores (see Supplemental Table A.2 for the entire fidelity checklist). The fidelity score percentages for providing instructions and prompts, reinforcements, and overall total fidelity scores were calculated and compared across the TH and F2F subgroups.
Data Analysis
For the quantitative outcome measures (BOT–2 FMC composite score, custom Lego test completion time, and percentage of socially directed verbalization), the mean and coefficient of variation (CV) were compared between the TH and F2F subgroups at the pretest. Independent t tests were used to compare the magnitude of training-related changes (posttest–pretest) in fine motor and social outcomes between the F2F and TH subgroups. Additionally, we calculated effect sizes using the Hedges’s g measure (considered ideal for small samples) for training-related improvements in BOT–2 and socially directed verbalization in both subgroups (TH and F2F). For each stakeholder feedback question, χ2 tests were used to compare the distribution of respondents (i.e., proportion of parents who indicated a specific response) between F2F and TH subgroups.
Results
Child Outcomes
In terms of standardized measures, the BOT–2 FMC composite mean score and variability (or CV) did not differ significantly between the TH and F2F subgroups at pretest, t(13) = −1.3, p > .05. Similarly, no differences were found in the magnitude of training-related improvements from pretest to posttest (i.e., posttest–pretest scores) for all outcome measures across F2F and TH subgroups, including the BOT–2 FM composite score: training-related improvement TH versus F2F, t(13) = 1.95, p > .05; TH, Hedge’s g = .28, 95% confidence interval (CI) [−0.57, 1.12]; F2F, Hedge’s g = .81, 95% CI [−0.31, 1.94] (Figures 2A–2C). For the custom Lego test, the mean, CV, and intervention-related reduction for time to task completion did not differ between the TH and F2F subgroups: pretest TH versus F2F, t(13) = −0.24, p > .05; training-related improvement TH versus F2F, t(13) = 0.68, p > .05; TH subgroup improvement, Hedge’s g = −.30, 95% CI [−1.15, 0.55]; F2F subgroup improvement, Hedge’s g = −.39, 95% CI [−1.35, 0.58] (Figures 2D–2F). Lastly, the means, CV, and the magnitude of training-related changes in socially directed verbalization did not differ between the TH and F2F subgroups: pretest, t(13) = −0.34, p > .05; training-related improvement TH versus F2F, t(13) = −0.42, p > .05; TH subgroup improvement, Hedge’s g = .31, 95% CI [−0.54, 1.16]; F2F subgroup improvement, Hedge’s g = .17, 95% CI [−0.75, 1.0] (Figures 2G–2I). Overall, no significant differences were found in fine motor or social verbalization variables at baseline and after training (i.e., mean, CV, or change scores) between the TH and F2F subgroups.
Parent Feedback
Compared with the F2F subgroup, parents of children in the TH subgroup reported lower feasibility of preparation time during the training sessions (i.e., more demanding prep time; percentage of parents stating preparation time feasible): F2F, 83%; TH, 67%; χ2(2, 15) = 1,042.1, p < .01. These parents also reported lower parental effort (percentage of parents stating parent effort feasible): F2F, 100%; TH, 0%; χ2(2, 15) = 4,898.0, p < .01 (Figure 3A). The percentage of parents reporting high attentional focus in their children was the same in both subgroups (percentage of respondents stating their child was paying attention): F2F, 83%; TH, 83% (Figure 3A). Parental feedback on therapy-related satisfaction, appropriateness, and benefits did not differ between the F2F and TH subgroups (all parents in both groups expressed high levels of satisfaction; Figure 3B). In terms of parents’ preferred method of intervention delivery for future interventions, 83% of the parents in the F2F subgroup preferred F2F, 33% of parents in the TH subgroup preferred TH, and 17% of the parents in the F2F and 67% of the parents in the TH subgroup were open to either F2F or TH interventions, χ2(1, 15) = 6,792.3, p < .05.
Trainer Feedback and Training Fidelity
Compared with the F2F subgroup, trainers of children in the TH subgroup reported greater communication difficulties (percentage of trainers reporting difficulty with communication): F2F, 16.7%; TH, 62.5%; χ2(2) = 86.1, p < .05 (Figure 3C). However, trainers reported less parental involvement in the F2F subgroup compared with the TH subgroup (percentage of trainers reporting minimal parent involvement): F2F, 83.3%; TH, 62.5%; χ2(2) = 2,425.4, p < .05 (Figure 3C). In terms of trainer reported challenges, 37.5% of trainers in the TH subgroup felt the training sessions were longer than the F2F subgroup because of the additional parent training time, and 75% of the trainers in the TH subgroup reported more technological issues during TH sessions compared with F2F sessions (Figure 3D). After checking the training fidelity scores, only the reinforcement-related fidelity score percentage differed between the TH and F2F subgroups, with the TH sessions involving slightly fewer reinforcements than the F2F sessions (TH vs. F2F reinforcement fidelity score): early session, t(13) = 3.02, p < .05; late session, t(13) = 2.48, p < .05 (Supplemental Table A.3). The session length in minutes and the fidelity percentage scores for the total session and the prompt–instructional bids did not differ between the two subgroups (ps > .05; Supplemental Table A.3).
Discussion
In this study, we compared the acceptability, feasibility, fidelity, and efficacy of TH versus F2F delivery of a standard of care, SP intervention to promote fine motor and social communication skills among children with ASD. Using standardized tests and training-specific measures, we confirmed our hypothesis of similar improvements in fine motor performance and socially directed verbalization among children with ASD after TH and F2F interventions. Moreover, no significant differences were found between the TH and F2F subgroups on mean scores and variability measures of fine motor performance and social verbalization. Parents rated both intervention delivery methods as satisfactory, appropriate, beneficial, and feasible to implement with their children; however, they reported longer preparation time and the need for greater effort on their part during the TH versus F2F training sessions. Similarly, the trainers reported a positive effect of TH in terms of encouraging greater parental involvement; however, negative effects were also reported, including additional parent training time and greater communication difficulties with the family and disruptions because of technological issues.
Training fidelity data or session durations did not differ across the two subgroups, except that the reinforcements offered were slightly lower for the TH subgroup compared with the F2F subgroup. In short, our pilot study suggests that TH is a valid, acceptable, feasible, and efficacious method to deliver standard interventions for improving fine motor and social communication skills among school-age children with ASD. Clinicians should expect to face communication-related and technological challenges such as audio–visual lag or interruptions and to be prepared for additional parent training time associated with TH interactions. Before committing to TH sessions, families need to be informed about increased parental involvement and pretraining time investment to ensure successful TH training delivery.
Similar Fine Motor and Social Communication Outcomes Between Telehealth and Face-to-Face Subgroups
Children with ASD demonstrate comparable improvements in fine motor and social communication skills after receiving SP intervention via F2F and TH methods. Specifically, using standardized fine motor subtests of the BOT–2 and training-specific Lego tasks, we found similar fine motor improvements (i.e., improved BOT–2 FMC composite scores and reduced time to completion for the Lego test) after TH or F2F methods of delivery. Fine motor improvements after interventions targeting fine motor skills have been reported earlier among children with ASD, including the use of art-based interventions and sensorimotor strategies for in-seat behaviors (Benson et al., 2022; Bernier et al., 2022,; Case-Smith & Miller, 1999; Criss, 2013; Pfeiffer et al., 2011; Srinivasan et al., 2015). For example, using the F2F training delivery method, our research group found improved fine motor skills among children with ASD after 8 wk of F2F fine motor SP interventions (Srinivasan et al., 2015). Pfeiffer et al. (2011) also found that parents and teachers reported improvements in sensory processing, motor skills, and social functioning among children with ASD after sensory integration and fine motor interventions. In fact, TH-based occupational therapy practice also resulted in improved handwriting among children with fine motor difficulties (Criss, 2013).
Similarly, Gibbs et al. (2021) found that a collaborative approach between occupational therapy practitioners and families via TH after an initial clinic-based interaction was effective for implementing a sensory diet at home among children with ASD; in fact, it led to greater carryover of skills. Our work adds to this literature by comparing F2F versus TH methods of delivery. Our findings of comparable fine motor improvements across TH and F2F delivery methods are encouraging and provide further support for the value of using TH-based fine motor training when F2F delivery formats are not feasible. To overcome the barriers of TH, such as difficulties in observing trainers’ hands or their materials, we provided clear demonstrations of fine motor skills by bringing the objects close to the camera (Figure 1B). Families were also provided step-by-step visual instruction sheets to help the child and parent follow along with the trainers during the sessions. We also encouraged and cued the parents to provide demonstrations and manual assistance when the child needed additional help. During conversations, trainers waited longer to account for the communication lag because of streaming issues.
Children in both TH and F2F subgroups showed similar improvements in socially directed verbalization, which aligns with other studies that have reported improved communication skills after TH-based ABA interventions (Ferguson et al., 2022). Training-related increase in socially directed verbalization could be attributed to the growing relationship between the child and the trainer as children spend more time with “friends” (Srinivasan et al., 2016; van de Groep et al., 2020). It has long been a concern that social interactions through videoconferencing technologies may not create the same level of social connections between the child and the trainers as that of F2F interactions. During the pandemic, when children lost access to a majority of F2F interactions, families shared that being able to connect with “virtual friends” was something that several children in the study looked forward to.
Moreover, because many children were also receiving academic programming and school-based therapy services online, we found that children were familiar with and open to online interactions. Trainers in our study also deliberately altered their interactions to provide comparable opportunities for social interactions (greetings, questions, choices, reflections, etc.) and verbal–gestural reinforcements (e.g., in-person high-fives were replaced by virtual high-fives) during TH versus F2F interventions. The similar levels of training-related social improvements across the TH and F2F subgroups in this study indicate that these strategies were successful and may be used during TH delivery to foster social connections between children and adult partners consistent with F2F delivery methods.
Increased Parental Burden but Greater Parental Involvement During Telehealth Interventions
In this study, parents of children who received TH-based standard of care interventions reported longer preparation time and greater effort in supervising their child compared with F2F intervention delivery. Similar findings were reported in previous survey studies in which parents expressed concerns about playing an active role in the intervention (Gabellone et al., 2022). Although TH intervention increases the oversight burden for parents, it also increases their involvement in their child’s intervention, making them more aware of the training activities and the challenges that their child face while completing them. This participation, in turn, leads to greater buy-in from families, increased collaboration with trainers, and better caregiver understanding of their child’s needs and abilities (Bhat et al., 2021 ; Srinivasan et al., 2021).
Using a TH-based, parent-mediated early intervention program, Gentile et al. (2022) found increased parental empowerment, reduced parental stress, and improved parental ability to support their child’s learning. In fact, TH interventions may teach parents strategies that they could use with their child even outside training sessions and therefore potentially increase the carryover of training to children’s naturalistic environments and interactions (Bhat et al., 2021 ; Srinivasan et al., 2021). However, for those children who need more support, TH delivery can increase parental burden significantly and may not be a preferred intervention delivery method. More research is clearly needed in this area to determine criteria for choosing pediatric clients suited for TH interventions.
Trainer’s Challenges With Communication and Technological Issues During Telehealth Interventions
Trainers in this study reported greater communication and technological issues during TH compared with F2F interventions. They also reported longer training lengths because of the additional parent training time before starting the intervention session. Compared with the F2F subgroup, trainers of the TH subgroup provided fewer reinforcements during training sessions, most likely because of the communication challenges associated with videoconferencing. Similar difficulties with TH-based interventions have been reported in previous studies (Gabellone et al., 2022). In a survey circulated to health care providers, therapists reported more fundamental barriers such as a lack of appropriate and effective technologies and devices, lack of familiarity with videoconferencing technology, and poor broadband internet connectivity when providing TH interventions (Gabellone et al., 2022).
To widely apply TH-based approaches for treating children with developmental disabilities, technologies and broadband internet connections must be made accessible, high speed, and easy to use for people with varying levels of technological literacy. In this study, we developed step-by-step instructions for installation and use of the videoconferencing software, camera or webcam setup, space setup to allow optimal view of child and trainer, and organization of supplies for training sessions. We met with the family (in person or online) before the training sessions and helped them set up the videoconferencing technology and training materials in their home, troubleshooted device and technology-related issues, ensured appropriate set up of workspace and supplies, and provided suggestions for activities and strategies to assist and reinforce their child during the training sessions.
Limitations and Future Studies
This study involved a small sample size (7–8 children per subgroup, TH or F2F). Future studies with larger sample sizes should further replicate our study findings. Although telepractice use of the BOT–2 is now an acceptable approach on the basis of publisher statements, BOT–2 administration using the TH format is a novel approach and needs further testing and validation.
Clinical Implications for Occupational Therapy Practitioners, Other Clinicians, and Caregivers
Our preliminary study comparing TH and F2F standard of care ASD interventions indicates that the TH delivery method is an acceptable, feasible, and effective way to provide therapeutic services to children with ASD. Our findings have the following clinical implications:
- ▪
TH could be equally effective in improving fine motor and social communication skills of children with ASD as the traditional F2F delivery method.
- ▪
TH interventions may not work well for children needing more support because of the increased parental burden of supervising the intervention.
- ▪
If TH intervention is a preferred method for a child and family, then occupational therapy practitioners must ensure additional technology- related training for families and must use optimal strategies to address communication challenges that they may face during interactions with children and their families. For example, occupational therapy practitioners and other clinicians should ask parents to use videoconferencing settings such as pinned speaker views for clear closeup views of the clinician.
- ▪
Clinicians must adjust their training strategies, such as pausing and slowing down their speech, providing clear demonstrations of fine motor skills (i.e., closeup views of the activity and supplies), sharing screen while reading books to maintain the child’s attention, and providing parents printable worksheets and instructions to follow along during training.
- ▪
It is important for occupational therapy practitioners, other clinicians, and caregivers to receive technology training to increase their familiarity and comfort level with videoconferencing for TH intervention delivery.
- ▪
For families that prefer TH, we recommend scheduling upfront a preintervention orientation session (and additional sessions based on family request) to setup videoconferencing software and any other devices needed, describe therapy activities, and clearly explain expectations around parental involvement to ensure family buy-in and success of TH-based intervention sessions.
Conclusion
In conclusion, our study offers valuable insights into the efficacy, fidelity, acceptability, and feasibility of TH interventions for children with ASD. We demonstrated that TH can be equally as effective as F2F interventions in enhancing fine motor skills and social communication among children with ASD. Moreover, both parents and trainers rated both interventions as acceptable and feasible. However, it is important to recognize that TH may not be suitable for all children, particularly those who require higher levels of support because of the increased parental burden. Thus, it is important for both clinicians and caregivers to receive adequate technology training to ensure smooth TH intervention delivery. Overall, our findings support the use of TH as a viable method for delivering standard-of-care interventions for children with ASD, with the caveat that tailored support and training are essential for maximizing its benefits.
Acknowledgments
We thank all the children and families who participated in this study. We also thank the Simons Foundation Powering Autism Research for Knowledge (SPARK) clinical sites and the research participant match service staff for their help with recruitment. Finally, we thank Marissa Heino, Sarah Williams, Jill Dolan, and Hannah Laue (undergraduate students from the University of Delaware) and Catherine Myers, Madeline Kaba, Andrea Hernandez, and Amber Bardsley (undergraduate students from the University of Connecticut) for their help with data collection and data analyses. This work was supported by the Institutional Development Award (IDeA) program (National Institute of General Medical Sciences Grants P20-GM-103446 and U54-GM104941) and a Dana Foundation Clinical Neuroscience Award. Sudha Srinivasan’s work on this article was supported by a Research Excellence Program Award from the University of Connecticut. The coauthors have no financial or other conflicts of interest to disclose. We are not consciously aware of, but would acknowledge the possibility of, author biases that might influence equitable presentation of data related to social, ethnic, or cultural variables.