Information Commonly Unreported in Intervention Effectiveness Studies

Titles help indexers categorize articles so that they can be easily accessed. If essential information is missing from the title, the article may never be accessed by interested researchers, policymakers, health insurers, practitioners, and consumers. Critical elements to include in titles to fully represent the type of research study completed are participant allocation and treatment design (e.g., randomized controlled study, two-group controlled study), intervention (and comparison treatment, if studied), and sample. An example of such a title is “Virtual Reality Versus Computer-Based Cognitive Therapy for Adults With Traumatic Brain Injury: A Pilot Randomized Controlled Study.”

Although AJOT article abstracts are limited to 150 words or less, key elements that must be reported are frequently left out. Because the abstract is the most commonly read section of an article, authors should provide information needed to assess the study's internal and external validity. At a minimum, they should include sample size, recruitment procedure, and experimental and control interventions (e.g., “20 participants recruited from a convenience sample participated in the fall prevention intervention”). If space allows, authors should also report the type of professionals providing the intervention, the setting in which the intervention was carried out, and blinding procedures (i.e., whether participants and therapists were blinded to group assignment, whether outcomes were assessed by a blinded researcher).

The last paragraph of the introduction should explain the study's purpose and state the specific research questions. These questions should be used to guide the presentation of data in the Results and Discussion sections. Although AJOT submissions commonly provide the purpose of studies, the actual research questions are frequently unreported.

The most compromised section of AJOT submissions is the Method section, which often lacks sufficient detail to appraise and replicate the studies presented. We recommend using the following subheadings in the order in which they are presented: Research Design, Participants, Instruments (or Measures), Intervention, Data Collection, and Data Analysis. Commonly unreported data in these sections are discussed below. (Because most authors correctly address the Data Analysis section, it is not discussed in the sections that follow.)

Including a Research Design section as the first subheading of the Method section is usually beneficial. Here, briefly describing the research design and stating that institutional review board approval and participant consent were obtained are important. Our experience has been that when this section is not provided in a manuscript, ascertaining this information elsewhere in the text is difficult. In many instances, the research design is never stated.

The Participants section should describe the inclusion and exclusion criteria, recruitment process, and setting. If randomization was used, the method used in the assignment sequence and concealment process should be made clear. Information regarding participant flow (e.g., number of participants assessed for eligibility, number enrolled, number assigned to specific intervention groups, number lost, final number included in analysis) should be provided in the first paragraph of the Results section, accompanied by a flow diagram (see Gutman, 2010). Similarly, participant demographics should also be provided in the first paragraph of the Results section rather than in the Participants section.

The Instruments section should describe each assessment used to measure an outcome variable. The instrument’s intended purpose, population for which it was developed, type of data collected (e.g., self-report, Likert, total score), and established psychometric properties should be clearly stated. Reporting precise numerical values established for specific types of reliability and validity rather than imprecise modifiers such as good, moderate, or high is necessary. Reporting these values is particularly important for readers who may be unfamiliar with the instruments and need the values to appraise their rigor.

Another section commonly underreported in AJOT submissions is the Intervention section. A thorough description of all interventions, procedures for and amount of contact in control conditions, type and number of interveners, and blinding of therapists is critical for study appraisal, comparison with other studies, replication, and inclusion in systematic reviews and meta-analyses.

Key elements of the intervention should be concisely described. Whether the intervention is manualized should be clearly stated, and authors should provide a Web site address (or contact information) to enable readers to access the manual and training documents. Intervention elements that must be precisely stated in the manuscript text are intervention duration and intensity (i.e., number and length of sessions, length of intervention over time), the content of each session and the delivery method (i.e., individual sessions vs. group treatment), and how the intervention was tailored for specific participants with comorbid conditions or unique clinical characteristics. Indicating whether the participants simultaneously received other services whose impact on study variables could not be controlled is also critical.

Information about the professionals who provided the intervention is frequently unreported in AJOT submissions. Authors should report the type and number of professionals who provided intervention, the eligibility criteria used to select therapists, the experience level of the participating therapists, whether therapists received training in intervention administration (and what that training consisted of), and whether therapists were blinded to group assignment. Describing this information for therapists providing intervention to the experimental and control groups is necessary, and indicating whether the same therapists provided intervention to both the experimental and control groups is critical. If therapists provided treatment to both groups, authors must indicate whether blinding of therapists to group assignment was achieved and how blinding was ensured.

Also frequently left out of AJOT submissions is information about intervention fidelity—the procedures used to ensure that the intervention was administered uniformly and in accordance with written manuals, particularly when multiple therapists were used. Methods to ensure intervention fidelity often include videotaping or audiotaping therapy sessions for later analysis, holding group supervision sessions in which therapists and principal investigators discuss sessions to ensure that therapy is being administered as intended, and using checklists completed by therapists and observing investigators to ensure that key elements were addressed in each session.

The section most frequently missing from the Methods section is the Data Collection section. This section should detail how, when, and by whom each set of data was collected. It is essential to tell readers who collected the data and whether multiple data collectors were used, whether data collectors were blinded to group assignment, whether data collectors were trained, whether interrater reliability was established between raters, and whether data collectors were different from those who provided intervention. Without this key information, readers cannot assess the possibility of bias or whether compromised data collection procedures affected the results.

As stated earlier, the Results section should begin with a brief description of participant flow, which should be precisely illustrated in a detailed flowchart. Participant demographics should also be provided in the Results section, or the reader should be referred to a table providing them. In accordance with minimal reporting standards, age, gender, race–ethnicity, and educational and socioeconomic level should be reported. After this first paragraph, the organization of the Results section should be guided by the order of the research questions listed in the purpose statement.

In addition to an interpretation of findings, the Discussion section should examine findings with regard to the type of control or comparison treatment used and the influence of compromised blinding procedures, lack of intervention fidelity, and different therapist skill levels. If specific clinical implications beyond the findings can be derived, this information should be placed under a separate subheading. The Discussion section should include a brief subsection with a bulleted list summarizing the implications of the research for occupational therapy practice. This new requirement went into effect with the current issue of AJOT. Additional subheadings in the Discussion section should include Limitations and Future Research.