Utilization-Focused Evaluation: Ten-Year Outcomes of a Fairweather Model Mental Health Program—Implications for Occupational Therapy

OBJECTIVE: This study utilized a mixed-design case-based utilization-focused evaluation (Patton, 1997, 2008) to explore (1) factors that contribute to success in a Fairweather model mental health housing program, (2) efficacy of a training program within the model, (3) client satisfaction within the model, and (4) occupational engagement in a peer-supported environment. Following the initial phase of the research, recommendations were made, and outcomes were followed over a 5- and 10-yr period.

SIGNIFICANCE: As psychiatric hospitalization stays have shortened, the need for quality mental health housing has increased. Occupational therapists are uniquely suited to facilitate transition to community-based residences.

Despite the social difficulties individuals with mental illness often have, persons with psychiatric disabilities are often placed in individual housing situations without consideration of other peer-support models that encourage interdependent living (Haertl, 2005, 2007). The inclusion of peer-supported models with emphasis on social and occupational engagement has been shown to facilitate personal satisfaction, occupational participation, and quality of life (Haertl, 2005, 2007; Haertl & Minato, 2006).

This study demonstrates the utility of program evaluation and the efficacy of peer-supported mental health housing models and discusses implications for occupational therapists.

INNOVATION: This study blended mixed methods (Creswell 1998a, 1998b; Creswell & Clark, 2007) and utilization–program evaluation (Patton, 1997, 2008) to evaluate outcomes in a peer-supported mental health housing program. Often, program evaluation and development result in recommendations with little long-term follow-up on the utility of recommendations and changes made.

The present study was unique in its collaboration with the case site and monitoring of long-term outcomes over a 10-yr period. Occupational therapy benefits from methodology that examines not only initial response to therapy but also outcomes that extend beyond a year. The collective and reflexive approach to the mixed-design utilization-focused review provided significant benefit to the case site.

APPROACH: In line with utilization-focused evaluation (Patton, 1997, 2008), research questions were formulated for this project in conjunction with a large Fairweather mental health organization. A mixed-design case-based study was conducted to explore “within a Fairweather mental health model, what factors distinguish clients who successfully graduate from a training program to a lodge from those who don’t?”

Additional questions explored specific demographics, occupational engagement within a Fairweather program, and organizational and client factors that contribute to satisfaction and retention.

RATIONALE: At the time of the original study in 2000, despite the robust outcomes once clients successfully graduated to a lodge, there was a problem with increased referrals, longer training requirements, and difficulty for some clients in successfully graduating. Utilization-focused evaluation, paired with mixed methods, provided a structured research-based means to explore the identified problem.

Following the initial results, outcomes were tracked over a 5- and 10-yr period in order to ensure organizational follow through and assess the utility of the original recommendations.

METHOD AND RESULTS: The site of the research was a large Fairweather model mental health housing program in the Midwest. The site provides residential, vocational, mental health, recreational, and living skills services to >300 clients/yr. To answer the research question, purposive sampling was used with current and past clients and current staff.

Utilization-focused evaluation was paired with a mixed-methods design to explore the research questions. Data collection and analysis included a review of charts of clients over a 3-yr period, surveys of 28 staff, and individual and group interviews of 45 current and past clients. Quantitative descriptive and statistical analysis were conducted on the chart and survey data and qualitative thematic analysis on the open-ended survey data along with the transcription of taped interviews (Haertl, 2005, 2007; Haertl & Minato, 2006).

Following analysis, as per utilization-focused evaluation, specific recommendations were made for the study site. Rates of hospitalization, satisfaction, retention, and levels of occupational engagement were tracked following the original research to continue the utilization-focused process.

Results from this project revealed more than 90% reduction in hospitalization, more than 500% increase in personal earned income, and high satisfaction with the program and model. The presentation will explore the role of the occupational therapist in collaborating with clients, family members, and service organizations in the utilization of a client-centered occupation-based practice to promote quality of life. The presentation will conclude with a discussion on evidence-based practice, the relevance of the results to occupational therapy, and suggestions for means by which therapists can increase their involvement in the development, management, and implementation of similar mental health service delivery models and program evaluation designs.

IMPACT STATEMENT: The results have important implications for occupational therapy practitioners working in mental health and those consulting with programs and systems. Often within occupational therapy research, emphasis is placed on individual interventions.

Comprehensive systems and program-based evaluation provides means to enhance service provision and positively affect the systems in which service delivery is provided. The presentation will focus on application of utilization-focused evaluation, mixed methods, and the consultant role and opportunities for occupational therapists.

References

Creswell, J. W. (1998a). Mixed-method research: Introduction and application. In G. Cizek (Ed.), Handbook of educational policy. San Diego, CA: Academic Press.

Creswell, J. W. (1998b). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage.

Creswell, J. W., & Clark, V. L. P. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.

Haertl, K. H. (2005). Factors influencing success in a Fairweather model mental health program. Psychiatric Rehabilitation Journal, 28, 370–377. http://dx.doi.org/10.2975/28.2005.370.377

Haertl, K. H. (2007). The Fairweather mental health housing model—A peer supported environment: Implications for psychiatric rehabilitation. American Journal of Psychiatric Rehabilitation 10, 149–162. http://dx.doi.org/10.1080/15487760701508201

Haertl, K. H., & Minato, M. (2006). Daily occupations in persons with mental illness: Themes from Japan and America. Occupational Therapy in Mental Health Journal, 22, 19–32. http://dx.doi.org/10.1300/J004v22n01_02

Patton, M. Q. (1997). Utilization-focused evaluation: The new century text (3rd ed.). Thousand Oaks, CA: Sage.

Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage.