Vocational Fit Assessment (VFA) Clinical Utility & Algorithm Accuracy

PURPOSE: Individuals with disabilities experience decreased employment, socioeconomic status, health, and quality of life as compared to those without disabilities. This is especially true for the population of individuals living with intellectual disabilities. Most services for this group fail to facilitate a successful transition from school to integrated, community employment. ‘Job matching is the collaborative, data-based decision making process used by transition teams to determine the best fit between an individual’s abilities and preferences and the job’s environmental and occupational demands.’ Traditionally, a practitioner must evaluate each individual combination of abilities and demands, each and every time they engage in the job matching process. The Vocational Fit Assessment (VFA) is a measure of employment status appropriate for people with intellectual and developmental disabilities in transition to adulthood and employment and was created to match the worker’s abilities to the job demands, increasing the likelihood that the worker would be successful at that particular job. This study evaluates VFA clinical utility and algorithm accuracy.

DESIGN: Prospective cross-sectional study design.

METHOD: Participants (n=200) were employment support providers with greater than 1 year of experience using the VFA as registered users of VocFit.com. Electronic surveys (i.e., REDCap) were used to collect Likest-type ratings of VFA (1) relevance, (2) interpretability, (3) acceptability, (4) practicability, (5), accessibility, (6) and appropriateness. REDCap was also used to present trials of simulated decision-making scenarios. Herein, participants were presented with data from the VFA and asked whether they supported or opposed the (simulated) job match. These responses were then compared with the algorithm. If the algorithm matched the response, the trial was considered a success. If the algorithm did not match the response, the trial was considered a failure. We will use univariate statistical procedures to analyze the quantitative data arising from the survey of VFA clinical utility and mean and frequency counts for participants’ demographic information. VFA accuracy in simulated decision-making scenarios will be analyzed using contingency analysis.

RESULTS: Pilot studies and preliminaries analyses of clinical utility data indicate that >80% of participants agreed that VFA items were relevant, comprehensible, and clinically useful. Participants commented on needed improvements and suggested changes within open ended narratives that will be analyzed separately. Algorithm accuracy was 79.8%. The sensitivity of the algorithm was 89% and the specificity was 66.2%. The false negative rate was 11% and the false positive rate was 33.8%. Almost all participants responded that they trust the VFA algorithm.

CONCLUSION: These results demonstrate the strong clinical utility and basic logic of the VFA. The VFA’s dual focus on worker abilities and job demands is unique and is especially suited to those with intellectual and developmental disabilities in transition to employment.

IMPACT STATEMENT: There are more than 9,000 registered users of VocFit.com, representing all 50 states and 10 countries. These users have completed the VFA-Worker more than 25,000 times and used these data to generate employment reports for approximately 15,000 transition-aged youth with disabilities. Practitioners may use the VFA to support transition planning teams seeking age-appropriate transition assessment that guide pre-employment training and services.

References

Persch, A. C., Cleary, D. S., Rutkowski, S., Malone, H. C., Darragh, A. R., & Case-Smith, J. D. (2015). Current practices in job matching for individuals with intellectual and developmental disabilities. Journal of Vocational Rehabilitation, 43(3).

Persch, A. C., Gugiu, P. C., Onate, J. A., & Cleary, D. S. (2015). Development and psychometric evaluation of the Vocational Fit Assessment. American Journal of Occupational Therapy, 69(6).

Persch, A. C. (2014). The model of systematic job matching (Unpublished doctoral dissertation). The Ohio State University, Columbus.