Abstract
Date Presented 4/8/2016
The Vocational Fit Assessment comparative algorithm is capable of producing reliable, clinically sound, and useful decision support products that enhance the practices of professionals who support the transition to adulthood.
Primary Author and Speaker: Andrew Persch
Additional Author and Speaker: Dennis Cleary
PURPOSE: To determine the accuracy of the Vocational Fit Assessment (VFA) decision support system for job matching for individuals with developmental and intellectual disabilities
RATIONALE: The VFA was developed in an effort to operationalize the process of job matching by taking into consideration an individual’s abilities and the demands of the job. Individual abilities are assessed using a 3-point ordinal scale (2 = high ability, 1 = some ability, 0 = low ability). Similarly, job demands are assessed using a 3-point ordinal scale (2 = high demand, 1 = some demand, 0 = low demand).
Traditionally, a professional must evaluate each individual combination of abilities and demands, each and every time they engage in the job matching process. A novel decision support system was developed to make evaluative judgments for each of the nine possible combinations of abilities and demands. The use of clinical reasoning to evaluate these data electronically enables faster and more consistent processing, effectively increasing the reliability and efficiency of the job matching process.
DESIGN: Prospective, cross-sectional, simulation study
PARTICIPANTS: Key stakeholders (e.g., teachers, related service providers) involved in the job matching process for individuals with disabilities
METHOD: Studies were simulated decision making scenarios. Participants were presented with single data points from the VFA–J and from the VFA–W and asked to make an evaluative judgment. That is, given these data alone, would they choose to support or oppose a job match? Responses that aligned with the comparative algorithm were recorded as successful trials. Responses that did not support the comparative algorithm were recorded as trial failures.
ANALYSIS: Percentage correct, one-sample test for binomial proportions, and binary logistic regression
RESULTS: Study 1 resulted in a total of 335 simulated trials, 246 of which supported the comparative algorithm, a 73.4% success rate. Study 2, which consisted of 185 trials, resulted in a 100% success rate. Study 3 resulted in 291 successful trials out of 360 opportunities, an 80.8% success rate.
DISCUSSION: Examination of Study 1 algorithm failures revealed that 81 of 89 failed trials occurred when job demands were high and individuals demonstrate some. Further analysis of this response pattern suggests that respondents gave preference to the demands of the job rather than to the abilities of the individual.
The VFA comparative algorithm does just the opposite. When an individual demonstrates some ability and a job has high demand, then the match between abilities and demands may be improved through intervention. This could include direct instruction to develop individual abilities, workplace accommodations, and/or modifications of the environment. The success rate was 100% at the edges (i.e., pros, cons) of the algorithm. Taken together, the results of these studies strongly support the basic logic of the VFA decision support algorithm.
IMPACT STATEMENT: The validation of the VFA comparative algorithm presented here demonstrates that data on individual abilities and job demands can be integrated in reliable and reproducible manner. These findings demonstrate that clinical reasoning decisions can be systematized in ways that improve efficiency while maintaining clinical utility.
