Using Keyform Maps to Measure Confidence & Competency Ratios to Promote OT Student Success

Understanding competency and confidence in terms of its implications on clinical experiences in a variety of medical professions has proven to be a useful tool to increase student success and client safety (Gottlieb et al., 2021). Assessing student confidence to competence ratios (CCR) has been found to be helpful in identifying overconfident or overcautious behaviors in medical education. Overconfidence can lead to overly aggressive and incorrect treatments while underconfidence may lead to more conservative treatments that are not at a client’s just right challenge (Gottlieb et al., 2021). This phenomenon is supported by the Dunning-Kruger effect which states that individuals often believe that they are more knowledgeable than they truly are and highlights the need for accurate measures of competence and confidence to promote student success and positive client outcomes (McIntosh et al., 2019). The purpose of this research was to utilize an evidence-based keyform ability map of a common occupational therapy assessment (GG Self Care Scale) to measure occupational therapy students’ confidence with task analysis of assessment items to determine easy and hard items and then compare it to their skills accuracy (competence) when ranking the 7 items in an easy to hard hierarchical order. The use of keyform maps of common OT assessments is helpful for OT student use during the evaluation and intervention process because they reveal the just-right challenge for the client (Sears & Egan, 2022). A keyform map is an evidence-based tool, derived from Rasch analyzed measures, that reveals the relationship of a client to the test item challenges revealing a task hierarchy of easy to hard test items. The relationship of the items to the client helps to personalize the OT intervention plan (Sears & Egan, 2022). Keyform maps can be used to measure students’ confidence in their knowledge versus competence of knowledge by measuring students’ ability to task analyze assessment items. During the third semester of their OT program, 38 MSOT students were surveyed and asked to report their confidence in ranking the GG self-care assessment items in the hierarchical order on a scale from 0-to 7. After ranking their confidence, students were prompted to rank the 7 self-care tasks in hierarchical order from easiest to hardest skills. The GG Self-Care Scale keyform map was used to score student hierarchy ranking accuracy. Researchers compared confidence rankings and self-care tasks hierarchy accuracy to determine individual confidence to competence ratios. The GG Self-Care assessment items include oral hygiene, eating, toilet hygiene, upper body dressing, lower body dressing,*shower/bathing self, and putting on and taking off footwear. *To minimize confusion washing the upper body was not included as a separate item. The quantitative data showed that 83.8% of students displayed overconfidence ratings when compared to competence accuracy relative to ranking the GG Self-Care assessment items. 5.4 % of students were over-cautious and 10.8% of students were able to accurately match their skills confidence to their skills accuracy for understanding assessment item difficulty. Professional confidence appears to underpin competence, both of which are linked to professional identity, and for this reason, professional confidence relative to skills competence may need to be considered as an important component of occupational therapy education to support student success and progression into level II fieldwork and entry-level practice (Holland et al., 2012).

References

Gottlieb, M., Chan, T. M., Zaver, F., & Ellaway, R. (2021). Confidence-competence alignment and the role of self-confidence in medical education: A conceptual review. Medical Education, https://doi.org/10.1111/medu.14592

Holland, Kathy, Middleton, Lyn, & Uys, Leana. (2012). The sources of professional confidence in occupational therapy students. South African Journal of Occupational Therapy, 42(3), 19-25. Retrieved October 13, 2021 from http://www.scielo.org.za/pdf/sajot/v42n3/05.pdf

McIntosh, R. D., Fowler, E. A., Lyu, T., & Della Sala, S. (2019). Wise up: Clarifying the role of metacognition in the dunning-kruger effect. Journal of Experimental Psychology. General, 148(11), 1882-1897. https://doi.org/10.1037/xge0000579

Sears, C., & Egan, B. (2022). A Qualitative Study on Occupational Therapy Students’ Perceptions of Using Keyform Maps. Journal of Education and Training Studies, 10(2), 29-34. https://doi.org/10.11114/jets.v10i2.5430