We refer to skill observation assessments as those that observe and assess student psychomotor skills. Many of these recommendations are geared toward observation of clinical skills, however, the principles are applicable to observation of multiple skill types.
Advantages and challenges of skill observation assessment
Advantages of skill observation assessment
- Allows assessing students on contextualized, authentic tasks
- Allows for providing immediate feedback to students
- Identifies concrete areas in which students excel and need to improve
Challenges of skill observation assessment
- Can be time-consuming to administer
- Can be resource-intensive if multiple assessors are needed
- If timed, recognize that timed assessments in general add stress unrelated to student’s mastery of the material
Framework for assessing skills or competencies
Many complex psychomotor skills also include student competence in the cognitive and affective domains. In the health professions, skill observation often involves referring to assessment aims as competencies, with students progressing from novice to expert across a program. A framework for assessing skills or competencies is known as Miller’s pyramid (Violato, 2019; Pangaro, 2013) with examples of constructively aligned assessments for each level.
Creating a skill observation assessment
Adapted from Kogan, 2017:
- Make assessments as authentic as possible – simulation, role-play, case-study, standardized patients or ideally in actual clinical environments.
- Ensure that standards are clear to students by constructing a checklist or rubric and sharing it with them ahead of time.
- Recognize that cognitive and implicit bias and impression formation can influence inferences drawn during the observation.
- Observe the skill longitudinally, if possible, to facilitate learners’ integration of feedback.
- Include descriptive comments in addition to a quantitative score.
- Use existing tools that have been validated, when possible, rather than creating a new tool for the observation.
- When using multiple observers, use common standards and train observers in conducting observations.
Reference: Kogan, J.R., Hatala, R., Hauer, K.E. et al. Guidelines: The do’s, don’ts and don’t knows of direct observation of clinical skills in medical education. Perspect Med Educ 6, 286–305 (2017). https://doi.org/10.1007/s40037-017-0376-7
Resource document: Bloom’s Psychomotor Domains for suggested verbs and examples to use with skill observation.
Preparing students for a skill observation assessment
Adapted from Kogan, 2017:
- Make the goals, expectations, standards, and consequences of the assessment clear ahead of time.
- Provide opportunities for practice with feedback by using formative skill observation activities.
- Establish a culture that invites learners to practice authentically and welcome feedback.
- Consider having students practice and provide peer feedback to each other guided by the scoring checklist or rubrics.
Administering a skill observation assessment
- Provide students with clear instructions regarding the conditions of the assessment.
- Focus feedback on observable behaviors.
- Observe the student without interrupting the encounter.
- Provide students with feedback on their performance as soon as feasible.
- If using multiple observers, provide standardization information and training so that observers are using the same standards.
Grading a skill observation assessment
- Create and use a checklist or rubric that you provide to students ahead of time.
- Frame feedback with a future orientation. Rather than stating that a student is not competent in a skill, state that the student is not competent yet.
- Consider allowing retests for full or partial points.
- Recognize that cognitive bias, impression formation, and implicit bias can influence your inferences drawn during the observation.