Skill Observation

We refer to skill observation assessments as those that observe and assess student psychomotor skills. Many of these recommendations are geared toward observation of clinical skills, however, the principles are applicable to observation of multiple skill types.

Expand all

Advantages and challenges of skill observation assessment

Advantages of skill observation assessment

  • Allows assessing students on contextualized, authentic tasks 
  • Allows for providing immediate feedback to students
  • Identifies concrete areas in which students excel and need to improve

Challenges of skill observation assessment

  • Can be time-consuming to administer
  • Can be resource-intensive if multiple assessors are needed
  • If timed, recognize that timed assessments in general add stress unrelated to student’s mastery of the material

Framework for assessing skills or competencies

Many complex psychomotor skills also include student competence in the cognitive and affective domains. In the health professions, skill observation often involves referring to assessment aims as competencies, with students progressing from novice to expert across a program. A framework for assessing skills or competencies is known as Miller’s pyramid (Violato, 2019; Pangaro, 2013) with examples of constructively aligned assessments for each level.

pyramid shape with novice at the bottom and expert at the top. Starting with knows - lectures and textbooks, Assessed by application-based questions; Knows how - Discussion in lectures/tutorials, Assessed by application-based questions; Shows How - Assessed by Objective Structural Clinical Examinations (OSCEs), Role plays in classrooms; Does - Assessed in a workplace environment, Entrustable Professional Activities (EPAs)
Miller’s Pyramid model adapted from Bajis, 2020.

Creating a skill observation assessment

Adapted from Kogan, 2017:

  • Make assessments as authentic as possible – simulation, role-play, case-study, standardized patients or ideally in actual clinical environments.
  • Ensure that standards are clear to students by constructing a checklist or rubric and sharing it with them ahead of time.
  • Recognize that cognitive and implicit bias and impression formation can influence inferences drawn during the observation.
  • Observe the skill longitudinally, if possible, to facilitate learners’ integration of feedback.
  • Include descriptive comments in addition to a quantitative score. 
  • Use existing tools that have been validated, when possible, rather than creating a new tool for the observation.
  • When using multiple observers, use common standards and train observers in conducting observations.

Reference: Kogan, J.R., Hatala, R., Hauer, K.E. et al. Guidelines: The do’s, don’ts and don’t knows of direct observation of clinical skills in medical education. Perspect Med Educ 6, 286–305 (2017).

Resource document: Bloom’s Psychomotor Domains for suggested verbs and examples to use with skill observation.

Preparing students for a skill observation assessment

Adapted from Kogan, 2017:

  • Make the goals, expectations, standards, and consequences of the assessment clear ahead of time. 
  • Provide opportunities for practice with feedback by using formative skill observation activities.
  • Establish a culture that invites learners to practice authentically and welcome feedback.
  • Consider having students practice and provide peer feedback to each other guided by the scoring checklist or rubrics.

Administering a skill observation assessment

  • Provide students with clear instructions regarding the conditions of the assessment.
  • Focus feedback on observable behaviors.
  • Observe the student without interrupting the encounter.
  • Provide students with feedback on their performance as soon as feasible.
  • If using multiple observers, provide standardization information and training so that observers are using the same standards. 

Grading a skill observation assessment

  • Create and use a checklist or rubric that you provide to students ahead of time.
  • Frame feedback with a future orientation. Rather than stating that a student is not competent in a skill, state that the student is not competent yet.
  • Consider allowing retests for full or partial points.
  • Recognize that cognitive bias, impression formation, and implicit bias can influence your inferences drawn during the observation.