History
The Innovation Agenda + The Expert Advisory Panel
An Explanation
CAPR Evaluation Services are supported by the Evaluation Services Committee (ESC) – a Board Committee – and its sub-committees. The mandate of the ESC is to provide proactive, strategic advice and makes recommendations to the CAPR Board of Directors (Board) on matters pertaining to the Evaluation Services’ Credentialling and Examination programs. In accordance with ISO standard 17024, and the Standards for the Accreditation of Certification Programs of the National Commission for Certifying Agencies, the ESC provides oversight for the Credentialling and Examination Programs and assures the effective delivery of those services.
2016-2020
While CAPR regularly follows international best practice standards for high-stakes licensure exams and undertakes external expert reviews of its program, in 2016, the Board sought to strengthen CAPR’s work from “meeting” best practice standards to “exceeding” them.
The overarching goal of entry-to-practice competency assessment is public protection. CAPR’s Strategic Framework 2018-2023 includes a Strategic Objective to Ensure Excellence in Evaluation Services, by
- developing and implementing a leading-edge competency assessment through innovation and evidence-based policy making, and
- continuing to be leaders in education credential assessment
As such, the Board, in accordance with the strategic plan, directed CAPR staff to develop an innovation agenda starting in the third year of the strategic plan.
Work on the innovation agenda began in earnest in 2019 with a broad Stakeholder Visioning Day. At its February 2020 meeting, the CAPR Board of Directors approved key priorities for the innovation agenda and staff prepared for implementation. At the time, potential innovations included:
With data gathering to include:
And a timeline somewhat like:
But…
…and CAPR’s attention turned elsewhere.
2020-2021
While the implementation of the innovation agenda was interrupted by the declaration of the global pandemic just two weeks after the February 2020 Board meeting, CAPR did continue with key elements of the innovation plan as best as possible. Innovations included:
- the inclusion of a remote proctoring delivery option for candidates challenging the Written Component of the PCE
- the reduction of the number of stations in the Clinical Component (including the elimination of the written questions of the Objective Structured Clinical Examination or OSCE)
- exploration of a virtual delivery format of the Clinical Component (including electronic scoring), and more frequent exam administrations
In April 2021, in agreement with Board direction, CAPR returned in earnest to its interrupted plan to consider major innovations to its evaluation services. But the pandemic has changed the world in ways that were not foreseen, and with those changes, assessment organizations have changed. Innovation has been seen in education, regulation, accreditation, credentialling, evaluation, assessment, and others. CAPR must now re-envision its future, in alignment with the new realities of the post-pandemic world. CAPR innovations must be future-oriented, but also grounded in evidence and risk-stratified.
The Expert Advisory Panel
The Expert Advisory Panel (the Panel) includes nine members with expertise in measurement, assessment, and competence from various contexts, and who are innovative and forward-thinking.
The Panel is an ad-hoc committee of the Evaluation Services Committee. The members will lead the exploratory background work and preparatory research towards re-envisioning CAPR’s work – our evaluation services. The Panel will develop recommendations to present to the CAPR Board of Directors in 2023 (through the Evaluation Services Committee), and these recommendations will be the foundation of CAPR’s innovations for the future. Members are responsible for using their collective expertise to identify, gather, review, and discuss input from a broad range of sources. We’ve asked them to:
- Invoke an innovative thinking mindset to entry-to-practice competency assessment for physiotherapists in preparation for licensure.
- Contribute to the identification and scope of background research necessary to inform their work (e.g. environmental scans, surveys, focus groups, literature review, etc.). External consultant(s) will be engaged to conduct the necessary background research.
- Apply a hub and spoke model to gather and review input from a broad range of stakeholders.
- Seek input from Fairness Commissioners across Canada as appropriate/necessary.
- Participate in collaborative discussions to formulate recommendation(s) about the future of the credentialling and examination services provided by CAPR, taking into consideration stakeholder input, and applying their collective expertise in measurement, assessment, and competence.
- Contribute to validating the recommendation(s) with key stakeholders prior to presentation to the Evaluation Services Committee and the Board of Directors.
- Contribute to an implementation plan for launch beginning after Board approval of the recommendation(s).
The Evaluation Services Committee will act as the steering committee for the current project. The lines of accountability of various groups involved in the work of the Expert Advisory Panel are described below.
CAPR Board of Directors: approves project funding and final recommendation(s) generated by the project (the future of CAPR evaluation services).
CAPR Evaluation Services Committee: acts as the steering committee for the project – provision of guidance and input about the project to the Expert Advisory Panel, review/agreement of final recommendation(s), and presentation of the recommendation(s) for approval to Board of Directors. The Chair of the Evaluation Services Committee will act as liaison between the Expert Advisory Panel (see below) and the Board of Directors of CAPR.
Expert Advisory Panel: acts as the working committee to re-envision CAPR competency assessment based on data gathered through a broad consultations process (primary/secondary research including surveys, focus groups, interviews, environmental scan and peer-reviewed literature review). The Chair of the Expert Advisory Panel will work hand in hand with the consultants, will assist in facilitating Panel meetings, provide input into meeting agendas, and liaise if necessary with the Board of Directors.
CAPR Staff: support the work of the Evaluation Services Committee and the Expert Advisory Panel, arrange for translation of materials through the project as required (e.g. communications, survey, invitations to participate). Facilitate communications between all of the above groups (Expert Advisory Panel, Evaluation Services Committee, Board of Directors).