Assessing Education: What We Do and Why We Do It
“Medical education is a lifelong process embracing premedical experience, undergraduate education, general clinical training, specialist or vocational training, subspecialty training, and continuing medical education.” [BMJ. 1999 May 8; 318(7193): 1280].” With the need to teach an increasing body of knowledge and skill, and give more training in a fixed amount of time, medical education—undergraduate, graduate, and continuing—is evolving. As Dr. Waguespack’s column states, traditional “continuing medical education” is now more correctly described as “continuing professional development (CPD),” involving not just the acquisition of new knowledge and facts, but also the integration of that knowledge into practice, the changing of behavior, and the documenting of the improvements in patient care and outcomes. Our educational methods should change in concert with our needs; thus it is essential to strengthen the influence and effectiveness of the AAO-HNS Foundation’s CPD programming. With that in mind, how do we assess what we need? Where do we begin to effectively support medical students, GME, CPD, otolaryngologists, other physicians, physician assistants, nurses, audiologists, SLPs, and others who need otolaryngology education programming? Gaps: What Do We Mean? The answer requires a robust system of ongoing measurement of learner needs in regard to education and performance improvement. One of the most important recent changes in our requirements for accreditation by the ACCME is identifying gaps in care and ensuring that our educational programming addresses these gaps. Gaps in clinical care can include gaps in knowledge, in dissemination of knowledge, in implementation in practice, in documentation, and in reporting and comparing/benchmarking improvement. Therefore, even with new emphasis on education and changing practice behavior through evidence-based guidelines and performance measures, one can easily see that knowledge-based medical education still remains a foundational component, and some form of didactic and instructional programming will always be needed. A strategic issue to consolidate and enhance the otolaryngology practice gap analysis and needs assessment process was included in the 2013 AAO-HNS/F strategic plan. Beginning in late 2012 and continuing into 2013, Foundation Education Leadership embarked on an initiative to collect gap analysis and needs assessment data for the Foundation’s CPD function. Gap Analysis and Needs Assessment In accordance with our CPD Guidelines the Foundation utilizes a planning process that links identified professional practice gaps and education needs with expected outcomes (knowledge, competence, or performance) in its provision of CPD activities. This analysis data is incorporated into the planning of all education activities. Professional practice gaps are identified through: Environmental scans of research reports, Peer-reviewed journals, Healthcare quality assurance data, Research abstracts, Activity evaluation summaries and member surveys, ABMS/ABOto certification and maintenance of certification requirements, and Evidence-based clinical guidelines development. Various member needs assessment surveys are conducted on an ongoing basis in an effort to gauge whether existing education and knowledge products are meeting the learning needs of our members. They include: Education Needs Survey with Education Committee members (biannually). Course Participant Survey on quality and effectiveness of education activities (biannually). Overall Membership Survey assessing their perceptions of the Foundation’s CPD program and how it can better meet their education needs (biannually). Target Audience Focus Groups. The Gap Analysis and Needs Assessment Initiative Process requires first that we survey and conduct a SWOT (strengths, weaknesses, opportunities, and threats) analysis with the members of the Foundation’s Education Steering Committee and specialty education committees—roughly 250 individuals via online survey. Second, we gather and analyze existing member/participant usage and evaluation data from current education activities, demographic data, and web-based analytics. Information collected as part of the 2012 Voice of the Member Survey, 2012 Section for Residents and Fellow Survey, and the recent Residency Program Directors Survey along with data collected by Research and Quality Improvement staff used to develop clinical practice guidelines and to establish research priorities will be reviewed and analyzed. Third, we distribute a membership-wide education needs survey to assess perceived practice gaps and education needs in order to plan more focused education activities. Otolaryngology education and the Academy/Foundation programming continue to evolve to provide comprehensive, traditional and point-of-care, mobile, real time access to the information, skills, techniques, and processes that will advance quality and safety, and ensure that we continue to provide the best possible care to our patients.
“Medical education is a lifelong process embracing premedical experience, undergraduate education, general clinical training, specialist or vocational training, subspecialty training, and continuing medical education.” [BMJ. 1999 May 8; 318(7193): 1280].”
With the need to teach an increasing body of knowledge and skill, and give more training in a fixed amount of time, medical education—undergraduate, graduate, and continuing—is evolving. As Dr. Waguespack’s column states, traditional “continuing medical education” is now more correctly described as “continuing professional development (CPD),” involving not just the acquisition of new knowledge and facts, but also the integration of that knowledge into practice, the changing of behavior, and the documenting of the improvements in patient care and outcomes. Our educational methods should change in concert with our needs; thus it is essential to strengthen the influence and effectiveness of the AAO-HNS Foundation’s CPD programming.
With that in mind, how do we assess what we need? Where do we begin to effectively support medical students, GME, CPD, otolaryngologists, other physicians, physician assistants, nurses, audiologists, SLPs, and others who need otolaryngology education programming?
Gaps: What Do We Mean?
The answer requires a robust system of ongoing measurement of learner needs in regard to education and performance improvement. One of the most important recent changes in our requirements for accreditation by the ACCME is identifying gaps in care and ensuring that our educational programming addresses these gaps. Gaps in clinical care can include gaps in knowledge, in dissemination of knowledge, in implementation in practice, in documentation, and in reporting and comparing/benchmarking improvement. Therefore, even with new emphasis on education and changing practice behavior through evidence-based guidelines and performance measures, one can easily see that knowledge-based medical education still remains a foundational component, and some form of didactic and instructional programming will always be needed.
A strategic issue to consolidate and enhance the otolaryngology practice gap analysis and needs assessment process was included in the 2013 AAO-HNS/F strategic plan. Beginning in late 2012 and continuing into 2013, Foundation Education Leadership embarked on an initiative to collect gap analysis and needs assessment data for the Foundation’s CPD function.
Gap Analysis and Needs Assessment
In accordance with our CPD Guidelines the Foundation utilizes a planning process that links identified professional practice gaps and education needs with expected outcomes (knowledge, competence, or performance) in its provision of CPD activities. This analysis data is incorporated into the planning of all education activities.
Professional practice gaps are identified through:
- Environmental scans of research reports,
- Peer-reviewed journals,
- Healthcare quality assurance data,
- Research abstracts,
- Activity evaluation summaries and member surveys,
- ABMS/ABOto certification and maintenance of certification requirements, and
- Evidence-based clinical guidelines development.
Various member needs assessment surveys are conducted on an ongoing basis in an effort to gauge whether existing education and knowledge products are meeting the learning needs of our members.
They include:
- Education Needs Survey with Education Committee members (biannually).
- Course Participant Survey on quality and effectiveness of education activities (biannually).
- Overall Membership Survey assessing their perceptions of the Foundation’s CPD program and how it can better meet their education needs (biannually).
- Target Audience Focus Groups.
The Gap Analysis and Needs Assessment Initiative Process requires first that we survey and conduct a SWOT (strengths, weaknesses, opportunities, and threats) analysis with the members of the Foundation’s Education Steering Committee and specialty education committees—roughly 250 individuals via online survey. Second, we gather and analyze existing member/participant usage and evaluation data from current education activities, demographic data, and web-based analytics. Information collected as part of the 2012 Voice of the Member Survey, 2012 Section for Residents and Fellow Survey, and the recent Residency Program Directors Survey along with data collected by Research and Quality Improvement staff used to develop clinical practice guidelines and to establish research priorities will be reviewed and analyzed. Third, we distribute a membership-wide education needs survey to assess perceived practice gaps and education needs in order to plan more focused education activities.
Otolaryngology education and the Academy/Foundation programming continue to evolve to provide comprehensive, traditional and point-of-care, mobile, real time access to the information, skills, techniques, and processes that will advance quality and safety, and ensure that we continue to provide the best possible care to our patients.