Published: November 4, 2015

AAO-HNSF publishes methodology for developing clinical consensus statements

Since 2010, the American Academy of Otolaryngology—Head and Neck Surgery Foundation (AAO-HNSF) has published five clinical consensus statements (CCSs) in Otolaryngology–Head and Neck Surgery, including: Diagnosis and Management of Nasal Valve Compromise (2010); CT Imaging Indications for Paranasal Sinus Disease (2012); Tracheostomy Care (2012); Pediatric Chronic Rhinosinusitis (2014); and this month Septoplasty with or without Inferior Turbinoplasty (2015). The CCS development manual supplement also appears in this month’s issue. The manual describes the methodology used by the AAO-HNSF to promote rapid and consistent development of CCSs when the evidence is lacking for development of a clinical practice guideline (CPG). A CCS integrates structured expert opinions with the existing literature to try and provide some clarification on points that are quality improvement opportunities related to a particular topic. In contrast to CPGs, which are based primarily on high-level evidence, clinical consensus statements are more applicable to situations where evidence is limited or lacking, yet there are still opportunities to reduce uncertainty and improve quality of care. Much like the AAO-HNSF Clinical Practice Guideline Development Manual, Third Edition: A Quality-Driven Approach for Translating Evidence into Action, which has now been cited 119 times, the goal of the Clinical Consensus Statement Development Manual is to provide readers with the detailed methodology used by the AAO-HNSF to develop its clinical consensus statements. Publication of the manual facilitates transparency of the AAO-HNSF process and also provides a tool for other organizations to replicate our process. In summary, the AAO-HNSF utilizes a modified Delphi Survey Method to develop its CCSs. The Delphi Method involves using multiple anonymous surveys to assess for objective consensus within an expert panel. This rigorous and standardized approach minimized bias and facilitated content expert consensus. While the CCS development manual contains specific practices relevant to the AAO-HNSF, we believe that the principles explained therein will be a valuable tool for our Members, the subspecialty societies, and to external organizations as well. Comparison of key characteristics of consensus statements vs. guidelines* Characteristic Clinical consensus statement Clinical practice guideline Primary output Statements of fact based on best evidence and expert consensus Recommendations for action based on best evidence and explicit consideration of benefits, harms, values, and preferences Level of evidence Observational studies and expert consensus; higher levels of evidence when available Systematic reviews and randomized controlled trials; lower level evidence as needed for research gaps Size of development group 8 to 10; possibly more 15 to 20 Composition of development group Otolaryngologists; content experts a majority; may include other disciplines as needed Multidisciplinary, including consumers; content experts a minority; includes all stakeholders in the target audience Perspective of development group member Member serves as a content expert based on individual knowledge and experience Member advocates for the discipline or constituency they were appointed to represent Time frame 6 to 8 months 12 to 18 months Meeting venues Conference calls and electronic mail In-person meetings, conference calls, and electronic mail External review Limited review by relevant stakeholders Extensive review by all stakeholders, including open public comment *From the AAO-HNSF Clinical Consensus Statement Development Manual (2015). (In press)


Since 2010, the American Academy of Otolaryngology—Head and Neck Surgery Foundation (AAO-HNSF) has published five clinical consensus statements (CCSs) in Otolaryngology–Head and Neck Surgery, including: Diagnosis and Management of Nasal Valve Compromise (2010); CT Imaging Indications for Paranasal Sinus Disease (2012); Tracheostomy Care (2012); Pediatric Chronic Rhinosinusitis (2014); and this month Septoplasty with or without Inferior Turbinoplasty (2015).

The CCS development manual supplement also appears in this month’s issue. The manual describes the methodology used by the AAO-HNSF to promote rapid and consistent development of CCSs when the evidence is lacking for development of a clinical practice guideline (CPG). A CCS integrates structured expert opinions with the existing literature to try and provide some clarification on points that are quality improvement opportunities related to a particular topic. In contrast to CPGs, which are based primarily on high-level evidence, clinical consensus statements are more applicable to situations where evidence is limited or lacking, yet there are still opportunities to reduce uncertainty and improve quality of care.

Much like the AAO-HNSF Clinical Practice Guideline Development Manual, Third Edition: A Quality-Driven Approach for Translating Evidence into Action, which has now been cited 119 times, the goal of the Clinical Consensus Statement Development Manual is to provide readers with the detailed methodology used by the AAO-HNSF to develop its clinical consensus statements. Publication of the manual facilitates transparency of the AAO-HNSF process and also provides a tool for other organizations to replicate our process.

In summary, the AAO-HNSF utilizes a modified Delphi Survey Method to develop its CCSs. The Delphi Method involves using multiple anonymous surveys to assess for objective consensus within an expert panel. This rigorous and standardized approach minimized bias and facilitated content expert consensus.

While the CCS development manual contains specific practices relevant to the AAO-HNSF, we believe that the principles explained therein will be a valuable tool for our Members, the subspecialty societies, and to external organizations as well.

Comparison of key characteristics of consensus statements vs. guidelines*

Characteristic Clinical consensus statement Clinical practice guideline
Primary output Statements of fact based on best evidence and expert consensus Recommendations for action based on best evidence and explicit consideration of benefits, harms, values, and preferences
Level of evidence Observational studies and expert consensus; higher levels of evidence when available Systematic reviews and randomized controlled trials; lower level evidence as needed for research gaps
Size of development group 8 to 10; possibly more 15 to 20
Composition of development group Otolaryngologists; content experts a majority; may include other disciplines as needed Multidisciplinary, including consumers; content experts a minority; includes all stakeholders in the target audience
Perspective of development group member Member serves as a content expert based on individual knowledge and experience Member advocates for the discipline or constituency they were appointed to represent
Time frame 6 to 8 months 12 to 18 months
Meeting venues Conference calls and electronic mail In-person meetings, conference calls, and electronic mail
External review Limited review by relevant stakeholders Extensive review by all stakeholders, including open public comment
*From the AAO-HNSF Clinical Consensus Statement Development Manual (2015). (In press)

 

 

 


More from November 2015 - Vol. 34, No. 10