Survey research in anesthesiology: a field guide to interpretation

Background: Survey research, indispensable for assessing subjective outcomes in anesthesiology, can nonetheless be challenging to undertake and interpret.

Objective: To present a user-friendly guide for the appraisal of survey-derived evidence, and to apply it to published survey research in the anesthesia literature.

Methods: Synthesizing published expert guidance regarding methodology and reporting, we discuss five essential criteria (with subcomponents) for evaluating survey research: (1) relevance of survey outcome to research objective, (2) trustworthiness of the instrument (testing/validation, availability), (3) collecting information well (sampling, administration), (4) representativeness (response rate), and (5) guidance towards interpretation of survey findings (generalizability, interpretation of numerical outcomes). These criteria were subsequently applied by two independent assessors to original research articles reporting survey findings, published in the five highest impact general anesthesia journals ('Anaesthesia', 'Anesthesia & Analgesia', 'Anesthesiology', 'British Journal of Anaesthesia' and 'European Journal of Anaesthesiology') between July 01, 2016, and December 31, 2017, which were identified using a prespecified PubMed search strategy.

Results: Among 1107 original articles published, we identified 97 reporting survey research either employing novel survey instruments (58%), established surveys (30%), or sets of single-item scores (12%). The extent to which reader-oriented benchmarks were achieved varied by component and between survey types. Results were particularly mixed for validation (mentioned for 41% of novel and 86% of established surveys) and discussion of generalizability (59% of novel survey reports, 45% of established surveys, and 17% of sets of single-item scores).

Conclusion: Survey research is not uncommon in anesthesiology, frequently employs novel survey instruments, and demonstrates mixed results in terms of transparency and interpretability. We provide readers with a practical framework for critical interpretation of survey-derived outcomes.