A randomised controlled trial of email versus mailed invitation letter in a national longitudinal survey of physicians

Benjamin Harrap; Tamara Taylor; Grant Russell; Anthony Scott

doi:10.1371/journal.pone.0289628

A randomised controlled trial of email versus mailed invitation letter in a national longitudinal survey of physicians

PLoS One. 2023 Aug 22;18(8):e0289628. doi: 10.1371/journal.pone.0289628. eCollection 2023.

Authors

Benjamin Harrap¹, Tamara Taylor², Grant Russell³, Anthony Scott⁴

Affiliations

¹ Centre for Health Policy, Melbourne School of Population and Global Health, The University of Melbourne, Carlton, Victoria, Australia.
² Government and Social Research Division, Big Village, Melbourne, Australia.
³ Department of General Practice, Monash University, Melbourne, Victoria, Australia.
⁴ Centre for Health Economics, Monash University, Caulfield East, Victoria, Australia.

Abstract

Despite their low cost, the use of email invitations to distribute surveys to medical practitioners have been associated with lower response rates. This research compares the difference in response rates from using email approach plus online completion rather than a mailed invitation letter plus a choice of online or paper completion. A parallel randomised controlled trial was conducted during the 11th annual wave of the nationally representative Medicine in Australia: Balancing Employment and Life (MABEL) longitudinal survey of doctors. The control group was invited using a mailed paper letter (including a paper survey plus instructions to complete online) and three mailed paper reminders. The intervention group was approached in the same way apart from the second reminder when they were approached by email only. The primary outcome is the response rate and the statistical analysis was blinded. 18,247 doctors were randomly allocated to the control (9,125) or intervention group (9,127), with 9,108 and 9,107 included in the analysis. Using intention to treat analysis, the response rate in the intervention group was 35.92% compared to 37.59% in the control group, a difference of -1.66 percentage points (95% CI: -3.06 to -0.26). The difference was larger for General Practitioners (-2.76 percentage points, 95% CI: -4.65 to -0.87) compared to other specialists (-0.47 percentage points, 95% CI: -2.53 to 1.60). For those who supplied an email address, the average treatment effect on the treated was higher at -2.63 percentage points (95% CI: -4.50 to -0.75) for all physicians, -3.17 percentage points (95% CI: -5.83 to -0.53) for General Practitioners, and -2.1 percentage points (95% CI: -4.75 to 0.56) for other specialists. For qualified physicians, using email to invite participants to complete a survey leads to lower response rates compared to a mailed letter. Lower response rates need to be traded off with the lower costs of using email rather than mailed letters.

Copyright: © 2023 Harrap et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Publication types

Randomized Controlled Trial
Research Support, Non-U.S. Gov't

MeSH terms

Australia
Control Groups
Electronic Mail*
Employment
General Practitioners*
Humans

Grants and funding

This research used data from the MABEL longitudinal survey of doctors. Funding for MABEL was provided by the National Health and Medical Research Council (2007 to 2016: 454799 and 1019605); the Australian Department of Health and Ageing (2008); Health Workforce Australia (2013); The University of Melbourne, Medibank Better Health Foundation, the NSW Department of Health, and the Victorian Department of Health and Human Services (2017); and the Australian Government Department of Health, the Australian Digital Health Agency, and the Victorian Department of Health and Human Services (2018). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. AS was the Chief Investigator for all above grants but did not receive salary or payments. BH and TT were employed using the above research funding. GR did not receive salary or payments.