Unblinding: Difference between revisions
David notMD (talk | contribs) corrected duplicate use of same ref Bello 2014 |
David notMD (talk | contribs) →Premature unblinding: ref fix |
||
Line 12: | Line 12: | ||
Premature unblinding may also occur when a participant infers from experimental conditions information that has been masked to them. A common cause for unblinding is the presence of side effects (or effects) in the treatment group. In pharmacological trials, premature unblinding can be reduced with the use of an [[active placebo]], which conceals treatment allocation by ensuring the presence of side effects in both groups.<ref>{{cite journal |last1=Double |first1=D. B. |title=Placebo mania. Placebo controlled trials are needed to provide data on effectiveness of active treatment. |journal=BMJ : British Medical Journal |date=19 October 1996 |volume=313 |issue=7063 |pages=1009 |pmid=8892442 |url=https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2352320/ |accessdate=21 April 2019 |issn=0959-8138}}</ref> However, side effects are not the only cause of unblinding; any perceptible difference between the treatment and control groups can contribute to premature unblinding. |
Premature unblinding may also occur when a participant infers from experimental conditions information that has been masked to them. A common cause for unblinding is the presence of side effects (or effects) in the treatment group. In pharmacological trials, premature unblinding can be reduced with the use of an [[active placebo]], which conceals treatment allocation by ensuring the presence of side effects in both groups.<ref>{{cite journal |last1=Double |first1=D. B. |title=Placebo mania. Placebo controlled trials are needed to provide data on effectiveness of active treatment. |journal=BMJ : British Medical Journal |date=19 October 1996 |volume=313 |issue=7063 |pages=1009 |pmid=8892442 |url=https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2352320/ |accessdate=21 April 2019 |issn=0959-8138}}</ref> However, side effects are not the only cause of unblinding; any perceptible difference between the treatment and control groups can contribute to premature unblinding. |
||
A problem arises in the assessment of blinding because asking subjects to guess masked information may prompt them to try to infer that information. Researchers speculate that this may contribute to premature unblinding.<ref>{{cite journal |last1=Rees |first1=Judy R. |last2=Wade |first2=Timothy J. |last3=Levy |first3=Deborah A. |last4=Colford |first4=John M. |last5=Hilton |first5=Joan F. |title=Changes in beliefs identify unblinding in randomized controlled trials: a method to meet CONSORT guidelines |journal=Contemporary Clinical Trials |date=February 2005 |volume=26 |issue=1 |pages=25–37 |doi=10.1016/j.cct.2004.11.020 |url=https://www.sciencedirect.com/science/article/pii/S1551714405000157 |accessdate=24 April 2019}}</ref> Furthermore, it has been reported that some subjects of clinical trials attempt to determine if they have received an active treatment by gathering information on social media and message boards. While researchers counsel patients not to use social media to discuss clinical trials, their accounts are not monitored. This behavior is believed to be a source of unblinding.<ref>{{cite web |last1=Ledford |first1=Heidi |title=A question of Control |url=https://www.nature.com/magazine-assets/d41586-018-07351-8/d41586-018-07351-8.pdf |website=nature.com |publisher=Nature Magazine |accessdate=24 April 2019}}</ref> [[CONSORT]] standards and [[good clinical practice]] guidelines recommend the reporting of all premature unblinding.<ref>{{cite journal |last1=Moher |first1=David |last2=Altman |first2=Douglas G. |last3=Schulz |first3=Kenneth F. |title=CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials |journal=BMJ |date=24 March 2010 |volume=340 |pages=c332 |doi=10.1136/bmj.c332 |url=https://www.bmj.com/content/340/bmj.c332 |accessdate=24 April 2019 |language=en |issn=0959-8138}}</ref><ref>{{cite web |title=E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1) Guidance for Industry |url=https://www.fda.gov/downloads/Drugs/Guidances/UCM464506.pdf#page=24 |website=fda.gov |accessdate=21 April 2019}}</ref> In practice, unintentional unblinding is rarely reported.<ref name=Bello2014>{{cite journal | |
A problem arises in the assessment of blinding because asking subjects to guess masked information may prompt them to try to infer that information. Researchers speculate that this may contribute to premature unblinding.<ref>{{cite journal |last1=Rees |first1=Judy R. |last2=Wade |first2=Timothy J. |last3=Levy |first3=Deborah A. |last4=Colford |first4=John M. |last5=Hilton |first5=Joan F. |title=Changes in beliefs identify unblinding in randomized controlled trials: a method to meet CONSORT guidelines |journal=Contemporary Clinical Trials |date=February 2005 |volume=26 |issue=1 |pages=25–37 |doi=10.1016/j.cct.2004.11.020 |url=https://www.sciencedirect.com/science/article/pii/S1551714405000157 |accessdate=24 April 2019}}</ref> Furthermore, it has been reported that some subjects of clinical trials attempt to determine if they have received an active treatment by gathering information on social media and message boards. While researchers counsel patients not to use social media to discuss clinical trials, their accounts are not monitored. This behavior is believed to be a source of unblinding.<ref>{{cite web |last1=Ledford |first1=Heidi |title=A question of Control |url=https://www.nature.com/magazine-assets/d41586-018-07351-8/d41586-018-07351-8.pdf |website=nature.com |publisher=Nature Magazine |accessdate=24 April 2019}}</ref> [[CONSORT]] standards and [[good clinical practice]] guidelines recommend the reporting of all premature unblinding.<ref>{{cite journal |last1=Moher |first1=David |last2=Altman |first2=Douglas G. |last3=Schulz |first3=Kenneth F. |title=CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials |journal=BMJ |date=24 March 2010 |volume=340 |pages=c332 |doi=10.1136/bmj.c332 |url=https://www.bmj.com/content/340/bmj.c332 |accessdate=24 April 2019 |language=en |issn=0959-8138}}</ref><ref>{{cite web |title=E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1) Guidance for Industry |url=https://www.fda.gov/downloads/Drugs/Guidances/UCM464506.pdf#page=24 |website=fda.gov |accessdate=21 April 2019}}</ref> In practice, unintentional unblinding is rarely reported.<ref name=Bello2014>{{cite journal |vauthors=Bello S, Moustgaard H, Hróbjartsson A |title=The risk of unblinding was infrequently and incompletely reported in 300 randomized clinical trial publications |journal=J Clin Epidemiol |volume=67 |issue=10 |pages=1059–1069 |date=2014 |pmid=24973822 |doi=10.1016/j.jclinepi.2014.05.007 |url=}}</ref> |
||
==In clinical research== |
==In clinical research== |
Revision as of 18:07, 9 May 2019
Unblinding, sometimes called code-break, occurs in a blinded experiment when information becomes available to one from whom it has been masked. Unblinding that occurs before the conclusion of an experiment is a source of bias. In blinded experiments, some degree of premature unblinding is common.[1] When a blind is imperfect, its success is judged on a spectrum with no blind (complete failure of blind) on one end, perfect blinding on the other, and poor or good blinding between. Thus, the common view or studies as blinded or unblinded is an example of a false dichotomy.[2]
Success of blinding is assessed by questioning study participants about information that has been masked to them (e.g. did you receive the drug or placebo?). In a perfectly blinded experiment, the responses should be consistent with no knowledge of the masked information. However, if unblinding has occurred, the responses will indicate the degree of unblinding. Since unblinding cannot be measured directly, but must be inferred from participants' responses, its measured value will depend on the nature of the questions asked. As a result, it is not possible to measure unblinding in a way that is completely objective. Nonetheless, it is still possible to make informed judgments about the quality of a blind. Poorly blinded studies rank above unblinded studies and below well-blinded studies in the hierarchy of evidence.[3] Meta-scientific research has relieved high levels of unblinding in pharmacological trials. In particular, antidepressant trials are poorly blinded. Some researchers believe that the entire observed effect of antidepressant drugs is due to failure of blinding, and that they have no genuine value in the treatment of depression.[4] CONSORT standards and good clinical practice guidelines recommend that all studies assess and report quality of blinding. In practice, this rarely occurs.[5]
Causes
Post-study unblinding
Post-study unblinding is the release of masked data upon completion of a study. In clinical studies, post-study unblinding serves to inform subjects of their treatment allocation. Removing a blind upon completion of a study is never mandatory, but is typically performed as a courtesy to study participants. Unblinding that occurs after the conclusion of a study is not a source of bias, as data analysis is complete at this time.[6]
Premature unblinding
Premature unblinding is any unblinding that occurs before the conclusion of a study. In contrast with post-study unblinding, premature unblinding is a source of bias. A code-break procedure dictates when a subject should be unblinded prematurely. A code-break procedure should only allows for unblinding in cases of emergency. Unblinding that occurs in compliance with code-break procedure is strictly documented and reported.[7]
Premature unblinding may also occur when a participant infers from experimental conditions information that has been masked to them. A common cause for unblinding is the presence of side effects (or effects) in the treatment group. In pharmacological trials, premature unblinding can be reduced with the use of an active placebo, which conceals treatment allocation by ensuring the presence of side effects in both groups.[8] However, side effects are not the only cause of unblinding; any perceptible difference between the treatment and control groups can contribute to premature unblinding.
A problem arises in the assessment of blinding because asking subjects to guess masked information may prompt them to try to infer that information. Researchers speculate that this may contribute to premature unblinding.[9] Furthermore, it has been reported that some subjects of clinical trials attempt to determine if they have received an active treatment by gathering information on social media and message boards. While researchers counsel patients not to use social media to discuss clinical trials, their accounts are not monitored. This behavior is believed to be a source of unblinding.[10] CONSORT standards and good clinical practice guidelines recommend the reporting of all premature unblinding.[11][12] In practice, unintentional unblinding is rarely reported.[5]
In clinical research
Studies of blinded pharmacological trials across widely varying domains find evidence of significant levels of unblinding. Unblinding has been shown to effect both patients and clinicians. This evidence challenges the common assumption that blinding is highly effective in pharmacological trials. Unblinding has also been documented in clinical trials outside of pharmacology.[13]
Pain
A 2018 meta-analysis found that assessment of blinding was reported in only 23 out of 408 randomized controlled trials for chronic pain (5.6%). The study concluded upon analysis of pooled data that the overall quality of the blinding was poor, and the blinding was "not successful." Additionally, both pharmaceutical sponsorship and the presence of side effects were associated with lower rates of reporting assessment of blinding.[14]
Depression
Studies have found evidence of extensive unblinding in antidepressant trials: at least three quarters of patients are able to correctly guess their treatment assignment.[15] Unblinding also occurs in clinicians.[16] Better blinding of patients and clinicians reduces effect size. Researchers concluded that unblinding inflates effect size in antidepressant trials.[17][18] Some researchers believe that antidepressants are not effective for the treatment of depression, and only outperform placebos due to systematic error. These researchers argue that antidepressants are just active placebos.[4]
Zinc
One clinical trial on the efficacy of zinc supplementation in the treatment of the common flue experienced failure of blinding due to the strong metallic after-taste of zinc.[19]
Acupuncture
While the possibility of blinded trials on acupuncture is controversial, a 2003 review of 47 randomized controlled trials found no fewer than four methods of blinding patients to acupuncture treatment: 1) superficial needling of true acupuncture points, 2) use of acupuncture points which are not indicated for the condition being treated, 3) insertion of needles outside of true acupuncture points, and 4) the use of placebo needles which are designed not to penetrate the skin. The authors concluded that there was "no clear association between type of sham intervention used and the results of the trials."[20]
A 2018 study on acupuncture which used needles that did not penetrate the skin as a sham treatment found that 68% of patients and 83% of acupuncturists correctly identified their group allocation. The authors concluded that the blinding had failed, but that more advanced placebos may someday offer the possibility of well-blinded studies in acupuncture.[21]
Significance
Bias due to poor blinding tends to favor the experimental group, resulting in inflated effect size and risk of type I error.[22] Success or failure of blinding is rarely reported or measured; it is implicitly assumed that experiments reported as "blind" are truly blind.[5] Critics have pointed out that without assessment and reporting, there is no way to know if a blind succeed. This shortcoming is especially concerning given that even a small error in blinding can produce a statistically significant result in the absence of any real difference between test groups when a study is sufficiently powered (i.e. statistical significance is not robust to bias). As such, many statistically significant results in randomized controlled trials may be caused by error in blinding.[23] Some researchers have called for mandatory assessments of blinding efficacy in clinical trials.[3]
References
- ^ Bello S, Moustgaard H, Hróbjartsson A (2017). "Unreported formal assessment of unblinding occurred in 4 of 10 randomized clinical trials, unreported loss of blinding in 1 of 10 trials". J Clin Epidemiol. 81: 42–50. doi:10.1016/j.jclinepi.2016.08.002. PMID 27555081.
- ^ Schulz KF, Grimes DA (2002). "Blinding in randomised trials: hiding who got what". Lancet. 359 (9307): 696–700. doi:10.1016/S0140-6736(02)07816-9. PMID 11879884.
- ^ a b Kolahi J, Bang H, Park J (2009). "Towards a proposal for assessment of blinding success in clinical trials: up-to-date review". Community Dent Oral Epidemiol. 37 (6): 477–484. doi:10.1111/j.1600-0528.2009.00494.x. PMC 3044082. PMID 19758415.
- ^ a b Ioannidis, John PA (27 May 2008). "Effectiveness of antidepressants: an evidence myth constructed from a thousand randomized trials?". Philosophy, ethics, and humanities in medicine : PEHM. 3: 14. doi:10.1186/1747-5341-3-14. ISSN 1747-5341. Retrieved 23 April 2019.
{{cite journal}}
: CS1 maint: unflagged free DOI (link) - ^ a b c Bello S, Moustgaard H, Hróbjartsson A (2014). "The risk of unblinding was infrequently and incompletely reported in 300 randomized clinical trial publications". J Clin Epidemiol. 67 (10): 1059–1069. doi:10.1016/j.jclinepi.2014.05.007. PMID 24973822.
- ^ Dinnett EM, Mungall MM, Kent JA, Ronald ES, McIntyre KE, Anderson E, Gaw A (2005). "Unblinding of trial participants to their treatment allocation: lessons from the Prospective Study of Pravastatin in the Elderly at Risk (PROSPER)". Clin Trials. 2 (3): 254–259. doi:10.1191/1740774505cn089oa. PMID 16279148.
- ^ Quittell, Lynne M. (3 October 2018). "The Scientific and Social Implications of Unblinding a Study Subject". The American Journal of Bioethics. 18 (10): 71–73. doi:10.1080/15265161.2018.1513589. ISSN 1526-5161. Retrieved 20 April 2019.
- ^ Double, D. B. (19 October 1996). "Placebo mania. Placebo controlled trials are needed to provide data on effectiveness of active treatment". BMJ : British Medical Journal. 313 (7063): 1009. ISSN 0959-8138. PMID 8892442. Retrieved 21 April 2019.
- ^ Rees, Judy R.; Wade, Timothy J.; Levy, Deborah A.; Colford, John M.; Hilton, Joan F. (February 2005). "Changes in beliefs identify unblinding in randomized controlled trials: a method to meet CONSORT guidelines". Contemporary Clinical Trials. 26 (1): 25–37. doi:10.1016/j.cct.2004.11.020. Retrieved 24 April 2019.
- ^ Ledford, Heidi. "A question of Control" (PDF). nature.com. Nature Magazine. Retrieved 24 April 2019.
- ^ Moher, David; Altman, Douglas G.; Schulz, Kenneth F. (24 March 2010). "CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials". BMJ. 340: c332. doi:10.1136/bmj.c332. ISSN 0959-8138. Retrieved 24 April 2019.
- ^ "E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1) Guidance for Industry" (PDF). fda.gov. Retrieved 21 April 2019.
- ^ JUL 2009, The Pharmaceutical Journal31 (31 July 2009). "An example of problems that arise from clinical trials and how to avoid them". Pharmaceutical Journal. 283: 129–130. Retrieved 24 April 2019.
{{cite journal}}
: CS1 maint: numeric names: authors list (link) - ^ Colagiuri, Ben; Sharpe, Louise; Scott, Amelia (September 2018). "The Blind Leading the Not-So-Blind: A Meta-Analysis of Blinding in Pharmacological Trials for Chronic Pain". The Journal of Pain. doi:10.1016/j.jpain.2018.09.002. ISSN 1526-5900. Retrieved 22 April 2019.
- ^ Perlis, Roy H.; Ostacher, Michael; Fava, Maurizio; Nierenberg, Andrew A.; Sachs, Gary S.; Rosenbaum, Jerrold F. (1 March 2010). "Assuring That Double-Blind Is Blind". American Journal of Psychiatry. 167 (3): 250–252. doi:10.1176/appi.ajp.2009.09060820. ISSN 0002-953X. Retrieved 23 April 2019.
- ^ "Side effects and the "blindability" of clinical drug trials". American Journal of Psychiatry. 149 (12): 1730–1731. December 1992. doi:10.1176/ajp.149.12.1730. ISSN 0002-953X. Retrieved 22 April 2019.
- ^ Hardy, Rebecca; Wessely, Simon; Moncrieff, Joanna (1998). "Meta-analysis of trials comparing antidepressants with active placebos". The British Journal of Psychiatry. 172 (3): 227–231. doi:10.1192/bjp.172.3.227. ISSN 0007-1250. Retrieved 23 April 2019.
- ^ Greenberg, Roger P.; Bornstein, Robert F.; Greenberg, Michael D.; Fisher, Seymour (1992). "A meta-analysis of antidepressant outcome under "blinder" conditions". Journal of Consulting and Clinical Psychology. 60 (5): 664–669. doi:10.1037/0022-006X.60.5.664. Retrieved 22 April 2019.
- ^ Hróbjartsson, A; Boutron, I (12 October 2011). "Blinding in Randomized Clinical Trials: Imposed Impartiality". Clinical Pharmacology & Therapeutics. 90 (5): 732–736. doi:10.1038/clpt.2011.207. ISSN 1532-6535. Retrieved 22 April 2019.
- ^ Dincer, F; Linde, K. (December 2003). "Sham interventions in randomized clinical trials of acupuncture—a review". Complementary Therapies in Medicine. 11 (4): 235–242. doi:10.1016/S0965-2299(03)00124-9. Retrieved 23 April 2019.
- ^ Vase, L; Baram, S; Takakura, N; Takayama, M; Yajima, H; Kawase, A; Schuster, L; Kaptchuk, TJ; Schou, S; Jensen, TS; Zachariae, R; Svensson, P (2015). "Can acupuncture treatment be double-blinded? An evaluation of double-blind acupuncture treatment of postoperative pain". PloS one. 10 (3): e0119612. doi:10.1371/journal.pone.0119612. ISSN 1932-6203. PMID 25747157. Retrieved 23 April 2019.
{{cite journal}}
: CS1 maint: unflagged free DOI (link) - ^ Pandis, Nikolaos (March 2012). "Blinding or masking". American Journal of Orthodontics and Dentofacial Orthopedics. 141 (3): 389–390. doi:10.1016/j.ajodo.2011.10.019. Retrieved 20 April 2019.
- ^ Siegfried, Tom (2010). "Odds are, it's wrong: Science fails to face the shortcomings of statistics". Science News. 177 (7): 26–29. doi:10.1002/scin.5591770721. ISSN 1943-0930. Retrieved 22 April 2019.