Systematic Review of Measures Used in Pictorial Cigarette Pack Warning Experiments

Background: We sought to describe characteristics and psychometric properties of measures used in pictorial cigarette pack warning experiments and provide recommendations for future studies.

Methods: Our systematic review identified 68 pictorial cigarette pack warning experiments conducted between 2000 and 2016 in 22 countries. Two independent coders coded all studies on study features, including sample characteristics, theoretical framework, and constructs assessed. We also coded measurement characteristics, including construct, number of items, source, reliability, and validity.

Results: We identified 278 measures representing 61 constructs. The most commonly assessed construct categories were warning reactions (62% of studies) and perceived effectiveness (60%). The most commonly used outcomes were affective reactions (35%), perceived likelihood of harm (22%), intention to quit smoking (22%), perceptions that warnings motivate people to quit smoking (18%), and credibility (16%). Only 4 studies assessed smoking behavior. More than half (54%) of all measures were single items. For multi-item measures, studies reported reliability data 68% of the time (mean α = 0.88, range α = 0.68-0.98). Studies reported sources of measures only 33% of the time and rarely reported validity data. Of 68 studies, 37 (54%) mentioned a theory as informing the study.

Conclusions: Our review found great variability in constructs and measures used to evaluate the impact of cigarette pack pictorial warnings. Many measures were single items with unknown psychometric properties. Recommendations for future studies include a greater emphasis on theoretical models that inform measurement, use of reliable and validated (preferably multi-item) measures, and better reporting of measure sources.

Implications: Robust and consistent measurement is important for building a strong, cumulative evidence base to support pictorial cigarette pack warning policies. This systematic review of experimental studies of pictorial cigarette warnings demonstrates the need for standardized, theory-based measures.