Reliability of online self-report versions of the Australian Treatment Outcome Profile (ATOP) and Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) among people in residential treatment for substance use problems

Background: Online self-report measures are resource-efficient and widely used for monitoring substance use, yet few studies have assessed their reliability. This study assessed the reliability of online self-report versions of the Australian Treatment Outcomes Profile (ATOP) and Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) among people seeking treatment for substance use problems.

Procedures: One-hundred and five clients entering residential treatment for substance use problems (Mage = 33.34, 65% male) were recruited from two facilities in Queensland, Australia. Using a repeated-measures design, we compared online self-report with the original interview versions of the ASSIST, which measured (i) lifetime substance use and (ii) past 3-month substance-use and related harms, and the ATOP, which measured (i) past month frequency of substance use and (ii) the typical quantity used per day. Assessments were administered 1-7 days apart.

Findings: The ATOP demonstrated moderate-excellent inter-rater reliability for the past month use (yes/no) for all substance types, but had poor reliability for alcohol and cannabis. ATOP reliability was high-excellent for the total number of days used in the past month for all substances. The ASSIST demonstrated moderate-excellent inter-rater reliability for substance-use and related harms for all substances except tobacco, however was poor for lifetime use for most substances due to greater reporting in the interview assessment.

Conclusions: Reliable responding was observed for the frequency of substance use in the past month on the ATOP, and past 3-month substance-use and related harms on the ASSIST. These findings support use of online-self report measures a resource-efficient method to monitor substance use. Underreporting of lifetime use was found in self-report online version of the ASSIST, highlighting the need for improved instruction or interviewer/clinician assistance for lifetime use.