Inter- and intra-observer variability of time-lapse annotations

Study question: How consistent is the time-lapse annotation of dynamic and static morphologic parameters of embryo development, within and between observers?

Summary answer: The assessment of dynamic parameters is characterized by almost perfect agreement within and between observers.

What is known already: The commonly employed method used to assess embryos in IVF treatments is based on static evaluation of morphology in a microscope, but this is limited by substantial intra- and inter-observer variation. Time-lapse imaging has been proposed as a method to refine embryo selection by adding new dynamic predictors of viability to the assessment. Yet, there are no data regarding the consistency of estimates of the time-lapse parameters.

Study design, size, duration: Infertile patients were recruited at the Fertility Clinic, Arhus University Hospital from February 2011 to June 2012. All embryos were cultured for 6 days in a time-lapse incubator (EmbryoScope(™)). Automated image recording was performed every 20 min.

Participants/materials, setting, methods: In total, 158 fertilized embryos from 20 different patients were annotated. Three observers made independent annotations on time-lapse recordings. One observer performed the assessment twice. Twenty-five parameters were annotated and the inter- and intra-observer agreement was assessed by calculating intra-class correlation coefficients (ICCs).

Main results and the role of chance: Extremely close agreement (ICC 0.99) was found for dynamic parameters including the timing of the following: pronuclei breakdown, completion of blastocyst hatching and the appearance and disappearance of the first nucleus after the first division. Observations of cleavage divisions were strongly correlated (ICC > 0.8), indicating close agreement. Measurements of the static morphologic parameters, i.e. multi-nucleation and evenness of blastomeres at 2-cell stage showed fair-to-moderate agreement (ICC ≤ 0.5).

Limitations, reason for caution: The study was conducted at a single clinic. Only embryos with a good prognosis were included. The influence of training sessions was not measured.

Wider implications of the findings: Consistency is crucial to the validity of embryo scoring and selection. All of the time-lapse parameters suggested by the literature showed in our study high intra- and inter-observer correlation, thus validating the precision of time-lapse annotations. This provides the basis for further investigation of embryo assessment and selection by time-lapse imaging in prospective trials.

Study funding/competing interest(s): Research at the Fertility Clinic was funded by an unrestricted grant from Ferring and MSD. The authors have no competing interests to declare.

Trial registration number: NCT01139268.