Background and purpose: It is important to adjust stroke outcomes for differences in initial stroke severity. The NIH Stroke Scale (NIHSS) is a commonly used stroke severity measure but has been validated for retrospective scoring only in a subset of stroke clinical trial participants. The purpose of this research was to assess the validity and reliability of an algorithm for retrospective NIHSS scoring in a setting with usual chart documentation.
Methods: An algorithm for retrospective NIHSS scoring was developed with written history and physical admission notes. Missing physical examination data were scored as normal. One investigator prospectively scored the admission NIHSS in 32 consecutive stroke patients. Two raters retrospectively scored the NIHSS by applying the algorithm to photocopied admission notes. Linear regression was used to assess interrater reliability and agreement between prospective and retrospective NIHSS scores. The Wilcoxon signed rank test was used to assess systematic scoring bias. Weighted kappa statistics were calculated to assess the level of agreement of individual NIHSS items.
Results: Only 1 admission note was complete for all NIHSS elements. Interrater reliability was near perfect (r(2)=0.98, P<0. 001). Agreement between prospective and retrospective NIHSS score was also excellent (r(2)=0.94, P<0.001) and there was no systematic bias in retrospective scores. Agreement for individual items was moderate to high for all items except level of consciousness.
Conclusions: Retrospective NIHSS scoring with the algorithm is reliable and unbiased even when physical examination elements are missing from the written record. Stroke research using retrospective review of charts or of administrative databases should adjust for differences in stroke severity using such an algorithm.