Dependability of individualized Direct Behavior Rating Multi-Item Scales (DBR-MIS) for academic enablers

The present study examined the dependability of three newly developed direct behavior rating multi-item scales (DBR-MIS) of academic enablers (i.e., academic engagement, interpersonal skills, and study skills). Twenty-two K-5 teachers completed all three 5-item DBR-MIS daily for 1 week for one student in their class. Teachers' ratings on each item during the first occasion were used to create individualized DBR scales with 1-4 items. Items with the lowest ratings (indicating least frequent academic enablers) were included first and subsequent items were added in ascending order. Dependability of both full DBR-MIS and individualized DBR scales was evaluated using generalizability theory. Results indicated that the full DBR-MIS demonstrated high dependability and required only 1-4 assessment occasions (i.e., < 10 as the criterion) to inform absolute decision-making for progress monitoring. The three- and four-item individualized DBR-MIS demonstrated comparable dependability to their respective full five-item DBR-MIS. Dependability estimates of individualized scales in general were higher than standard D study-derived estimates with the same number of items (i.e., dependability estimates obtained by manipulating the number of items from the full standard scales modeled in D studies). Results support continued investigation of the DBR-MIS as a viable progress monitoring tool for school-based applications. Further research and implications for practice were discussed.