Statistical pitfalls in the comparison of multivariate causality measures for effective causality

The study of Wu et al. (2011) compared the performance of six different causality measures when the autoregressive process was estimated with the Dynamic Autoregressive Neuromagnetic Causal Imaging (DANCI) algorithm to help applied researchers in choosing the best method to estimate effective connectivity. This letter to the editor argues that four methodological restrictions limit the applicability of the results to actual applied research. First, there is no formal test for the significance of a connection between two channels. Second, the simulation results are affected by sizeable sampling variability. Third, only overestimation of the true model order is considered. Fourth, the comparison between methods always involves a joint hypothesis test. The letter discusses the limitations for applied researchers resulting from those restrictions and points to future research directions to overcome them.