The electrocardiogram (ECG) is a highly complex, dynamic and stochastic phenomenon. Although it provides a valuable, noninvasive and rapid means of assessing cardiac state and its change, uncertainties in its measurement and variation in the underlying electrophysiology that generates the ECG make difficult further improvement in its reliability for detecting and monitoring cardiac pathologies and conditions. This article reviews the sources of variability and uncertainty in ECG measurement and interpretation, revisits some old ideas for dealing with them, and proposes some novel directions for improving accuracy of ECG assessment and interpretation. We shall explore relative information content of lead systems, representation of ECG signals and patterns, and estimation of ECG distributions from limited lead systems. In addition, we will compare strategies for measuring ECG information and suggest new paradigms for feature extraction that reduce the sensitivity of assessment accuracy to intrinsic and extrinsic measurement errors. Finally, we review the importance of including dynamic information in ECG assessment, both for interpreting current cardiac state as well as for monitoring its change and significance.