We present a new algorithm to compute the voxel-wise genetic contribution to brain fiber microstructure using diffusion tensor imaging (DTI) in a dataset of 25 pairs of monozygotic (MZ) twins and 25 pairs of dizygotic (DZ) twins. First, the structural and DT scans were linearly co-registered. The structural MR scans were nonlinear mapped via a 3D fluid transformation to a geometrically centered mean template, and the deformation fields were applied to the DTI volumes. After tensor re-orientation to realign them to the anatomy, we computed several scalar and multivariate DT-derived measures including the geodesic anisotropy (GA), the tensor eigenvalues and the full diffusion tensors. A covariance-weighted distance was found between twins in the Log-Euclidean framework [2], and used as input to a maximum-likelihood based algorithm to compute the contributions from genetics (A), common environmental factors (C) and unique environmental ones (E) to fiber architecture. Quantitative genetic studies can make use of the full information in the diffusion tensor, using covariance weighted distances and statistics on the tensor manifold.