A combination of approaches to the image analysis in cross-polarization optical coherence tomography (CP OCT) and high-resolution imaging by nonlinear microscopy and atomic force microscopy (AFM) at the different stages of atherosclerotic plaque development is studied. This combination allowed us to qualitatively and quantitatively assess the disorganization of collagen in the atherosclerotic arterial tissue (reduction and increase of CP backscatter), at the fiber (change of the geometric distribution of fibers in the second-harmonic generation microscopy images) and fibrillar (violation of packing and different nature of a basket-weave network of fibrils in the AFM images) organization levels. The calculated CP channel-related parameters are shown to have a statistically significant difference between stable and unstable (also called vulnerable) plaques, and hence, CP OCT could be a potentially powerful, minimally invasive method for vulnerable plaques detection.