We propose an approach that allows a rigorous understanding of the visual categorization and recognition process without asking direct questions about unobservable memory representations. Our approach builds on the selective use of visual information in recognition and a new method (Bubbles) to depict and measure what this information is. We examine three face-recognition tasks (identity, gender, expressive or not) and establish the componential and holistic information responsible for recognition performance. On the basis of this information, we derive task-specific gradients of probability for the allocation of attention to the different regions of the face.