Good feature design is important to achieve effective image classification. This paper presents a novel feature design with two main contributions. First, prior to computing the feature descriptors, we propose to transform the images with learning-based filters to obtain more representative feature descriptors. Second, we propose to transform the computed descriptors with another set of learning-based filters to further improve the classification accuracy. In this way, while generic feature descriptors are used, data-adaptive information is integrated into the feature extraction process based on the optimization objective to enhance the discriminative power of feature descriptors. The feature design is applicable to different application domains, and is evaluated on both lung tissue classification in high-resolution computed tomography (HRCT) images and apoptosis detection in time-lapse phase contrast microscopy image sequences. Both experiments show promising performance improvements over the state-of-the-art.