In this paper we propose a method for the weakly supervised learning of sparse appearance models from medical image data based on Markov random fields (MRF). The models are learnt from a single annotated example and additional training samples without annotations. The approach formulates the model learning as solving a set of MRFs. Both the model training and the resulting model are able to cope with complex and repetitive structures. The weakly supervised model learning yields sparse MRF appearance models that perform equally well as those trained with manual annotations, thereby eliminating the need for tedious manual training supervision. Evaluation results are reported for hand radiographs and cardiac MRI slices.