We propose a coupled dictionary learning method to predict deformation fields based on image appearance. Rather than estimating deformations by standard image registration methods, we investigate how to obtain a basis of the space of deformations. In particular, we explore how image appearance differences with respect to a common atlas image can be used to predict deformations represented by such a basis. We use a coupled dictionary learning method to jointly learn a basis for image appearance differences and their related deformations. Our proposed method is based on local image patches. We evaluate our method on synthetically generated datasets as well as on a structural magnetic resonance brain imaging (MRI) dataset. Our method results in an improved prediction accuracy while reducing the search space compared to nearest neighbor search and demonstrates that learning a deformation basis is feasible.