Respiratory motion challenges lung radiation therapy with uncertainties of the location of important anatomical structures in the thorax. To capture the trajectory of the motion, dense image matching methods and learning-based motion prediction methods have been commonly used. However, both methods have limitations. Serious motion artifacts in treatment-guidance images, such as streak artifacts in respiration-correlated cone-beam CT, challenge the intensity-based image matching; the learning-based prediction methods require consistency between the training data for planning and the data for treatment. This paper proposes a prediction-driven motion atlas framework for motion estimation with artifact-laden images, using a Frechet-mean-image matching scheme that is softly constrained by deformation predictions. In this framework, all the respiration phase-stamped images within a breathing cycle are diffeomorphically deformed to their Frechet mean. The iterative optimization is driven by both intensity matching forces and the prediction forces trained from patient-specific planning images. The effectiveness of the framework is demonstrated with computational phantom and real cone-beam CT images.