The automatic and accurate segmentation of different organs is a critical step for computer-aided diagnosis, treatment planning and clinical decision support. However, for small organs such as the gallbladder, pancreas, and thyroid, accurate segmentation remains challenging due to their limited fraction in the image, high anatomical variability, and inhomogeneity. This paper presents a new fully automated multi-atlas segmentation approach to segment small organs using template construction, robust principal component analysis, and K-nearest neighbor classifier. Qualitative and quantitative evaluation has been evaluated on the VISCERAL challenge dataset. Experimental results show that the proposed system outperforms other multi-atlas based methods and forest-based methods in the segmentation of small organs.