Most available studies in lithological mapping using spaceborne multispectral and hyperspectral remote sensing images employ different classification and spectral matching algorithms for performing this task; however, our experiment reveals that no single algorithm renders satisfactory results. Therefore, a new approach based on an ensemble of classifiers is presented for lithological mapping using remote sensing images in this paper, which returns enhanced accuracy. The proposed method uses a weighted pooling approach for lithological mapping at each pixel level using the agreement of the class accuracy, overall accuracy and kappa coefficient from the multi-classifiers of an image. The technique is implemented in four steps; (1) classification images are generated using a variety of classifiers; (2) accuracy assessments are performed for each class, overall classification and estimation of kappa coefficient for every classifier; (3) an overall within-class accuracy index is estimated by weighting class accuracy, overall accuracy and kappa coefficient for each class and every classifier; (4) finally each pixel is assigned to a class for which it has the highest overall within-class accuracy index amongst all classes in all classifiers. To demonstrate the strength of the developed approach, four supervised classifiers (minimum distance (MD), spectral angle mapper (SAM), spectral information divergence (SID), support vector machine (SVM)) are used on one hyperspectral image (Hyperion) and two multispectral images (ASTER, Landsat 8-OLI) for mapping lithological units of the Udaipur area, Rajasthan, western India. The method is found significantly effective in increasing the accuracy in lithological mapping.