Ensemble learning methods, particularly stacking, are often expected to enhance the performance of machine learning models. In this study, an investigation was carried out on whether stacking consistently improves classification accuracy in the context of fault diagnosis. Vibration signals collected from a reciprocating air compressor wherein three distinct features such as statistical, histogram and autoregressive moving average (ARMA) features were extracted. The most significant features were selected using the J48 algorithm and a variety of machine learning classifiers were trained on these features. The performances of individual classifiers were recorded and compared against stacking ensembles built from the same models. The results show that while several individual models achieved high classification performance, stacking did not provide consistent improvements. These findings highlight that stacking was ineffective on the considered air compressor dataset and is not always advantageous in fault diagnosis.