Fault diagnosis is of great significance to ensure the reliability and safety of complex bevel gearbox systems. Most existing intelligent fault diagnosis approaches of bevel gearboxes are designed with vibration monitoring. However, the collected vibration data are vulnerable to noise pollution and machinery operating conditions. Besides, traditional fault diagnosis models highly rely on numerous labeled samples, and neglect the high cost of label annotation in real-world applications. Therefore, a novel fault diagnosis approach based on semi-supervised probability support matrix machine (SPSMM) and infrared imaging is proposed for bevel gearboxes in this paper, which has the following properties. Firstly, SPSMM classifies 2D matrix data directly without vectorization, thus fully utilizing the spatial information in infrared images. Secondly, a probability output strategy is designed for SPSMM to calculate the posterior class probability estimation of matrix inputs, and consequently enhance the diagnostic accuracy and robustness of the model. Thirdly, a semi-supervised learning (SSL) framework is proposed for SPSMM to carry out sample transfer from the unlabeled sample pool to the labeled sample pool, which can effectively alleviate the problem of insufficient labeled samples. The superiority of the proposed diagnosis approach is demonstrated with an infrared imaging dataset of a bevel gearbox.