Support vector machines (SVM) are well known to give good results on pattern recognition problems, but for large scale problems, they exhibit substantially slower classification speeds than neural networks. It has been proposed to speed the SVM classification by approximating the decision function of SVM with a reduced vector set. A new method to construct the reduced vector set is proposed in this paper, which is constructed by merging the closest support vectors in an iterative fashion. A minor modification on the proposed method also has been made in order to simplify the decision function of reduced support vector machines (RSVM). The proposed method was compared with previous study on several benchmark data sets, and the computational results indicated that our method could simplify SVMs and RSVMs effectively, which will speed the classification for large scale problems