This paper presents a novel and robust algorithm, for multiple motion detection and tracking in dynamic and complex scenes. The algorithm consists of two steps: at first, we use a robust algorithm for human detection. Then, Gaussian mixture model (GMM), Neighborhood-based difference and Overlapping-based classification are applied to improve human detection performance .The conventional mixture Gaussian method suffers from false motion detection in complex backgrounds and slow convergence. We combine three above mentioned methods to obtain detection. The second step of the proposed algorithm is object tracking framework based on Kalman filtering which works well in dynamic scenes. Experimental results show the high performance of the proposed method for multiple object tracking in complex and noisy backgrounds.