This paper presents a new idea for improving text detection and recognition performances by detecting defects in the text detection results. Despite the rapid development of powerful deep learning based models for scene text detection and recognition in the wild, in complex situations (logos or decorated components connected with text), existing methods do not yield satisfactory results. In this paper, we propose to use post-processing method to improve the text detection and recognition performance. The proposed method extracts features, namely phase congruency, entropy and compactness for the text detection results. To strengthen discriminative power for feature extraction, we explore the combination of SVM classifier and Gaussian distribution of text components to determine proper weight, which represents true text component. The weights are multiplied with the features to detect defect components though clustering. The bounding boxes are redrawn, which results proper bounding box without defects components. Experimental results show that the proposed defect detection reports satisfactory results. To validate the effectiveness of defect detection, we conduct experiments on benchmark datasets of MSRA-TD-500 and SVT for detection and recognition before and after defect detection. The result shows that the performance of text detection and recognition improves significantly after defect detection.