This paper presents a non-intrusive system for detecting driver fatigue in real-time. To determine the level of fatigue the system uses various visual features, namely head nodding, eye closure duration and yawning. A state-of-the-art facial landmark detector ’IntraFace’ has been adopted to determine the eye state, mouth state and head pose estimation. However, different forms of uncertainties such as vagueness, imprecision, ambiguity and incompleteness are involved in calculating these visual parameters. Therefore, a Belief Rule-Based Expert System (BRBES) is employed, which has the ability to handle the uncertainties. The information of the visual parameters is sent to BRBES as input to determine the level of fatigue. An optimal learning model has been developed to improve the performance and accuracy of the BRBES. A comparison between the system and the fuzzy rulebased expert system has been carried out. The system generates more effective and reliable results than the fuzzy rule-based expert system.