Defective railway wheels can cause severe damages to both track and vehicle items, which in worst case can lead to derailments with extensive losses. Wheel impact detection systems are intended to support prevention of railway damages and its related losses, through recognition of wheel defects and generation of alarms. While commissioning of a wheel impact detection system in Sweden, it was revealed that the detection system was reliable as they were not generating any false alarms. However, during operation some of the system's alarms resulted in No Fault Found (NFF) events. Hence an investigation was launched to determine if the cause of the NFF events were false alarms generated by the detection system, or the inability of subsequent manual inspection to replicate the detection system's test result. This paper presents experiences from the verification study and links these to international experiences illustrating how wheel impact detection systems can support continuous assessment of wheel condition and related condition based maintenance efforts.