This study presents a machine learning-based framework for detecting critical events in railway infrastructure by analyzing vibration signals from trackside accelerometers. Traditional maintenance is often reactive and labor-intensive, but this approach uses continuous sensing and data analytics to enable proactive, real-time monitoring. The research leverages a comprehensive pipeline that includes data preprocessing, segmentation of time-series data into one-second intervals labeled as "event" or "no-event", and the extraction of statistical, temporal, and spectral features like crest factor and kurtosis. Key contribution of this work is the systematic evaluation of 72 algorithm-feature selection configurations. Twelve diverse classification algorithms were compared, including tree-based, linear, and neural network models. Extensive hyperparameter optimization was performed to benchmark performance using metrics such as accuracy, precision, recall, and F1-score. The Multi-Layer Perceptron (MLPClassifier) achieved a peak cross-validation accuracy of 98.89% with the full feature set. The study also found that comparable accuracy (98.67%) could be achieved with a 47% dimensionality reduction using Recursive Feature Elimination (RFE) with only eight features, demonstrating a balance between efficiency and performance. The findings provide actionable insights for developing scalable, high-performance event detection systems.