Most high risk occupations involve a stressful environment and auditory alarms designed to capture operator's attention and alert them about potential incidents. Most studies on auditory alarms have been conducted using subjective measurements to explore, for example, perceived urgency, highlighting factor such as the spoken intonation as important. The present study aimed to investigate the effect of intonation and valence on behavioral performance in using a cross-modal oddball measuring the involuntary capture of attention by sound. Participants judged if visually presented digits were odd or even while exposed to task-irrelevant sounds. In 80% of the trials, a sine wave tone (standard) preceded each digit, while on 20% of the trials the standard was replaced by a spoken word (novel). Novels varied in semantic valence (negative versus neutral) and intonation (urgent versus calm). Subjective ratings of perceived "urgency" and "attention grabbingness" were subsequently collected for these words from the same participants. The results revealed that, compared to the standard condition, all novels increased accuracy slightly and equally. Response latencies proved more sensitive, however, yielding a reduced distraction effect for urgent than non-urgent words, while the words' valence had no impact. The results from the subjective ratings on the other hand showed that both the words urgency and content increased significantly perceived "urgency" and "attention grabbingness". In conclusion, some of our findings fit well with alarm studies on alarms using subjective ratings and their assumption that subjective ratings are valuable for the design of better alarms. However, our results also highlight the lack of correspondence between subjective and objective measures of attention capture with respect to the words' content.