Unexpected task-irrelevant changes in the auditory or visual sensory channels have been shown to capture attention in an ineluctable manner and distract participants away from ongoing auditory or visual categorization tasks. We extend the study of this phenomenon by reporting the first within-participant comparison of deviance distraction in the tactile and auditory modalities. Using vibro-tactile-visual and auditory-visual cross-modal oddball tasks, we found that unexpected changes in the tactile and auditory modalities produced a number of functional similarities: A negative impact of distracter deviance on performance in the ongoing visual task, distraction on the subsequent trial (post-deviance distraction), and a similar decrease - but not the disappearance - of these effects across blocks. Despite these functional similarities, deviance distraction only correlated between the auditory and tactile modalities for the accuracy-based measure of deviance distraction and not for response latencies. Post-deviance distraction showed no correlation between modalities. Overall, the results suggest that behavioral deviance distraction may be underpinned by both modality-specific and multimodal mechanisms, while post-deviance distraction may predominantly relate to modality-specific processes.