This study investigated how the semantic and phonological properties of background speech affect reading, depending on primary task processing. Chinese participants were randomly assigned to two groups and read Chinese tongue twisters while exposed to meaningful, meaningless, spectrally-rotated speech (acoustically similar to normal speech but without linguistic information), or silence. One group engaged in a semantic task, comprehending sentences and responding to “yes-no” questions, while the other performed a phonological task, identifying the most frequent initial phoneme in sentences and selecting a corresponding character. Although background speech did not significantly influence accuracy for either task, it differentially impacted eye movements and reading rates. Semantic properties disrupted the semantic task without significantly affecting the phonological task, while phonological properties influenced both tasks, particularly the phonological one. These findings indicate that the nature of the reading task modulates the disruptive effects of background speech, supporting the interference-by-process account.