Quality of Experience (QoE) subjective assessment often demands setting up expensive lab experiments that involve controlling several software programs and services. In addition, these experiments may pose significant challenges regarding man-agement of testbed software components as they may have to be synchronized for efficient data collection, leading to human errors or loss of time. Further, maintaining error-free repeatability between subsequent subjective tests and comprehensive data collection is essential. Therefore, this paper proposes, develops and validates ALTRUIST, a multi-platform tool that assists the experimenter in conducting subjective tests by controlling external applications, facilitates data collection and automates test execution for conducting repeatable subjective tests in broad application areas.