The growing global demand for non-ferrous metals has led to serious environmental issues involving uncovered mine site slag dumps that threaten the surrounding soils, surface waters, groundwater, and the atmosphere. Remediation of these slags using substitute cement materials for ordinary Portland cement (OPC) and precursors for alkali-activated materials (AAMs) can convert hazardous solid wastes into valuable construction materials, as well as to attain the desired solidification and stabilization (S/S) of heavy metal(loid)s (HM). This review discusses the current research on the effect of non-ferrous slags on the reaction mechanisms of the OPC and AAM. The S/S of HM from the non-ferrous slags in AAM and OPC is also reviewed. HM can be stabilized in these materials based on the complex salt effect and isomorphic effects. The major challenges faced in AAMs and OPC for HM stabilization include the long-term durability of the matrix (e.g., sulfate attack, stability of volume). The existing knowledge gaps and future trends for the sustainable application of non-ferrous slags are also discussed.