The influence of ultrasonic treatment (UST) on the microstructure of AISI‐304 and AISI‐316 stainless steels as two common commercial grades with similar properties but different levels of stacking fault energy (SFE) are compared in this study. The softening effect of the ultrasonic wave on the pre‐deformed structure is demonstrated by microhardness measurements, while the relaxation of tensile residual stresses is affirmed through the X‐ray diffraction (XRD) peak shifting. Electron and optical microscopy revealed a significant impact of ultrasound on the reduction of deformation twins’ fraction. A new mechanism for de‐twinning under the action of ultrasonic vibration is proposed using electron backscatter diffraction (EBSD) analysis. The reduction of 25% and 34% are detected in dislocation density of 10% pre‐deformed tensile sample after 300 W UST for 304SS and 316SS alloys, respectively. The effect of SFE is discussed, and it turned out that cross‐slip is the main mechanism of dislocation annihilation as a result of UST. Observation of the low‐deformation regions close to the grain boundaries indicated the occurrence of the recrystallization phenomenon during UST. Dislocation annihilation, de‐twinning, dislocation absorption to grain boundaries, and recrystallization are regarded as the softening and relaxation mechanisms of UST for austenitic stainless steels.