Wire Arc Additive Manufacturing allows the cost-effective manufacturing of customized, large-scale metal parts. As the post-process quality assurance of large parts is costly and time-consuming, process monitoring is inevitable. In the present study, a context-aware monitoring solution was investigated by integrating machine, temporal, and spatial context in the data analysis. By analyzing the voltage patterns of each cycle in the oscillating cold metal transfer process with a deep neural network, temporal context was included. Spatial context awareness was enabled by building a digital twin of the manufactured part using an Octree as spatial indexing data structure. By means of the spatial context awareness, two quality metrics—the defect expansion and the local anomaly density—were introduced. The defect expansion was tracked in-process by assigning detected defects to the same defect cluster in case of spatial correlation. The local anomaly density was derived by defining a spherical region of interest which enabled the detection of aggregations of anomalies. By means of the context aware monitoring system, defects were detected in-process with a higher sensitivity as common defect detectors for welding applications, showing less false-positives and false-negatives. A quantitative evaluation of defect expansion and densities of various defect types such as pore nests was enabled.