Condition Monitoring (CM) is widely used in industry to meet sustainability, safety, and equipment efficiency requirements. Intelligent Fault Diagnosis (IFD) research focuses on automating CM data analysis tasks, to detect and prevent machine faults, and provide decision support. IFD enables trained analysts to focus their efforts on advanced tasks such as fault severity estimation and preventive maintenance optimization, instead of performing routine tasks. Industry datasets are rarely labelled, and IFD models are therefore typically trained on labelled data generated in laboratory environments with artificial or accelerated fault development. In the process industry, fault characteristics are often context-dependent and difficult to predict in sufficient detail due to the heterogeneous environment of machine parts. Furthermore, fault development is non-linear and measurements are subject to varying background noise. Thus, IFD models trained on lab data are not expected to transfer well to process industry environments, and require on-site pre-training or fine-tuning to facilitate accurate and advanced fault diagnosis. While ground truth labels are absent in industrial CM datasets, analysts sometimes write annotations of faults and maintenance work orders that describe the fault characteristics and required actions. These annotations deviate from typical natural language due to the technical language used, characterised by a high frequency of technical terms and abbreviations. Recent advances in natural language processing have enabled simultaneous learning from unlabelled pairs of images and captions through Natural Language Supervision (NLS). In this thesis, opportunities to enable weakly supervised IFD using annotated but otherwise unlabelled CM data are investigated. This thesis proposes novel machine learning methods for joint representation learning for IFD directly on annotated CM data. The main contributions are: (1) the introduction and implementation of technical language supervision to merge advances in natural language processing and, including a literature survey; (2) the creation of a method to improve technical languageprocessing by substituting out-of-vocabulary technical words with natural language descriptions, and to evaluate language model performance without explicit labels or downstream tasks; (3) the creation of a method for small-data language-based fault classification using human-centricvisualisation and clustering. Preliminary results for sensor and cable fault detection show an accuracy of over 90%. These results imply a considerable increase in the value of annotated CM datasets through the implementation of IFD models directly on industry data, e.g. for improving the decision support to avoid unplanned stops.

Recent advances in reasoning artificial intelligence (AI) agents powered by language models (LMs) and custom tools open new opportunities for AI-assisted condition monitoring (CM) involving unlabelled but annotated, complex industrial data. Technical language annotations written by domain experts include unstructured information regarding machine condition, maintenance actions, and tacit knowledge. This thesis investigates how LMs and agents can improve human-machine interaction and facilitate training of AI models on CM industry data using annotations as surrogate ground-truth labels. The main contribution is the introduction of technical language supervision to address the long-standing gap between idealised labelled lab datasets and complex unlabelled field data, and the development of AI agents for condition monitoring including a multimodal vector store with domain specific retrieval and generation modules.

The main contributions are: (1) the introduction and implementation of technical language supervision (TLS) for CM inspired by contrastive learning on images with natural language captions, including a literature survey and implementations of zero-shot fault diagnosis on unlabelled industry data; (2) the creation of a method to improve technical language processing by augmenting out-of-vocabulary technical words with natural language descriptions and evaluating semantic similarities of technical language representations; (3) the development of a human-centric method for language-based fault classification using visualisation and clustering; (4) the development of an open source chatbot agent as well as a published annotated industry dataset; (5) an investigation of specific CM data processing challenges, such as different data modalities, time-delays between annotations and signal properties, component-specific noise and feature levels, and nonlinear fault development over time in different data sources.

The results from the studies indicate that annotations are a viable substitute for labels when processed with regard to the technical language therein, and integrating LM powered agents on annotated CM data facilitates answering real industry queries more efficiently than current systems. By augmenting out-of-vocabulary technical words with natural language descriptions, LM performance is improved, as demonstrated in initial work on classifying technical fault descriptions with the BERT LM improving accuracy from 88.3 to 94.2%. In the industrial datasets analysed, gathered from Kraftliner paper machines over four years, the most common faults and alarms seen were cable and sensor faults, while bearing faults were the most common causes of follow up analysis and maintenance stops. Clustering CM data based on both signal and language properties indicates that cable and sensor faults can be differentiated from bearing faults with an F1-score of 92.6%, which suggests that the high number of false and redundant alarms that require follow-up analysis can be reduced. Finally, the usefulness of the developed agents was evaluated in typical CM workflows based on response usefulness and truthfulness, and the results indicate that AI agents with custom tools are capable of generating historic insight and meaningful fault descriptions. In particular, using a novel multimodal CM retrieval augmented generation approach with a custom CM vector store, the false alarm rate for sensor and cable faults is shown to be lowered from over 80% in current work flows, to under 30% with the proposed system.