The primary objective of inspecting concrete civil structures is to gather information concerning the deterioration of concrete elements, including issues like concrete cover cracking, delamination, or corrosion. Typically, this data is documented through field inspection notes, hand-drawn sketches, and photographs. Unfortunately, this information is often stored in diverse formats, relying on close-range images and paper-based records. Moreover, the process heavily depends on the inspector's experience, structural knowledge, and familiarity with the material properties of the system under investigation. Traditional inspection methods have inherent limitations, as they generally focus on easily accessible areas due to time constraints, safety concerns, or the challenging terrains often encountered in the field. This is particularly relevant when inspecting large structures like bridges, where examining the entire area would be time-consuming and potentially unsafe. The transfer of knowledge from one inspection period to the next becomes problematic when different inspectors are involved. Hence, there is a compelling need to explore modern inspection and monitoring techniques for structures, with a focus on reducing disruption and enhancing the efficiency and reliability of data acquisition.

In this context, the previously published licentiate thesis was aimed to contribute by developing optical alternatives that complement existing techniques. These alternatives should be cost-effective, suitable for on-site application, and easily deployable. To align with the objectives of this research project, the following research questions were addressed before in the licentiate seminar:

1.    How accurate is Close-Range Photogrammetry (CRP) for monitoring geometric deviations and detecting defects?

2.    In pursuit of maximum accuracy and minimal computation time in crack detection, which convolutional neural network (CNN) architecture performs best in classification and semantic segmentation tasks?

3.    Is there a correlation between surface deformations in reinforced concrete, measured through Digital Image Correlation (DIC), and strains in the embedded reinforcement?

However, there are still challenges to be addressed. Concrete civil structure inspection involves more than just defect detection and measurement. In this final thesis, the objective is to discuss an algorithm for creating an intelligent machine capable of classifying concrete defects. This requires the computer, acting as the inspector, to possess substantial knowledge about the concrete structure, including protocols, standards, guidelines, and an understanding of the overall structure's status. Consequently, two additional research questions are introduced:

1.    Building on our previous discussion in the licentiate seminar regarding the correlation between surface deformations and strains in embedded rebars, we aim to enhance the accuracy of the strain estimation. To achieve this, we intend to train intelligent regression models using available experimental data and newly generated synthetic data. Research Question 1: How can we estimate strains on embedded rebars with the application of machine learning regression, employing a hybrid-learning approach? This question is explored in the paper "Prediction of strain in embedded rebars for RC member: application of a hybrid learning approach."

2.    While computer vision techniques are effective in defect detection, structural health assessment encompasses more than just identifying defects. The objective is to provide a comprehensive solution that bridges the gap between defect detection, classification, and assessment, ultimately contributing to a more accurate understanding of detected defects. Research Question 2: How can we bridge the gap between defect detection and classification to achieve effective defect classification? This question is the subject of the forthcoming manuscript, "Defect Classification and Structural Assessment of Concrete Bridges: A Data-Driven Decision-Making Approach".

The aim of this study was to find strains in embedded reinforcement by monitoring surface deformations. Compared with analytical methods, application of the machine learning regression technique imparts a noteworthy reduction in modeling complexity caused by the tension stiffening effect. The present research aimed to achieve a hybrid learning approach for non-contact prediction of embedded strains based on surface deformations monitored by digital image correlation (DIC). However, due to the small training dataset collected by the installed strain gauges, the input dataset was enriched by a semi-empirical equation proposed in a previous study. Therefore, the present study discussed (i) instrumentation by strain gauge and DIC as well as data acquisition and post-processing of the data, accounting for strain gradients on the concrete surface and embedded reinforcement; (ii) input dataset generation for training machine learning regression models approaching hybrid learning; (iii) data regression to predict strains in embedded reinforcement based on monitored surface deformations; and (iv) the results, validation, and post-processing responses to make the method more robust. Finally, the developed model was evaluated through numerous statistical performance measures. The results showed that the proposed method can reasonably predict strain in embedded reinforcement, providing an innovative type of sensing application with highly improved performance.

Bridge inspections are relied heavily on visual inspection, and usually conducted within limited time windows, typically at night, to minimize their impact on traffic. This makes it difficult to inspect every meter of the structure, especially for large-scale bridges with hard-to-access areas, which creates a risk of missing serious defects or even safety hazards. This paper presents a new technique for the semi-automated damage detection in tunnel linings and bridges using a hybrid approach based on photogrammetry and deep learning. The first approach involves using photogrammetry to reconstruct a 3D model. It is shown that a model with sub-centimeter accuracy can be obtained after noise removal. However, noise removal also reduces the point cloud density, making the 3D point cloud unsuitable for quantification of small-scale damages such as fine cracks. Therefore, the captured images are also analyzed using deep convolutional neural network (CNN) models to enable crack detection and segmentation. For this aim, in the second approach, the 3D model is generated by the output of CNN models to enable crack localization and quantification on 3D digital model. These two approaches were evaluated in separate case studies, showing that the proposed technique could be a valuable tool to assist human inspectors in detecting, localizing, and quantifying defects on concrete structures.

Railway owners manage geographically dispersed networks comprising major elements of ageing infrastructure that are very susceptible to natural hazards. Consequently, transport agencies must address maintenance issues to guarantee serviceability and safety. This includes increased inspections and investing into structural health monitoring (SHM) programs. Regular SHM of existing bridges are usually scheduled during their service life to evaluate their health and as part of proactive maintenance where future deterioration is anticipated. Typically, a routine inspection consists of field measurements and visual observations made by a bridge inspector. However, disruption to civil infrastructure services due to scheduled maintenance work, visual inspection, etc. is increasing. 

The main purpose of SHM is to collect information such as geometry, previous and ongoing concrete deterioration, steel rebar corrosion, water seepage, concrete cover delamination, deflections, or settlements etc. The way such data are documented is through field inspection notes, freehand sketches, and photographs. Oftentimes, the data is stored in different systems and data collection and visualization still relies on paper-based record keeping processes. In addition, the procedure is highly dependent on the inspector’s experience [1], and knowledge of the structural behavior, together with the material properties of the system being investigated. The method has its limitations in the sense that only accessible parts are investigated due to time shortage, safety issues, or the difficult terrain in which the structure is sometimes located. This is especially true for large structures, such as bridges, where investigating the whole area would be highly time-consuming and potentially unsafe [2]. Honfi et al. [3] noted that the inspection’s duration is highly dependent on the bridge span (less than 10m can amount to 0.5 days and bigger than 100 m can amount to 20 days). In addition, defects can only be detected when their presence is visible to the naked eye, so they may already affect the life of the structure. Graybeal et al. [4] noted that routine inspections have relatively poor accuracy, with the following factors affecting the reliability of these results: inspector fear of traffic, near visual acuity, color vision, accessibility, and complexity. Furthermore, knowledge transfer from one inspection period to another becomes difficult when different inspectors carry out the investigation. Therefore, there is a strong need to identify new inspection and monitoring techniques for infrastructure that, in addition to being contactless and productive, reduce disruption, and improve the efficiency and reliability of the acquired data.

With the expansion of the low-cost consumer cameras, photogrammetry could play an important role in supporting SHM on existing bridges. Vaghefi et al. [5] carried out a study of 12 remote sensing technologies and their potential to detect a series of common problems on US bridges. They concluded that 3D optical technologies have potential for documenting surface-related defects. Faster bridge inspection and visualization was described when aiming to quantify the defects over bridge deck surfaces with a low cost and easily deployable technology. Other studies reported on the use of photogrammetry as alternative to traditional measurement applied in laboratory environment; a review is done by Baqersad et al. [6]. The researchers themselves have a well-proven experience in applying photogrammetry for determination of failure mechanisms in concrete structures, defect detection, and monitoring full-field deformations. In an effort, by Popescu et al. [7], to develop new monitoring and inspection methods with a preliminary study, photogrammetry and terrestrial laser scanning utilized to generate the 3D model of six railway bridges located in northern Sweden. Results have shown an acceptable performance of 3D model of existed infrastructure generated by photogrammetry. Therefore, the current project will contribute with optical alternatives to traditional SHM approaches that are low-cost, suitable for field application, and easily deployable.

The results indicate that bridge inspection on generated digital model is more reliable, productive, and accessible than traditional surveys. For 3D model generation we used photogrammetry technique, which is more efficient and cost-effective compared to the laser scanning, but improvements in accuracy and automation during the image acquisition phase are still required. 

The approach of autonomous defect detection performed on two case studies. Two types of defects including cracks, and block opening in a hard-to-access area was successfully detected and measured by pixel-wise mapping to an orthophoto. The proposed method has considerable potential in automated infrastructure inspection but some problems due to background noise remain to be overcome. The existence of noisy patterns such as shadows, dirt, and snow or water spots on surfaces makes damage detection very challenging, especially for the fine cracks. Overall, while the automated inspection technique proposed herein performs well, it clearly still requires supervision by a human inspector.

As reinforced concrete structures reach the end of their design lives, technology for improving accuracy and efficiency of inspections and structural health monitoring rapidly progresses. Concrete cracking and reinforcement strains are two relevant parameters in assessing damage and safety ofthese structures. The use of Digital Image Correlation (DIC) systems and distributed Fibre Optic Sensors (FOS) to evaluate these parameters are two of the technologies that have been gaining momentum due to their advantages over other approaches. This study presents an experimental investigation of crack propagation of a reinforced concrete beam specimen through FOS and DIC.The FOS were positioned inside a groove carved in the rebar and in the concrete immediately outside the bar for comparison. The results showed a significant difference between both positions, with more reliable data coming from inside the bar. The addition of the DIC crack propagation images to the FOS analysis complemented the results, and good visual correlation was identified between both methods. This study is part of a broader research program, which aims at applying DIC and FOS for structural health monitoring of a real scale bridge structure.

In bridge inspection, vertical displacement is a relevant parameter for both short and long-term health monitoring. Assessing change in deflections could also simplify the assessment work for inspectors. Recent developments in digital camera technology and photogrammetry software enables point cloud with colour information (RGB values) to be generated. Thus, close range photogrammetry offers the potential of monitoring big and small-scale damages by point clouds. The current paper aims to monitor geometrical deviations in Pahtajokk Bridge, Northern Sweden, using an optical data acquisition technique. The bridge in this study is scanned two times by almost one year a part. After point cloud generation the datasets were compared to detect geometrical deviations. First scanning was carried out by both close range photogrammetry (CRP) and terrestrial laser scanning (TLS), while second scanning was performed by CRP only. Analyzing the results has shown the potential of CRP in bridge inspection.

The aim of the current project is to digitalize inspections and monitoring of structures’ health using drones in order to identify and allow for easier inspection of damages in transport infrastructure. The objectives set are to perform aerial photogrammetry to recreate the as-is condition to enable off-site inspection of difficult to reach areas in structures and identify damages – e.g. cracks, spalling, corrosion. The drone is controlled either autonomously or with the use of a remote control by a pilot from the ground. The drone can carry a wide range of imaging technologies including still, video and infrared sensors. The high flexibility and accessibility of drones in hard-to-reach or risk exposed areas makes the airborne photogrammetry a better alternative to the ground-based method. Given the potential of UAVs to help bridge inspectors performing inspections off-site, the Swedish Transport Administration developed a demonstration project to evaluate the effectiveness and future opportunities within inspection field. Five bridges of varying sizes and types were selected as demonstrators. Data collection including the 3D model creation has been performed by three different contractors while the model-based inspection for all bridges was performed by the same team. It has been shown that the 3D models could serve as a tool for bridge inspectors from which measurements could be extracted and certain damages identified. A full off-site inspection is currently not feasible as some areas of the bridges were difficult to capture. The models are only providing near-surface information, and therefore, in-depth inspection should not be overlooked. The difficulty to capture local defects such as delaminations and narrow cracks also reduces versatility. The main conclusion from the study is that drones cannot be used independently to conduct inspections. Currently, they can only be used as a complement to traditional inspections. The added value of a 3D model derives from the possibility of using it as tool to better plan large inspections in the field and/or future maintenance work.

This study is carried out to assess the applicability of using a digital image correlation (DIC) system in structural inspection, leading to deploy innovative instruments for strain/stress estimation along embedded rebars. A semi-empirical equation is proposed to predict the strain in embedded rebars as a function of surface strain in RC members. The proposed equation is validated by monitoring the surface strain in ten concrete tensile members, which are instrumented by strain gauges along the internal steel rebar. One advantage with this proposed model is the possibility to predict the local strain along the rebar, unlike previous models that only monitored average strain on the rebar. The results show the feasibility of strain prediction in embedded reinforcement using surface strain obtained by DIC.

Conventional bridge inspection is usually performed by experienced engineers, trying to detect and document damage patterns manually. By increased number of built Bridges, there is a growing interest in automated damage detection methods. Therefore, the field of autonomous bridge inspection with the application of machine learning techniques on UAV-taken images is gaining popularity. Due to recent technological advancement, a large number of datasets can be collected, with a high rate of productivity and accuracy, to train convolutional neural networks (CNNs) leading us to automated Structural health monitoring (SHM). In this paper, a case study is chosen to scan two times with almost one year as a time interval. In the first scanning, dataset was gathered to train four different CNNs. Then, the performance of CNNs was compared for the purpose of autonomous crack detection in the second round of scanning. Models evaluated on a number of performance metrics, namely- (i) accuracy, (ii) loss, (iii) computation time, (iv) model size, and (v) architectural depth. Finally, the performance of studied CNNs is discussed, which can lead researchers in the Transfer-Learning approach to generate a model for damage detection with a limited number of datasets prepared in the first turn of bridge inspection. 

For the inspection of structures, particularly bridges, it is becoming common to replace humans with autonomous systems that use unmanned aerial vehicles (UAV). In this paper, a framework for autonomous bridge inspection using a UAV is proposed with a four-step workflow: (a) data acquisition with an efficient UAV flight path, (b) computer vision comprising training, testing and validation of convolutional neural networks (ConvNets), (c) point cloud generation using intelligent hierarchical dense structure from motion (DSfM), and (d) damage quantification. This workflow starts with planning the most efficient flight path that allows for capturing of the minimum number of images required to achieve the maximum accuracy for the desired defect size, then followed by bridge and damage recognition. Three types of autonomous detection are used: masking the background of the images, detecting areas of potential damage, and pixel-wise damage segmentation. Detection of bridge components by masking extraneous parts of the image, such as vegetation, sky, roads or rivers, can improve the 3D reconstruction in the feature detection and matching stages. In addition, detecting damaged areas involves the UAV capturing close-range images of these critical regions, and damage segmentation facilitates damage quantification using 2D images. By application of DSfM, a denser and more accurate point cloud can be generated for these detected areas, and aligned to the overall point cloud to create a digital model of the bridge. Then, this generated point cloud is evaluated in terms of outlier noise, and surface deviation. Finally, damage that has been detected is quantified and verified, based on the point cloud generated using the Terrestrial Laser Scanning (TLS) method. The results indicate this workflow for autonomous bridge inspection has potential.