This article presents the first ever fully autonomous UAV (Unmanned Aerial Vehicle) mission to perform gas measurements after a real blast in an underground mine. The demonstration mission was deployed around 40 minutes after the blast took place, and as such realistic gas levels were measured. We also present multiple field robotics experiments in different mines detailing the development process. The presented novel autonomy stack, denoted as the Routine Inspection Autonomy (RIA) framework, combines a risk-aware 3D path planning D + ∗ , with 3D LiDAR-based global relocalization on a known map, and it is integrated on a custom hardware and a sensing stack with an onboard gas sensing device. In the presented framework, the autonomous UAV can be deployed in incredibly harsh conditions (dust, significant deformations of the map) shortly after blasting to perform inspections of lingering gases that present a significant safety risk to workers. We also present a change detection framework that can extract and visualize the areas that were changed in the blasting procedure, a critical parameter for planning the extraction of materials, and for updating existing mine maps. As will be demonstrated, the RIA stack can enable robust autonomy in harsh conditions, and provides reliable and safe navigation behavior for autonomous Routine Inspection missions.

This work presents a fully integrated tree-based combined exploration-planning algorithm: exploration-rapidly-exploring random trees (RRT) (ERRT). The algorithm is focused on providing real-time solutions for local exploration in a fully unknown and unstructured environment while directly incorporating exploratory behavior, robot-safe path planning, and robot actuation into the central problem. ERRT provides a complete sampling and tree-based solution for evaluating “where to go next” by considering a tradeoff between maximizing information gain and minimizing the distances traveled and the robot actuation along the path. The complete scheme is evaluated in extensive simulations, comparisons, and real-world field experiments in constrained and narrow subterranean and GPS-denied environments. The framework is fully robot operating system (ROS) integrated and straightforward to use.

This article presents an appendix to the original NeBula autonomy solution developed by the Team Collaborative SubTerranean Autonomous Robots (CoSTAR), participating in the DARPA Subterranean Challenge. Specifically, this article presents extensions to NeBula’s hardware, software, and algorithmic components that focus on increasing the range and scale of the exploration environment. From the algorithmic perspective, we discuss the following extensions to the original NeBula framework: 1) large-scale geometric and semantic environment mapping; 2) an adaptive positioning system; 3) probabilistic traversability analysis and local planning; 4) large-scale partially observable Markov decision process (POMDP)-based global motion planning and exploration behavior; 5) large-scale networking and decentralized reasoning; 6) communicationaware mission planning; and 7) multimodal ground–aerial exploration solutions.We demonstrate the application and deployment of the presented systems and solutions in various large-scale underground environments, including limestone mine exploration scenarios as well as deployment in the DARPA Subterranean challenge.

In this article, we present an edge-based architecture for enhancing the autonomous capabilities of resource-constrained aerial robots by enabling a remote nonlinear model predictive control scheme, which can be computationally heavy to run on the aerial robots' onboard processors. The nonlinear model predictive control is used to control the trajectory of an unmanned aerial vehicle while detecting, and preventing potential collisions. The proposed edge architecture enables trajectory recalculation for resource-constrained unmanned aerial vehicles in relatively real-time, which will allow them to have fully autonomous behaviors. The architecture is implemented with a remote Kubernetes cluster on the edge side, and it is evaluated on an unmanned aerial vehicle as our controllable robot, while the robotic operating system is used for managing the source codes, and overall communication. With the utilization of edge computing and the architecture presented in this work, we can overcome computational limitations, that resource-constrained robots have, and provide or improve features that are essential for autonomous missions. At the same time, we can minimize the relative travel time delays for time-critical missions over the edge, in comparison to the cloud. We investigate the validity of this hypothesis by evaluating the system's behavior through a series of experiments by utilizing either the unmanned aerial vehicle or the edge resources for the collision avoidance mission.

This paper presents a complete system architecture for multi-robot coordination for unbalanced task assignments, where a number of robots are supposed to visit and accomplish missions at different locations. The proposed method first clusters tasks into clusters according to the number of robots, then the assignment is done in the form of one-cluster-to-one-robot, followed by solving the traveling salesman problem (TSP) to determine the visiting order of tasks within each cluster. A nonlinear model predictive controller (NMPC) is designed for robots to navigate to their assigned tasks while avoiding colliding with other robots. Several simulations are conducted to evaluate the feasibility of the proposed architecture. Video examples of the simulations can be viewed at https://youtu.be/5C7zTnv2sfo and https://youtu.be/-JtSg5V2fTI?si=7PfzZbleOOsRdzRd. Besides, we compare the cluster-based assignment with a simulated annealing (SA) algorithm, one of the typical solutions for the multiple traveling salesman problem (mTSP), and the result reveals that with a similar optimization effect, the cluster-based assignment demonstrates a notable reduction in computation time. This efficiency becomes increasingly pronounced as the task-to-agent ratio grows.

In this article, a novel approach for merging 3-D point cloud maps in the context of egocentric multirobot exploration is presented. Unlike traditional methods, the proposed approach leverages state-of-the-art place recognition and learned descriptors to efficiently detect overlap between maps, eliminating the need for the time-consuming global feature extraction and feature matching process. The estimated overlapping regions are used to calculate a homogeneous rigid transform, which serves as an initial condition for the general iterative closest point (GICP) point cloud registration algorithm to refine the alignment between the maps. The advantages of this approach include faster processing time, improved accuracy, and increased robustness in challenging environments. Furthermore, the effectiveness of the proposed framework is successfully demonstrated through multiple field missions of robot exploration in a variety of different underground environments.

Visual servoing plays a crucial role in robotics, spanning across a great spectrum of applications from autonomous cars to aerial manipulation. This article proposes TFMarker, a novel tangible fiducial pattern for enabling camera-assisted guided landing of UAVs by using the visual features from color markers as the main source of information. TFMarker is structured around a 4-point fiducial marker, allowing for accurate, precise, and consistent pose estimation in different environments and lighting conditions, while also offering resilience to motion blur. The presented detection framework is based on a three-step architecture, where the first step uses Gaussian and color filtering in addition to morphological operation in order to generate a robust detection of the markers. The second step uses the Gift Wrapping Algorithm, to organize the same-color markers based on their relative positioning with respect to the off-color marker. Finally, the Perspective-n-Point optimization problem is solved in order to extract the pose (i.e. position and orientation) of the proposed pattern with respect to the vision sensor. The efficacy of the proposed scheme has been extensively validated in indoor and SubT environments for the task of autonomous landing using a custom-made UAV. The experimental results showcase the performance of the proposed method, which presents a better detection rate in both environments while retaining similar accuracy and precision to the baseline approach. For the video of the experimental evaluation please refer to the following link: https://youtu.be/Zh13OObp15Q

Exploration and safe navigation in previously unknown GPS-denied obstructed areas are major challenges for autonomous robots when deployed in subterranean environments. In response, this work establishes an Exploration-Planning framework developed for the real-world deployment of Micro Aerial Vehicles (MAVs) in subterranean exploration missions. The fundamental task for an autonomous MAV to navigate in an unknown area, is to decide where to look while navigating such that the MAV will acquire more information about the surrounding. The work presented in this article focuses around 3D exploration of large-scale caves or multi-branched tunnel like structures, while still prioritizing the Look-Ahead and Move-Forward approach for fast navigation in previously unknown areas. In order to achieve such exploration behaviour, the proposed work utilizes a two-layer navigation approach. The first layer deals with computing traversable frontiers to generate the look ahead poses in the constrained field of view, aligned with the MAV’s heading vector that leads to rapid continuous exploration. The proposed frontier distribution based switching goal selection approach allows the MAV to explore various terrains, while still regulating the MAV’s heading vector. The second layer of the proposed scheme deals with global cost based navigation of the MAV to the potential junction in a multi-branched tunnel system leading to a continuous exploration of partially seen areas. The proposed framework is a combination of a novel frontier goal selection approach, risk-aware expandable grid based path planning, nonlinear model predictive control and artificial potential fields based on local reactive navigation, obstacle avoidance, and control for the autonomous deployment of MAVs in extreme environments.

This chapter will present the application of deployment of full autonomous Aerial Robotic Workers for inspection and exploration tasks in Subterranean environments. The framework shown will focus on the navigation, control, and perception capabilities of resource-constrained aerial platforms, contributing to the development of consumable scouting robotic platforms for real-life applications in extreme environments. In the approach, the aerial platform will be treated as a floating object, commanded by a velocity controller on the x-y axes, a height controller, as well as a heading correction module aligning the platform with the mining tunnel open space. Multiple experimentally verified methods regarding the heading correction module, for dark environments with limited texture, using either a visual camera or a 2D LiDAR presented in real mining environments are presented.

This Chapter focuses on Aerial Robotic Workers (ARWs) various control methods to successfully track the desired states, waypoints, and trajectories. Additionally, this Chapter discusses the regulation from the motor commands level to the accurate tracking of waypoints in 3D space. Various model-based control frameworks are presented based on the modeled dynamics of the Modeling for ARWs (Chapter 3). Initially, a classical Proportional-Integral-Derivative (PID) control scheme is introduced, while in the sequel, a Linear Quadratic Regulator (LQR) and a Model Predictive Controller (MPC) are designed for the linearized dynamics of ARWs. In the sequel, a Nonlinear-MPC (NMPC) version of the simplified position control scheme is given. Finally, a switching MPC is presented for the attitude regulation of a reconfigurable ARW.