Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Ensuring Robot-Human Safety for the BD Spot Using Active Visual Tracking and NMPC With Velocity Obstacles
Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, Signals and Systems.ORCID iD: 0000-0002-1046-0305
Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, Signals and Systems.ORCID iD: 0000-0003-3922-1735
Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, Signals and Systems.ORCID iD: 0000-0003-0126-1897
2022 (English)In: IEEE Access, E-ISSN 2169-3536, Vol. 10, p. 100224-100233Article in journal (Refereed) Published
Abstract [en]

When humans and robots operate in and occupy the same local space, proximity detection and proactive collision avoidance is of high importance. As legged robots, such as the Boston Dynamics (BD) Spot, start to appear in real-world application environments, ensuring safe robot-human interactions while operating in full autonomy mode becomes a critical gate-keeping technology for trust in robotic workers. Towards that problem, this article proposes a track-and-avoid architecture for legged robots that combines a visual object detection and estimation pipeline with a Nonlinear Model Predictive Controller (NMPC) based on the Optimization Engine, capable of generating trajectories that satisfy the avoidance and tracking problems in real-time operations where the computation time never exceeded 40 ms. The system is experimentally evaluated using the BD Spot, in a custom sensor and computational suite, and in fully autonomous operational conditions, for the robot-human safety scenario of quickly moving noncooperative obstacles. The results demonstrate the efficacy of the scheme in multiple scenarios where the maximum safety distance violation was only 9 cm for an obstacle moving at 2.5 m/s while affected by both state estimation and object detection uncertainty and noise.

Place, publisher, year, edition, pages
IEEE, 2022. Vol. 10, p. 100224-100233
Keywords [en]
Human robot interaction, NMPC, object detection, object tracking, spot, velocity obstacle
National Category
Computer Sciences Control Engineering
Research subject
Robotics and Artificial Intelligence
Identifiers
URN: urn:nbn:se:ltu:diva-93404DOI: 10.1109/access.2022.3205611ISI: 000861351500001Scopus ID: 2-s2.0-85139414977OAI: oai:DiVA.org:ltu-93404DiVA, id: diva2:1701221
Funder
EU, Horizon 2020, 869379EU, Horizon 2020, 101003591
Note

Validerad;2022;Nivå 2;2022-10-05 (joosat);

Available from: 2022-10-05 Created: 2022-10-05 Last updated: 2023-01-20Bibliographically approved
In thesis
1. Risk Aware Path Planning and Dynamic Obstacle Avoidance towards Enabling Safe Robotic Missions
Open this publication in new window or tab >>Risk Aware Path Planning and Dynamic Obstacle Avoidance towards Enabling Safe Robotic Missions
2023 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

This compilation thesis presents two main contributions in path planning and obstacle avoidance, as well as an integration of the proposed modules with other frameworks to enable resilient robotic missions in complex environments.In general, through different types of robotic missions it is important to have a collision tolerant and reliable system, both regarding potential risks from collisions with dynamic and static obstacles, but also to secure the overall mission success.%Particularly, a common trend in the presented work is safety regarding collisions with dynamic and static obstacles, as well as reliable overall systems that are capable of executing missions.

The work included in this thesis presents the risk-aware path planner D$^*_+$ that is capable of planning traversable paths for both ground and aerial robots. D$^*_+$ is developed on top of D$^*$-lite with a risk layer close to occupied space, modeling the unknown areas as a risk, and is implemented with a dynamic map to enable updates and adjustments to a changing environment.

The risk layer aids in solving two common challenges with path planning for real robots: a) it creates a safety margin that gives free space between the path and obstacles so that robots with the corresponding size can follow the path, and b) it masks smaller holes in walls that occur when building maps from real data.

Using a dynamic map makes it possible to use D$^*_+$ for an exploration mission, it also enables for the re-planning of the path if the environment changes for example, if an obstacle suddenly blocks a path, a new path will be planned. D$^*_+$ have been tested in different real-life experiments with both an Unmanned Areal Vehicle (UAV) and a quadruped-legged robot and shown to produce traversable paths in different application scenarios, such as exploration, return to base, and navigation on known maps.

This thesis also presents an obstacle avoidance architecture for velocity objects, structured around an object detection and tracking scheme that is combined with non-linear model predictive controller (NMPC) to plan the avoidance maneuver. %that uses a Convolutional Neural Network to detect obstacles that are tracked so they can be avoided by a non-linear model predictive controller (NMPC).In this case, the detection is done with the Convoluitonal Neural Network (CNN) You Only Lock Once v4 (YOLO) where the most certain human is tracked with a Kalman filter, and the velocity of the human is estimated.The proposed scheme models the object motion as constant velocity, which is utilized from the NMPC to plan control inputs for the robot to avoid the identified obstacle. A merit of the approach is that the avoidance maneuver does not only consider the current identification position, but also considers the motion prediction of the object. This avoidance framework proved to be capable to avoid non-cooperative obstacles, such as humans moving towards it.Due to the fact that the avoidance is starting when a future collision is predicted, the avoidance maneuver is started early enough to avoid obstacles with a higher velocity than a classic ``static obstacle'' radius approach can handle.

An additional aim of this thesis is to showcase that the proposed contributions can be applied in full robotic missions/frameworks. Thus, this thesis presents a search and rescue mission with a quadruped-legged robot and a UAV on a partially known map to find the location of survivors and other objects of related interest. In this mission, the quadruped-legged robot carries the UAV to the edge of the known map from where it launches the UAV that then explores and detects any survival and other relevant objects.Also, an autonomy solution, based on Boston dynamics' quadruped-legged robot Spot, for enabling a map-based navigation in confined environments has been developed and tested. This Spot solution enables the robot to navigate to a user-selected point, rotate in the desired direction, and instruct the UAV, in the combined search and rescue mission, to take off.

Place, publisher, year, edition, pages
Luleå University of Technology, 2023
Series
Licentiate thesis / Luleå University of Technology, ISSN 1402-1757
Keywords
Robotic, Path planning, Obstacle avoidanc, Robotic missions, licenti thesis
National Category
Robotics and automation
Research subject
Robotics and Artificial Intelligence
Identifiers
urn:nbn:se:ltu:diva-95349 (URN)978-91-8048-248-6 (ISBN)978-91-8048-249-3 (ISBN)
Presentation
2023-03-09, A1545, Luleå tekniska universitet, Luleå, 09:00 (English)
Opponent
Supervisors
Available from: 2023-01-24 Created: 2023-01-20 Last updated: 2025-02-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Karlsson, SamuelLindqvist, BjörnNikolakopoulos, George

Search in DiVA

By author/editor
Karlsson, SamuelLindqvist, BjörnNikolakopoulos, George
By organisation
Signals and Systems
In the same journal
IEEE Access
Computer SciencesControl Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 102 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf