Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Platform-Agnostic Resilient Decentralized Multi-Sensor Fusion for Pose Estimation
Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, Signals and Systems.ORCID iD: 0000-0003-3530-1084
2024 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis presents an innovative decentralised sensor fusion framework with significant potential to improve navigation accuracy in autonomous vehicles. Its applicability is especially noteworthy in demanding scenarios, such as adverse weather conditions and intricate urban environments. In general, sensor fusion is a crucial method for integrating signals from various sources, extracting and integrating information from multiple inputs into a unified signal or data set. Frequently, sources of information are from sensors or devices designed for the perception and measurement of dynamic environmental changes. The collected data from diverse sensors undergoes processing through specialised algorithms, commonly referred to as "sensor fusion" or "data fusion" algorithms. This thesis describes sensor fusion's significance in processing data from multiple sources. It highlights the classification of fusion algorithms, demonstrating the versatility and applicability of sensor fusion across a range of redundant sensors. Moreover, various creative strategies for sensor fusion, including fault detection and isolation and methods for addressing non-Gaussian noise through smoothing filter techniques, are collectively introduced as part of a comprehensive navigation framework. The contributions of this thesis are summarized in the following. First, it introduces a decentralised two-layered fusion architecture for pose estimation, emphasising fault resilience. In a decentralised fashion, it utilises distributed nodes equipped with extended Kalman filters in the initial tier and optimal information filters in the subsequent tier to amalgamate pose data from multiple sensors. The design is named the Fault-Resilient Optimal Information Fusion (FR-OIF) architecture in this thesis, which guarantees reliable pose estimation, even in cases of sensor malfunctions. Secondly, this work proposes an Auto-encoder-based fault detection framework for a multi-sensorial distributed pose estimation. In this framework, auto-encoders are applied to detect anomalies in the raw signal measurements. At the same time, a fault-resilient optimal information filter (FROIF) approach is incorporated with the auto-encoder-based detection to improve estimation accuracy. The effectiveness of these methods is demonstrated through experimental results involving a micro aerial vehicle and is compared to a novel classical detection approach based on the Extended Kalman filter. Furthermore, it introduces an integrated multi-sensor fusion architecture enhanced by centralised Auto-encoder technology and an EKF framework. This approach effectively removes sensor data noise and anomalies, ensuring reliable data reconstruction, even when faced with time-dependent anomalies. The assessment of the framework's performance using actual sensor data collected from the onboard sensors of a micro aerial vehicle demonstrates its superiority compared to a centralised Extended Kalman filter without Auto-encoders. The next part of the thesis discusses the increasing need for resilient autonomy in complex space missions. It emphasises the challenges posed by interactions with non-cooperative objects and extreme environments, calling for advanced autonomy solutions.  Furthermore, this work introduces a decentralised multi-sensor fusion architecture for resilient satellite navigation around asteroids. It addresses challenges such as dynamic illumination, sensor drift, and momentary sensor failure. The approach includes fault detection and isolation methods, ensuring autonomous operation in adverse conditions. Finally, the last part of the thesis focuses on accurate localisation and deviation identification in multi-sensor fusion with Millimeter-Wave Radars. It presents a flexible, decentralised smoothing filter framework that effectively handles unwanted measurements and enhances Ego velocity estimation accuracy.  Overall, this thesis plays a significant role in advancing the field of decentralised sensor fusion, encompassing anomaly avoidance mechanisms, fault detection and isolation frameworks, and robust navigation algorithms applicable across a range of domains, covering everything from robotics to space exploration. In the initial section of this thesis, we delve into the backdrop, reasons behind the research, existing challenges, and the contributions made. Conversely, the subsequent section comprises the complete articles linked to the outlined contributions and a bibliography.

Place, publisher, year, edition, pages
Luleå: Luleå University of Technology, 2024. , p. 184
Series
Licentiate thesis / Luleå University of Technology, ISSN 1402-1757
National Category
Signal Processing
Research subject
Robotics and Artificial Intelligence
Identifiers
URN: urn:nbn:se:ltu:diva-104362ISBN: 978-91-8048-488-6 (print)ISBN: 978-91-8048-489-3 (electronic)OAI: oai:DiVA.org:ltu-104362DiVA, id: diva2:1840002
Presentation
2024-04-03, A1545, Luleå University of Technology, Luleå, 13:00 (English)
Opponent
Supervisors
Available from: 2024-02-22 Created: 2024-02-22 Last updated: 2024-03-13Bibliographically approved
List of papers
1. Fault Resilient Decentralized Multi-sensorial Fusion Based Pose Estimation for Autonomous Navigation Around Asteroid
Open this publication in new window or tab >>Fault Resilient Decentralized Multi-sensorial Fusion Based Pose Estimation for Autonomous Navigation Around Asteroid
2023 (English)In: International Journal of Control, Automation and Systems, ISSN 1598-6446, E-ISSN 2005-4092, Vol. 21, no 6, p. 2031-2042Article in journal (Refereed) Published
Abstract [en]

A decentralized multi-sensor fusion-based resilient pose estimation architecture for autonomous navigation of satellites around an asteroid is presented in this article. Navigation around an asteroid is challenging due to dynamic illumination conditions, which restricts the vision-based localization and is partially ineffective for a longer duration of the operation. Moreover, drift in sensor measurement and temporal sensor failure is often encountered in long-duration sustainable space missions. This is more so around a debris-prone region, where momentary obstruction leads to inaccurate sensor measurement for a temporary period of operation. In order to establish a resilient localization mechanism for satellites around an asteroid, the proposed framework embeds a unique automatic fault detection and isolation approach in a decentralized fusion formalism. Furthermore, a unified framework can operate autonomously during temporary and long-range inoperative periods. In the first stage, innovative fault detection is proposed, which operates based on the residual of a judiciously designed filter assembly. Secondly, a novel fault-resilient isolation fusion called the fault-resilient optimal information filter fusion (FR-OIF) technique is presented, enabling self-resiliency by embedding an inbuilt fault isolation mechanism. The proposed resilient asteroid navigation approach is demonstrated with a simulation study considering a satellite equipped with multiple onboard sensors such as an inertial measurement unit, star tracker, camera and 3D-Lidar in the proximity of the asteroid Ryugu. At the same time, its superiority is also demonstrated through a comparison with the centralized multi-sensorial fusion approach.

Place, publisher, year, edition, pages
Springer Nature, 2023
Keywords
Asteroid navigation, decentralize fusion, fault resilient decentralized fusion, filter bank, maximum likelihood function, multi-sensor fusion
National Category
Control Engineering
Research subject
Robotics and Artificial Intelligence
Identifiers
urn:nbn:se:ltu:diva-99119 (URN)10.1007/s12555-022-0528-3 (DOI)001010828700026 ()2-s2.0-85162254538 (Scopus ID)
Funder
EU, Horizon 2020, 101003591 NEX-GEN SIMS
Note

Validerad;2023;Nivå 2;2023-07-03 (hanlid)

Available from: 2023-07-03 Created: 2023-07-03 Last updated: 2024-02-22Bibliographically approved
2. A decentralized sensor fusion scheme for multi sensorial fault resilient pose estimation
Open this publication in new window or tab >>A decentralized sensor fusion scheme for multi sensorial fault resilient pose estimation
Show others...
2021 (English)In: Sensors, E-ISSN 1424-8220, Vol. 21, no 24, article id 8259Article in journal (Refereed) Published
Abstract [en]

This article proposes a novel decentralized two-layered and multi-sensorial based fusion architecture for establishing a novel resilient pose estimation scheme. As it will be presented, the first layer of the fusion architecture considers a set of distributed nodes. All the possible combinations of pose information, appearing from different sensors, are integrated to acquire various possibilities of estimated pose obtained by involving multiple extended Kalman filters. Based on the estimated poses, obtained from the first layer, a Fault Resilient Optimal Information Fusion (FR-OIF) paradigm is introduced in the second layer to provide a trusted pose estimation. The second layer incorporates the output of each node (constructed in the first layer) in a weighted linear combination form, while explicitly accounting for the maximum likelihood fusion criterion. Moreover, in the case of inaccurate measurements, the proposed FR-OIF formulation enables a self resiliency by embedding a built-in fault isolation mechanism. Additionally, the FR-OIF scheme is also able to address accurate localization in the presence of sensor failures or erroneous measurements. To demonstrate the effectiveness of the proposed fusion architecture, extensive experimental studies have been conducted with a micro aerial vehicle, equipped with various onboard pose sensors, such as a 3D lidar, a real-sense camera, an ultra wide band node, and an IMU. The efficiency of the proposed novel framework is extensively evaluated through multiple experimental results, while its superiority is also demonstrated through a comparison with the classical multi-sensorial centralized fusion approach. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.

Place, publisher, year, edition, pages
MDPI, 2021
Keywords
Information fusion, Kalman filters, Maximum likelihood estimation, Micro air vehicle (MAV), Decentralised, Decentralized fusion, Fault resilient optimal information fusion, Fusion architecture, Linear minimum variance, Maximum likelihood function, Multi-sensor fusion, Optimal information filter, Optimal information fusion, Pose-estimation, Antennas
National Category
Computer graphics and computer vision
Research subject
Robotics and Artificial Intelligence
Identifiers
urn:nbn:se:ltu:diva-88527 (URN)10.3390/s21248259 (DOI)000737322300001 ()34960352 (PubMedID)2-s2.0-85120770377 (Scopus ID)
Funder
EU, Horizon 2020, 101003591 NEXGEN SIMS
Note

Validerad;2022;Nivå 2;2022-01-01 (johcin)

Available from: 2021-12-20 Created: 2021-12-20 Last updated: 2025-02-07Bibliographically approved
3. A Resilient to Faults Auto-Encoder Enabled Kalman based Multi-Sensorial Fusion
Open this publication in new window or tab >>A Resilient to Faults Auto-Encoder Enabled Kalman based Multi-Sensorial Fusion
2022 (English)In: 2022 30th Mediterranean Conference on Control and Automation (MED), IEEE, 2022, p. 662-669Conference paper, Published paper (Refereed)
Abstract [en]

This article presents a novel Auto-encoder-enabled fault resilient multi-sensor fusion architecture while incorporating an extended Kalman filter framework. The auto-encoder facilitate reconstruction of the faulty measurements from multiple onboard sensors, while the centralized extended Kalman filter enables an accurate fusion architecture. Moreover, the process is capable of successfully eliminating the additive noise appearing from the raw sensor data. The proposed method provides a robust reconstruction mechanism in the presence of time-dependent anomalies and faulty sensor measurement. The efficacy of the proposed scheme is extensively evaluated in the context of pose estimation for a micro aerial vehicle equipped with multiple onboard sensors. In addition, the evaluation process incorporates various realistic failure scenarios with artificially introduced inaccurate measurements. The superiority of the proposed Auto-encoder enabled centralized Kalman filter (AEKF) fusion is demonstrated through an extensive comparison with a recently developed Fault Resilient Optimal Information Filter (FROIF) method.

Place, publisher, year, edition, pages
IEEE, 2022
Series
Mediterranean Conference on Control and Automation (MED), ISSN 2325-369X, E-ISSN 2473-3504
Keywords
Auto-encoder, FROIF, Kalman Filter, Aerial Robot, White Gaussian Noise
National Category
Computer graphics and computer vision
Research subject
Robotics and Artificial Intelligence
Identifiers
urn:nbn:se:ltu:diva-92640 (URN)10.1109/MED54222.2022.9837222 (DOI)000854013700108 ()2-s2.0-85136256447 (Scopus ID)
Conference
30th Mediterranean Conference on Control and Automation (MED), Vouliagmeni, Greece, June 28 - July 1, 2022
Note

ISBN för värdpublikation: 978-1-6654-0673-4 (electronic), 978-1-6654-0674-1 (print)

Available from: 2022-08-23 Created: 2022-08-23 Last updated: 2025-02-07Bibliographically approved
4. An Auto-Encoder enabled Fault Detection and Isolation Scheme for enabling a Multi-Sensorial Distributed Pose Estimation
Open this publication in new window or tab >>An Auto-Encoder enabled Fault Detection and Isolation Scheme for enabling a Multi-Sensorial Distributed Pose Estimation
2022 (English)In: 2022 IEEE 31st International Symposium on Industrial Electronics (ISIE), IEEE, 2022, p. 281-288Conference paper, Published paper (Refereed)
Abstract [en]

This article proposes a novel Auto-encoder based fault detection and isolation framework approach for supporting the operation of a novel multi-sensorial distributed pose estimations scheme. The proposed work detects weak and strong time-dependent anomalies in a decentralised fusion approach from the initial estimation layer. As it will be presented, at the end of the learning phase, the neural network-based auto-encoders provide synthetic actual position and orientation of the robotic system, based on the statistics of the learning data. As a result, the square error between the output and input signal of the auto-encoder can yield the actual outlier with reasonable success. On the other hand, an Extended Kalman Filter (EKF) based fault detection method has been introduced in this article, which consists of a set of judiciously designed EKF acts as filter assembly. Based on innovation obtained from each of the EKF an innovative detection logic is proposed to identify the outlier in sensor measurement autonomously at the appropriate time samples. Based on the degree of accuracy of detecting the anomaly, the estimated signal is accepted or rejected for each time sample in the second layer of the fusion architecture. Moreover, we will introduce two outlier detection methods for the demonstration purposes and outline a comparative study using experimental data from a micro aerial vehicle. An extensive analysis with supporting results demonstrate these two methods' effectiveness and accuracy.

Place, publisher, year, edition, pages
IEEE, 2022
Series
Proceedings of the IEEE International Symposium on Industrial Electronics, ISSN 2163-5137, E-ISSN 2163-5145
Keywords
Auto encoder, Multi sensor fusion, Decentralize fusion, Filter bank, Maximum likelihood function, Optimal information filter
National Category
Robotics and automation
Research subject
Robotics and Artificial Intelligence
Identifiers
urn:nbn:se:ltu:diva-92627 (URN)10.1109/ISIE51582.2022.9831458 (DOI)000946662000043 ()2-s2.0-85135844407 (Scopus ID)
Conference
31st International Symposium on Industrial Electronics (ISIE), Anchorage [Hybrid], Alaska, USA, June 1-3, 2022
Note

ISBN för värdpublikation: 978-1-6654-8240-0 (electronic), 978-1-6654-8241-7 (print)

Available from: 2022-08-22 Created: 2022-08-22 Last updated: 2025-02-09Bibliographically approved
5. Decentralized Fusion-Based Ego Velocity Estimation Using Multiple FMCW Radars
Open this publication in new window or tab >>Decentralized Fusion-Based Ego Velocity Estimation Using Multiple FMCW Radars
2023 (English)In: 2023 21st International Conference on Advanced Robotics (ICAR), IEEE, 2023, p. 244-251Conference paper, Published paper (Refereed)
Place, publisher, year, edition, pages
IEEE, 2023
Series
International Conference on Advanced Robotics and Intelligent Systems, ISSN 2374-3255, E-ISSN 2572-6919
National Category
Signal Processing
Research subject
Robotics and Artificial Intelligence
Identifiers
urn:nbn:se:ltu:diva-104348 (URN)10.1109/ICAR58858.2023.10406467 (DOI)2-s2.0-85185831433 (Scopus ID)979-8-3503-4229-1 (ISBN)979-8-3503-4230-7 (ISBN)
Conference
21st International Conference on Advanced Robotics (ICAR), Abu Dhabi, United Arab Emirates, 5-8 December, 2023
Funder
EU, Horizon 2020, 101003591 NEX-GEN SIMS
Available from: 2024-02-21 Created: 2024-02-21 Last updated: 2024-04-11Bibliographically approved
6. Resiliency in Space Autonomy: a Review
Open this publication in new window or tab >>Resiliency in Space Autonomy: a Review
2023 (English)In: Current Robotics Reports, E-ISSN 2662-4087, Vol. 4, p. 1-12Article, review/survey (Refereed) Published
Abstract [en]

Purpose of Review: The article provides an extensive overview on the resilient autonomy advances made across various missions, orbital or deep-space, that captures the current research approaches while investigating the possible future direction of resiliency in space autonomy.

Recent Findings: In recent years, the need for several automated operations in space applications has been rising, that ranges from the following: spacecraft proximity operations, navigation and some station keeping applications, entry, decent and landing, planetary surface exploration, etc. Also, with the rise of miniaturization concepts in spacecraft, advanced missions with multiple spacecraft platforms introduce more complex behaviours and interactions within the agents, which drives the need for higher levels of autonomy and accommodating collaborative behaviour coupled with robustness to counter unforeseen uncertainties. This collective behaviour is now referred to as resiliency in autonomy. As space missions are getting more and more complex, for example applications where a platform physically interacts with non-cooperative space objects (debris) or planetary bodies coupled with hostile, unpredictable, and extreme environments, there is a rising need for resilient autonomy solutions.

Summary: Resilience with its key attributes of robustness, redundancy and resourcefulness will lead toward new and enhanced mission paradigms of space missions.

Place, publisher, year, edition, pages
Springer, 2023
Keywords
Space robotics, Robot manipulators, Sub-T exploration, Distributed spacecraft, Resiliency, Autonomy
National Category
Computer graphics and computer vision
Research subject
Robotics and Artificial Intelligence
Identifiers
urn:nbn:se:ltu:diva-96173 (URN)10.1007/s43154-023-00097-w (DOI)
Note

Validerad;2023;Nivå 1;2023-07-20 (sofila);

Licens fulltext: CC BY License

Available from: 2023-03-17 Created: 2023-03-17 Last updated: 2025-02-07Bibliographically approved

Open Access in DiVA

fulltext(17976 kB)920 downloads
File information
File name FULLTEXT01.pdfFile size 17976 kBChecksum SHA-512
46782bb764ed38a1c208e3197cc636bd31a33daa1c7a0bf9f65cb04096e1de8e169f7a0bf5d321d8f6c9323532ccd1f1bb74ef493a399e2185bdb1d5d95b5c82
Type fulltextMimetype application/pdf

Authority records

Mukherjee, Moumita

Search in DiVA

By author/editor
Mukherjee, Moumita
By organisation
Signals and Systems
Signal Processing

Search outside of DiVA

GoogleGoogle Scholar
Total: 920 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 566 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf