Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Implementation of a System for Real-Time Detection and Localization of Terrain Objects on Harvested Forest Land
Luleå University of Technology, Department of Engineering Sciences and Mathematics, Product and Production Development.ORCID iD: 0000-0001-9965-6955
Luleå University of Technology, Department of Engineering Sciences and Mathematics, Product and Production Development.ORCID iD: 0000-0002-9862-828x
2021 (English)In: Forests, E-ISSN 1999-4907, Vol. 12, no 9, article id 1142Article in journal (Refereed) Published
Abstract [en]

Research highlights: An automatic localization system for ground obstacles on harvested forest land based on existing mature hardware and software architecture has been successfully implemented. In the tested area, 98% of objects were successfully detected and could on average be positioned within 0.33 m from their true position in the full range 1–10 m from the camera sensor. Background and objectives: Forestry operations in forest environments are full of challenges; detection and localization of objects in complex forest terrains often require a lot of patience and energy from operators. Successful automatic real-time detection and localization of terrain objects not only can reduce the difficulty for operators but are essential for the automation of harvesting and logging tasks. We intend to implement a system prototype that can automatically locate ground obstacles on harvested forest land based on accessible hardware and common software infrastructure. Materials and Methods: An automatic object detection and localization system based on stereo camera sensing is described and evaluated in this paper. This demonstrated system detects and locates objects of interest automatically utilizing the YOLO (You Only Look Once) object detection algorithm and derivation of object positions in 3D space. System performance is evaluated by comparing the automatic detection results of the tests to manual labeling and positioning results. Results: Results show high reliability of the system for automatic detection and location of stumps and large stones and shows good potential for practical application. Overall, object detection on test tracks was 98% successful, and positional location errors were on average 0.33 m in the full range from 1–10 m from the camera sensor. Conclusions: The results indicate that object detection and localization can be used for better operator assessment of surroundings, as well as input to control machines and equipment for object avoidance or targeting.

Place, publisher, year, edition, pages
MDPI, 2021. Vol. 12, no 9, article id 1142
Keywords [en]
automation, object identification, obstacle detection, terrain obstacle
National Category
Forest Science Computer Vision and Robotics (Autonomous Systems)
Research subject
Machine Design
Identifiers
URN: urn:nbn:se:ltu:diva-86809DOI: 10.3390/f12091142ISI: 000701035100001Scopus ID: 2-s2.0-85114723919OAI: oai:DiVA.org:ltu-86809DiVA, id: diva2:1587693
Funder
Swedish Energy Agency, 48003-1Vinnova, 2020-04202EU, Horizon 2020, H2020-EU.3.5.1.European Regional Development Fund (ERDF), NYPS 20202905
Note

Validerad;2021;Nivå 2;2021-09-01 (johcin)

Available from: 2021-08-25 Created: 2021-08-25 Last updated: 2024-07-04Bibliographically approved
In thesis
1. Object Identification for Autonomous Forest Operations
Open this publication in new window or tab >>Object Identification for Autonomous Forest Operations
2022 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

The need to further unlock productivity of forestry operations urges the increase of forestry automation. Many essential operations in forest production, such as harvesting, forwarding, planting, etc., have the potential to be automated and obtain benefits such as improved production efficiency, reduced operating costs, and an improved working environment. In view of the fact that forestry operations are performed in forest environments, the automation of forestry operations is thus complex and extremely challenging. To build the ability of forest machine automation, it is necessary to construct an environmental cognitive ability of the forest machine as basis. Through a combination of exteroceptive sensors and algorithms, forest machine vision can be realized. Using new and off-the-shelf solutions for detecting, locating, classifying and analyzing the status of objects of concern surrounding the machine during forestry operations in combination with smart judgement and control, forest operations can be automated. This thesis focuses on the introduction of vision systems on an unmanned forest platform, aiming to create the foundation for autonomous decision-making and execution in forestry operations. Initially, the vision system is designed to work on an unmanned forest machine platform, to create necessary conditions to either assist operators or to realize automatic operation as a further step.

In this thesis, vision systems based on stereo camera sensing are designed and deployed on an unmanned forest machine platform and the functions of detection, localization and pose estimation of objects that surround the machine are developed and evaluated. These mainly include a positioning function for forest terrain obstacles such as stones and stumps based on stereo camera data and deep learning, and a localization and pose estimation function for ground logs based on stereo camera and deep learning with added functionality of color difference comparison. By testing these systems’ performance in realistic scenarios, this thesis describe the feasibility of improving the automation level of forest machine operation by building a vision system. In addition, the thesis also demonstrate that the accuracy of stump detection can be improved without significantly increasing the processing load by introducing depth information into training and execution.

Place, publisher, year, edition, pages
Luleå: Luleå University of Technology, 2022
Series
Licentiate thesis / Luleå University of Technology, ISSN 1402-1757
National Category
Computer Vision and Robotics (Autonomous Systems)
Research subject
Machine Design
Identifiers
urn:nbn:se:ltu:diva-90263 (URN)978-91-8048-077-2 (ISBN)978-91-8048-078-9 (ISBN)
Presentation
2022-06-16, E632, Luleå tekniska universitet, Luleå, 09:00 (English)
Opponent
Supervisors
Available from: 2022-04-21 Created: 2022-04-20 Last updated: 2023-09-05Bibliographically approved
2. Vision based perception systems for unmanned forestry machines
Open this publication in new window or tab >>Vision based perception systems for unmanned forestry machines
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

After years of continuous development in the past decades, the field of forest production has achieved a notable degree of mechanization. Nonetheless, the unceasing demand for increased productivity in forestry operations continues to drive the rapid expansion of forestry automation. Critical aspects of forest production, including activities such as harvesting, forwarding, seedling planting, and soil preparation, hold significant potential for improved automation. These advancements promise numerous advantages, such as heightened production efficiency, reduced environmental impact, reduced operational costs, and the creation of a more favourable work environment. Beyond fostering industrial progress, such innovations could also make a positive contribution to the sustainable development of forestry.

Forestry operations take place in intricate natural forest environments, rendering the automation of these processes complex and challenging. To realize full automation of forest machines, it is imperative to equip them with environmental awareness and cognitive capabilities. This can be achieved by integrating advanced imaging sensors with sophisticated algorithms, thereby enabling forest machinery to possess a vision system. By utilizing both novel and existing solutions for object detection, positioning, categorization, and status analysis within the machine's surroundings, combined with intelligent decision-making and control mechanisms, forest machinery can attain a higher degree of operational automation.

This thesis primarily focuses on the development, deployment, and validation of vision systems on an unmanned forestry machine platform, with a special emphasis on the prototype development of essential features that are still manually executed in contemporary forestry operations. These features include forest terrain obstacle detection, roundwood pose estimation, automated selection of seedling planting locations, and autonomous obstacle avoidance for mounders.

In this thesis, vision systems rooted in color and stereo camera sensing are designed and deployed on an unmanned forest machinery platform. These systems encompass the following key functionalities:

•             A precise positioning system for detecting forest terrain obstacles, such as stones and stumps, using stereo camera data in conjunction with deep learning techniques.

•             Localization and pose estimation capabilities for ground logs, leveraging stereo cameras and deep learning.

•             An analysis of the planning area for obstacle detection, along with the extraction of feasible planting zones for establishing seedling planting locations.

•             The development of an automated obstacle avoidance system for forestry mounders, powered by visual solutions.

Through rigorous testing in real-world scenarios spanning logging, loading, planting, and site preparation, this paper demonstrates the feasibility and practicality of enhancing the level of automation in forest machinery operations through the integration of vision systems. It culminates in the creation of tangible and efficient functional prototypes, leading towards a new era of automation in the field of forestry.

Place, publisher, year, edition, pages
Luleå: Luleå University of Technology, 2023
Series
Doctoral thesis / Luleå University of Technology 1 jan 1997 → …, ISSN 1402-1544
National Category
Vehicle Engineering
Research subject
Machine Design
Identifiers
urn:nbn:se:ltu:diva-101829 (URN)978-91-8048-420-6 (ISBN)978-91-8048-421-3 (ISBN)
Public defence
2024-01-11, E632, Luleå tekniska universitet, Luleå, 09:00 (English)
Opponent
Supervisors
Available from: 2023-10-30 Created: 2023-10-27 Last updated: 2023-12-08Bibliographically approved

Open Access in DiVA

fulltext(3232 kB)726 downloads
File information
File name FULLTEXT01.pdfFile size 3232 kBChecksum SHA-512
5e8b7feb4d5aa8761e54a61b93521a0aba2a46bb80cd37174472cb96e409ce7581d4ad01dd248bb5c238451423e567adbc395337d1ee4e5c082a8b30247e6243
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Li, SongyuLideskog, Håkan

Search in DiVA

By author/editor
Li, SongyuLideskog, Håkan
By organisation
Product and Production Development
In the same journal
Forests
Forest ScienceComputer Vision and Robotics (Autonomous Systems)

Search outside of DiVA

GoogleGoogle Scholar
Total: 726 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 386 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf