Forestry Crane Automation using Learning-based Visual Grasping Point Prediction

Harald Gietler*, Christoph Böhm, Stefan Ainetter, Christian Schöffmann, Friedrich Fraundorfer, Stephan Weiss, Hubert Zangl

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review


This paper presents an approach to automate the log-grasping of a forestry crane. A common hydraulic actuated log-crane is converted into a robotic device by retrofitting it with various sensors yielding perception of internal and environmental states. The approach uses a learning-based visual grasp detection. Once a suitable grasping candidate is determined, the crane starts its kinematic controlled operation. The system’s design process is based on a real-sim-real transfer to avoid possibly harmful, to humans and itself, crane behavior. Firstly, the grasping position prediction network is trained with real-world images. Secondly, an accurate simulation model of the crane, including photo-realistic synthetic images, is established. Note that in simulation, the prediction network trained on real-world data can be used without re-training. The simulation is used to design and verify the crane’s control- and the path planning scheme. In this stage, potentially dangerous maneuvers or insufficient quality of sensory information become visible. Thirdly, the elaborated closed-loop system configuration is transferred to the real-world forestry crane. The pick and place capabilities are verified in simulation as well as experimentally. A comparison shows that simulation and real-world scenarios perform equally well, validating the proposed real-sim-real design procedure
Original languageEnglish
Title of host publication2022 IEEE Sensors Applications Symposium, SAS 2022 - Proceedings
Pages1 - 6
ISBN (Electronic)9781665409810
Publication statusPublished - 1 Aug 2022
Event17th IEEE Sensors Applications Symposium: SAS 2022 - Hybrider Event, Sundsvall, Sweden
Duration: 1 Aug 20223 Aug 2022


Conference17th IEEE Sensors Applications Symposium
Abbreviated titleSAS 2022
CityHybrider Event, Sundsvall

ASJC Scopus subject areas

  • Instrumentation
  • Electrical and Electronic Engineering
  • Computer Vision and Pattern Recognition

Cite this