Earth-moving machines, such as wheel loaders, are a type of heavy-duty machinery used within the construction industry to perform vital tasks, such as digging, transporting, and mining applications. One of these tasks is the short-loading cycle, where an operator manoeuvres the wheel loader to move material from a pile to the tipping body of a dump truck, through navigation, scooping, and dumping. The short-loading cycle is a repetitive task performed in high quantities, often as part of a larger refinement process, making it interesting for automation.

The main objective of this thesis work is to investigate challenges facing the automation of the short-loading cycle, focusing in particular on subtasks that can be efficiently addressed with deep learning methods. A secondary objective is to examine how alternative development paths, such as scale models, or simulations, can be used to enable data-driven automation of the short-loading cycle, as directly experimenting on real vehicles has a high associated cost when large numbers of timesteps are needed to gather enough data.

To investigate the two objectives, the literature is systematically reviewed to identify research gaps, challenges, and the usage of deep learning techniques. Secondly, a set of deep learning techniques is investigated to address perception and actuation problems identified as challenging and important for the automation of the short-loading cycle.

The investigation of deep learning techniques involves training and validating a realtime object detector neural network to identify key components (wheels, tipping body, and cab) on a scale model dump truck while testing on a real vehicle. This resulted in a localisation and classification degradation of only 14% between the scale model and the real dump truck, with no additional training. In addition, an examination to minimize the annotation workload of humans found that it is possible to decrease the workload by 95% while still retaining similar detection performance by leveraging linear interpolation.

Lastly, this thesis presents an investigation regarding the usage of reinforcement learning for navigation during the short-loading cycle. The results indicate that training the agent in simulation is currently required as the agent obtains the maximum reward after timesteps in the order of millions before being capable of performing the task. The results suggest that the trained agent is capable of bridging the gap between simulation and reality to complete a simplified version of the navigation task during the short-loading cycle.

The experiments presented in this thesis provide proof of concept that indicates deep learning techniques can aid in the realisation of an autonomous solution. Moreover, the results show that development paths allowing for experiments providing large numbers of timesteps can facilitate the practical use of such techniques.