Operating Micro Aerial Vehicles (MAVs) in subterranean environments is becoming more and more relevant in the field of aerial robotics. Despite the large spectrum of technological advances in the field, flying in such challenging environments is still an ongoing quest that requires the combination of multiple sensor modalities like visual/thermal cameras as well as 3D and 2D lidars. Nevertheless, there exist cases in subterranean environments where the aim is to deploy fast and lightweight aerial robots for area reckoning purposes after an event (e.g. blasting in production areas). This work proposes a novel baseline approach for the navigation of resource constrained robots, introducing the aerial underground scout, with the main goal to rapidly explore unknown areas and provide a feedback to the operator. The main proposed framework focuses on the navigation, control and vision capabilities of the aerial platforms with low-cost sensor suites, contributing significantly towards real-life applications. The merit of the proposed control architecture is that it considers the flying platform as a floating object, composing a velocity controller on the x, y axes and altitude control to navigate along the tunnel. Two novel approaches make up the cornerstone of the proposed contributions for the task of navigation: (1) a vector geometry method based on 2D lidar, and (2) a Deep Learning (DL) method through a classification process based on an on-board image stream, where both methods correct the heading towards the center of the mine tunnel. Finally, the framework has been evaluated in multiple field trials in an underground mine in Sweden.