Many algorithms used in machine learning and artificial intelligence rely on exact object identification and recognition as their foundation for efficiency and accuracy. Hardware implementation of such methods, when implemented, serves to boost the reliability and productivity of object detection in a wide range of contexts. Hardware implementation of such an algorithm takes a lot of resources and a huge amount of calculation time. The object detection and recognition process require a collection of complex algorithms and a series of filtering approaches to work beyond the boundary conditions. The YOLOv2 network is superior to filters and complicated algorithms for this problem. The authors of this study propose an enhanced YOLOv2 Network for object recognition and a novel approach for optimising the existing YOLOv2 Network for localization to pinpoint the ROI that can be used to scale down and contain the object's original area. The network is proposed by configuring the existing YOLOv2 with additional convolution layers and dropout layers. The dropout layers are added to reduce the dependency on a single neuron and is an effective way of preventing overfitting of the network. Also, instead of ReLU as the activation function, we are using the Swish activation function which tends to provide better results. By isolating and producing the region of interest (ROI) from the original image, the algorithm was able to significantly cut down on both the number of resources needed and the time needed to complete the task. The proposed work is implemented on an FPGA board (Xilinx Zynq-Z7010 FPGA board), and the dataset is collected and prepared by the authors. Data augmentation is done to enhance the training data to enhance the training data, which results in better trained network. MATLAB is used to demonstrate the feasibility of the work and provide a thorough evaluation of its merits. The results show that the accuracy of the conventional algorithm approach drops to 20–30% once you move outside the boundaries, whereas the accuracy of the proposed work increases to 60–70% and a 15–20% increase in efficiency with proposed network based on YOLOv2. The proposed algorithm is three times as fast as the standard method while using only 35 percent as much technology.