With the advancement of technologies such as the Internet of Things (IoT), Software-defined networks (SDN), 5G or 6G mobile networks, and so on, the complexity of the network is increasing, making it difficult to deploy or perform experiments. A Network Digital Twin (NDT) that can properly replicate the network structure of a system is the focal point of both academic and industrial researchers as they can be used for that purpose, especially artificial intelligent-based NDTs. However, even though deep learning-based NDTs provide better accuracy without increasing computational costs, because of their lack of explainability, they are not properly accepted in most sectors. Thus, a Graph Neural Network-based Network Digital Twin, RouteNet-Fermi has been selected to explore its functionality and usability. After training and testing the existing NDT, some explainable AI (XAI) methods such as Numerical (LIME, and SHAP) as well as Visual explanations have been implemented to explain the outcome of the decision-making process of the NDT. The thesis also focuses on performing an extensive novel Systematic Literature Review on the explainability of NDT to find relevant trends and patterns and future research directions. In addition, by analyzing the input-output features extracted by using XAI methods, the solution also provides valuable recommendations that can be implemented back into the network and can contribute to sustainability