Modern robots are used in many exploration, search and rescue applications nowadays. They are essentially coordinated by human operators and collaborate with inspection or rescue teams. Over time, robots (agents) have become more sophisticated with more autonomy, operating in complex environments. Therefore, the purpose of this paper is to present an approach for autonomous multi-agent coordination for exploring and covering unknown environments. The method we suggest combines reinforcement learning with multiple neural networks (Deep Learning) to plan the path for each agent separately and achieve collaborative behavior amongst them. Specifically, we have applied two recent techniques, namely the target neural network and the prioritized experience replay, which have been proven to stabilize and accelerate the training process. Agents should also avoid obstacles (walls, objects, etc.) throughout the exploration without prior information/knowledge about the environment; thus we use only local information available at any time instant to make the decision of each agent. Furthermore, two neural networks are used for generating actions, accompanied by an extra neural network with a switching logic that chooses one of them. The exploration of the unknown environment is conducted in a two-dimensional model (2D) using multiple agents for various maps, ranging from small to large size. Finally, the efficiency of the exploration is investigated for a different number of agents and various types of neural networks.