The industrial cyber-physical system enables collaboration between distributed nodes across industrial clouds and edge devices. Flexibility and interoperability could be enhanced significantly by introducing the service-oriented architecture to industrial edge devices. From the industrial edge computing perspective, software components shall be dynamically composed across heterogeneous edge devices to perform various functionalities. In this paper, a knowledge-driven Microservice-based architecture to enable plug-and-play software components is proposed for industrial edges. These software components can be dynamically configured based on the orchestration of microservices with the support of the knowledge base and the reasoning process. These semantically enhanced plug-and-play microservices could provide rapid online reconfiguration without any programming efforts. The use of the plug-and-play software components is demonstrated by an assembly line example.