The interoperability between various automation systems is considered as one of the major character of future automation systems. Service-oriented Architecture is a possible interoperability enabler between legacy and future automation systems. In order to prove the interoperability between those systems, a verification framework is essential. This paper proposes a configurable cloud-based validation environment for interoperability tests between various distributed automation systems. The testing framework is implemented in a multi-layer structure which provides automated closed-loop testing from the protocol level to the system level. The testing infrastructure is also capable for simulating automation systems as well as wireless sensor networks in the cloud. Test cases could be automatically generated and executed by the framework.