This article establishes a Constrained Finite Time Optimal Control (CFTOC) design for unknown, discrete-time systems with parametric uncertainty, whose dynamics are identified via Set-Membership Identification (SMI). The control scheme is composed of: a) the SMI module, which identifies the dynamics of the system through a Weighted Recursive Least Squares (WRLS) algorithm, and b) the CFTO controller, which can be online updated based on the SMI provided information, hence, providing the ability to adapt online the overall control framework based on the overall confidence intervals from the SMI. The proposed scheme has been adapted for the case of controlling a Vortex Actuator, based on an Electric Ducted Fan and utilized for wall climbing robotic applications. Simulation results are provided to demonstrate the efficacy of the overall proposed scheme.