Control of a Realistic Wave Energy Converter Model using Least-Squares Policy Iteration
IEEE Transactions on Sustainable Energy
Institute of Electrical and Electronics Engineers (IEEE)
(c) 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.
An algorithm has been developed for the resistive control of a non-linear model of a wave energy converter using least-squares policy iteration, which incorporates function approximation, with tabular and radial basis functions being used as features. With this method, the controller learns the optimal PTO damping coefficient in each sea state for the maximization of the mean generated power. The performance of the algorithm is assessed against two on-line reinforcement learning schemes: Q-learning and SARSA. In both regular and irregular waves, least-squares policy iteration outperforms the other strategies, especially when starting from unfavourable conditions for learning. Similar performance is observed for both basis functions, with a smaller number of radial basis functions underfitting the Q-function. The shorter learning time is fundamental for a practical application on a real wave energy converter. Furthermore, this work shows that least-squares policy iteration is able to maximize the energy absorption of a wave energy converter despite strongly non-linear effects due to its model-free nature, which removes the influence of modelling errors. Additionally, the floater geometry has been changed during a simulation to show that reinforcement learning control is able to adapt to variations in the system dynamics.
This work was supported partly by the Energy Technologies Institute and the Research Councils Energy Programme (grant EP/J500847/), partly by the Engineering and Physical Sciences Research Council (grant EP/J500847/1), and partly by Wave Energy Scotland.
This is the author accepted manuscript. The final version is available from Institute of Electrical and Electronics Engineers via the DOI in this record.
Date of Publication: 25 April 2017