Reinforcement Learning for Energy Storage Systems in Grid-Connected Microgrids: An Investigation of Online versus Offline Implementation
dc.contributor.author | Ali, KH | |
dc.contributor.author | Sigalo, M | |
dc.contributor.author | Das, S | |
dc.contributor.author | Anderlini, E | |
dc.contributor.author | Tahir, AA | |
dc.contributor.author | Abusara, M | |
dc.date.accessioned | 2021-09-07T09:38:01Z | |
dc.date.issued | 2021-09-09 | |
dc.description.abstract | Grid-connected microgrids consisting of renewable energy sources, battery storage, and load, require an appropriate energy management system that controls the battery operation. Traditionally, the operation of the battery is optimised using 24-hours of forecasted data of load demand and renewable energy sources (RES) generation using offline optimisation techniques, where the battery actions (charge/discharge/idle) are determined before the start of the day. Reinforcement Learning (RL) has recently been suggested as an alternative to these traditional techniques due to its ability to learn optimal policy online using real data. Two approaches of RL have been suggested in the literature viz. offline and online. In offline RL the agent learns the optimum policy using predicted generation and load data. Once convergence is achieved, battery commands are dispatched in real-time. This method is similar to traditional methods because it relies on forecasted data. In online RL, on the other hand, the agent learns the optimum policy by interacting with the system in real time using real data. This paper investigates the effectiveness of both the approaches. White Gaussian noise with different standard deviations was added to real data to create synthetic predicted data to validate the method. In the first approach, the predicted data was then used by an offline RL algorithm. In the second approach, the online RL algorithm interacted with real streaming data in real time and the agent was trained using real data. When energy costs of the two approaches were compared, it was found that the online RL provides better results than the offline approach if the difference between real and predicted data is greater than 1.6%. | en_GB |
dc.description.sponsorship | Engineering and Physical Sciences Research Council (EPSRC) | en_GB |
dc.identifier.citation | Vol. 14 (18), article 5688 | en_GB |
dc.identifier.doi | 10.3390/en14185688 | |
dc.identifier.grantnumber | EP/T025875/1 | en_GB |
dc.identifier.uri | http://hdl.handle.net/10871/126995 | |
dc.language.iso | en | en_GB |
dc.publisher | MDPI | en_GB |
dc.rights | © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). | |
dc.subject | Reinforcement learning (RL) | en_GB |
dc.subject | microgrid | en_GB |
dc.subject | battery management | en_GB |
dc.subject | offline and online RL | en_GB |
dc.subject | Optimisation | en_GB |
dc.title | Reinforcement Learning for Energy Storage Systems in Grid-Connected Microgrids: An Investigation of Online versus Offline Implementation | en_GB |
dc.type | Article | en_GB |
dc.date.available | 2021-09-07T09:38:01Z | |
dc.identifier.issn | 1996-1073 | |
dc.description | This is the final version. Available on open access from MDPI via the DOI in this record | en_GB |
dc.identifier.journal | Energies | en_GB |
dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ | en_GB |
dcterms.dateAccepted | 2021-09-07 | |
exeter.funder | ::Engineering and Physical Sciences Research Council (EPSRC) | en_GB |
rioxxterms.version | VoR | en_GB |
rioxxterms.licenseref.startdate | 2021-09-07 | |
rioxxterms.type | Journal Article/Review | en_GB |
refterms.dateFCD | 2021-09-07T08:13:15Z | |
refterms.versionFCD | AM | |
refterms.dateFOA | 2021-09-17T15:19:24Z | |
refterms.panel | B | en_GB |
Files in this item
This item appears in the following Collection(s)
Except where otherwise noted, this item's licence is described as © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).