One fundamental problem of content caching in
edge computing is how to replace contents in edge servers
with limited capacities to meet the dynamic requirements of
users without knowing their preferences in advance. Recently,
online deep reinforcement learning (DRL)-based caching methods
have been developed to address this problem ...
One fundamental problem of content caching in
edge computing is how to replace contents in edge servers
with limited capacities to meet the dynamic requirements of
users without knowing their preferences in advance. Recently,
online deep reinforcement learning (DRL)-based caching methods
have been developed to address this problem by learning an
edge cache replacement policy using samples collected from
continuous interactions (trial and error) with the environment.
However, in practice, the online data collection phase is often
expensive and time-consuming, thus hindering the practical
deployment of online DRL-based methods. To bridge this gap,
we propose a novel Agile edge Cache replacement method based
on Offline-online deep Reinforcement learNing (ACORN), which
can efficiently learn an edge cache replacement policy offline
from a training dataset collected by a behavior policy (e.g., Least
Recently Used) and then improve it with fast online fine-tuning.
We also design a specific convolutional neural network structure
with multiple branches to effectively extract content popularity
knowledge from the dataset. Experimental results show that the
offline policy generated by ACORN outperforms the behavior
policy by up to 38%. Through online fine-tuning, ACORN also
achieves the number of cache hits as good as that of several
advanced DRL-based methods while significantly reducing the
number of training epochs by up to 40%.
