 |
We train the agent with PPO algorithm. PPO is a policy-based algorithm under the Actor-Critic framework. In our charging duration strategy learning algorithm, the charging duration calculation policy 𝜋_𝜃 is determined by the actor network. The state value function 𝑉_𝜙 (𝑠_𝑡) which estimates the expected cumulative reward for a given state 𝑠_𝑡 can be calculated by critic network. During the environment interaction, MC observe current state 𝑠_𝑡 and interacts with the environment to execute the task, receiving a reward 𝑟_𝑡 (𝑠_𝑡,𝑎_𝑡). The agent observes the environment to obtain new state 𝑠_(𝑡+1) , and the transition (𝑠_𝑡,𝑎_𝑡,𝑟_𝑡,𝑠_(𝑡+1)) is stored in the replay buffer.
|
IEEE/ICACT20250091 Slide.14
[Big Slide]