ICACT20250091 Slide.24
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
This is my report of our paper. Thanks a lot for listening.
|
|
ICACT20250091 Slide.23
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
This is the references.
|
|
ICACT20250091 Slide.22
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Finally, here comes the conclusion. To improve the survival rate of sensor nodes and energy usage efficiency, we have proposed a temporal spatial partial charging scheme for WRSN with multiple mobile chargers in this paper. Our charging strategy, termed TSPC-DRL, automatically determines the charging duration of the target charging node. Our proposed scheme has promising performance in prolonging the survival rate of the sensor node and improving the energy usage efficiency of MC compared with the baseline algorithms, which has been verified by the simulation experiments. Our future research will focus on designing a multi-node charging scheme for multiple MCs in WRSN.
|
|
ICACT20250091 Slide.21
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Figure (c) explores how sensor battery capacity affects algorithm performance, revealing that larger capacities increase sensor node survival rates by extending lifespan, decreasing charging demands, and alleviating mobile charger load. Table IV demonstrates that enhanced battery capacity improves energy usage efficiency, with TSPC-DRL outperforming other algorithms in this regard.
|
|
ICACT20250091 Slide.20
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In Figure (b), we discuss the impact of the moving speed of mobile chargers on the performance of different algorithms in a network with 100 sensor nodes. Changing mobile charger speed from 1m/s to 5m/s enhances the sensor node survival rate by approximately 5% in TSPC-DRL. However, increasing the movement speed of mobile chargers does not significantly improve the node survival rate for other algorithms. Table III reveals that despite no improvement in energy efficiency with increased charger speed, TSPC-DRL maintains superior energy efficiency over other algorithms.
|
|
ICACT20250091 Slide.19
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
We first study the performance of different algorithms by increasing the network scale. Figure (a) shows sensor survival rates declining as sensor number rise from 100 to 300, attributed to increased mobile charger workload. TSPC-DRL outperforms others in survival rates. Table II indicates TSPC-DRL's energy efficiency at approximately 0.33, surpassing FLCSD (0.31), mTS (0.29), and IOCS (0.27).
|
|
ICACT20250091 Slide.18
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
After that, we conducted some experiments to test our algorithm in this section. We will start with the simulation parameter. In the experiment, 100 to 300 sensor nodes are randomly deployed in a 100m×100m area, with the base station at the center. All sensors start with a fully rechargeable 1000J battery, consuming energy at a rate of 0.2J/s to 0.5J/s. Mobile chargers, equipped with high-capacity batteries, are initially stationed at the base station. Simulation parameters are detailed in Table I. To evaluate the performance of the proposed TSPC-DRL algorithm, we compare it with the following charging algorithms: mTS, IOCS, and FLCSD. These algorithms are full charging algorithms. We modified the algorithms which limit the charging duration to no more exceed 𝜏.
|
|
ICACT20250091 Slide.17
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
With an objective function for policy improvement and a loss function for correcting the state-value function, the gradient of these functions enable gradient ascent on 𝜃 and gradient descent on 𝜙.
|
|
ICACT20250091 Slide.16
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
A mini-batch of transition is randomly sampled from the replay buffer, and the policy 𝜋_𝜃 is updated with the objective function 18. To learn the state-value function, a commonly used approach is to minimize a loss function based on mean squared error which can be represented by eqution 19.
|
|
ICACT20250091 Slide.15
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In the PPO update phase, each transition in the replay buffer is retrieved. We use the GAE to estimate the advantage. The advantage can be calculated by equation 16. The target value 𝑉_𝑡^𝑡𝑎𝑟𝑔𝑒𝑡 can be calculated by equation 17.
|
|
ICACT20250091 Slide.14
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
We train the agent with PPO algorithm. PPO is a policy-based algorithm under the Actor-Critic framework. In our charging duration strategy learning algorithm, the charging duration calculation policy 𝜋_𝜃 is determined by the actor network. The state value function 𝑉_𝜙 (𝑠_𝑡) which estimates the expected cumulative reward for a given state 𝑠_𝑡 can be calculated by critic network. During the environment interaction, MC observe current state 𝑠_𝑡 and interacts with the environment to execute the task, receiving a reward 𝑟_𝑡 (𝑠_𝑡,𝑎_𝑡). The agent observes the environment to obtain new state 𝑠_(𝑡+1) , and the transition (𝑠_𝑡,𝑎_𝑡,𝑟_𝑡,𝑠_(𝑡+1)) is stored in the replay buffer.
|
|
ICACT20250091 Slide.13
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Action: 𝑎_𝑡 denotes that the mobile charger 𝑤_𝑘 charges the target sensor node s_target for 𝑎_𝑡 seconds. Reward Function: The reward function represents the benefits the agent can acquire after taking action 𝑎_𝑡 in the current state 𝑠_𝑡 . To enhance survival rate and energy usage efficiency, we designed the reward function by equation 14.
|
|
ICACT20250091 Slide.12
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
After selecting the target charging node, a mobile charger needs to schedule the charging duration at this chosen node. We propose a charging duration calculation algorithm based on deep reinforcement learning. The mobile charger is an agent that interacts with the environment. In this slide, we introduce how to construct a learning environment, including the state, action, and reward function for training the agent. State: The environment state, denoted as <𝑆_𝑛𝑒𝑡𝑤𝑜𝑟𝑘^𝑘, 𝑆_𝑡𝑎𝑟𝑔𝑒𝑡^𝑘, 𝑆_𝑀𝐶^𝑘>, comprises three components: the sensor network state, the target charging node state, and the mobile charger state. The sensor network state, represented as 𝑆_𝑛𝑒𝑡𝑤𝑜𝑟𝑘^𝑘={𝑆_1,𝑆_2,⋯,𝑆_ℎ }, encompasses all sensor nodes serviced by the mobile charger 𝑤_𝑘. 𝑺_𝒊 is a five-tuple that contains: the coordinates of the node 𝒔_𝒊 , its battery capacity, energy consumption rate, and current residual energy. 𝑆_𝑡𝑎𝑟𝑔𝑒𝑡^𝑘 comprises the same five parts of information for the target charging node. 𝑆_𝑀𝐶^𝑘 is a seven-tuple which describes the state of mobile charger 𝑤_𝑘: its current coordinates, residual energy, battery capacity, moving energy consumption rate, charging energy consumption rate, and moving speed.
|
|
ICACT20250091 Slide.11
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
For the purpose of balancing the workload of each mobile charger, we proposed a sensor cluster algorithm to divide the 𝒏 sensors on the plane into 𝒎 clusters. Each cluster is assigned a mobile charger for responding charging requests within the cluster. Now, we introduce how to select the target charging node. Each request in the charging queue corresponds to a priority. If the queue is not empty, the mobile charger needs to select a charging request with the highest priority for response. The priority of a charging request consists of temporal priority and spatial priority. These two parts of priority are calculated by lambda_t and lambda_s. The priority of the charging request can be calculated by equation 12.
|
|
ICACT20250091 Slide.10
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Here is the problem formulation. Given a WRSN with multiple MCs, our objective is to schedule charging tasks for MCs (i.e. determine which sensor should be charged and corresponding charging duration) so as to maximize survival rate and energy usage efficiency. This problem can be formulated by following equations.
|
|
ICACT20250091 Slide.09
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The energy cost of MCs consist of two parts: the energy consumed by MCs moving between charging positions and the energy consumed for charging the sensors. To ensure that energy-limited MCs transmit more energy to sensors, an effective charging scheme should consider how to make full use of the battery energy of MCs. To measure the energy usage efficiency, the metric 𝜂 is defined by equation 4.
|
|
ICACT20250091 Slide.08
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
And now, we talk about the problem formulation. In a wireless sensor network, if sensors run out of energy and is unable to sense or transmit data, it will seriously impair system performance. Therefore, the survival rate of sensor nodes is a crucial performance metric of WRSN. To measure the survival rate, the metric δ is defined by equation 3.
|
|
ICACT20250091 Slide.07
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
During the process of charging the battery, the energy gained by the sensors is not at a stable rate but instead follows a pattern from fast to slow. The charging process where the sensor 𝑠_𝑖 gets energy from the MC can be calculated by following equation. The charging process reveals that the charging efficiency has a marginal effect. As the charging process continues, the charging speed will gradually slow down.
|
|
ICACT20250091 Slide.06
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Subsequently, the MC selects a request and moves to 𝒔_𝒋 and charges the sensor for 𝑡_𝑖 seconds. This process is referred to a charging task, denoted by 𝑘_𝑖. The energy required for MC to perform the charging task 𝑘_𝑖 and return to BS is calculated by equation 1. If residual energy of mobile charger re_w is lower than 𝒆_(𝒌_𝒊 ) , mobile charger will not executing 𝒌_𝒊 and return to the BS for replenishing energy.
|
|
ICACT20250091 Slide.05
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In this part, we define the system model. As shown in Figure 1, in a wireless rechargeable sensor network, there are 𝑚 wireless mobile chargers and 𝑛 stationary sensor nodes deployed in the two dimensions area. The sensors consume battery energy for sensing and transmitting data. When the residual lifetime falls below threshold Θ, sensor will send a charging request to the base station. After receiving a charging request, base station inserts the request into MC’s charging queue.
|
|
ICACT20250091 Slide.04
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In summary, Existing partial charging schemes typically use fixed time intervals or predetermined battery levels, leading to inefficiencies such as sensor death from inappropriate charging duration and increased waiting times. To address these limitations, In this paper, we introduce a deep reinforcement learning-based algorithm that dynamically optimizes charging duration allocation, enhancing network performance.The main contributions of this work are summarized as follows: First, the charging scheduling problem is modeled as a multiple-objective joint optimization problem, which aims at simultaneously maximizing survival rate and energy usage efficiency. Second, the proposed algorithm based on deep reinforcement learning will train a deep neural network that schedules mobile chargers automatically allocate an appropriate charging duration to the sensors. Third, We conducted the experiment showing the superiority of our algorithm compared to existing on-demand methods in several critical performance metrics.
|
|
ICACT20250091 Slide.03
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
First, let’s start with the background of this paper. Recent advancements in wireless power transfer technology offer a promising solution for wirelessly powering sensors. In a wireless rechargeable sensor network (WRSN), mobile chargers are capable of moving and wirelessly recharging the sensors. Additionally, the sensors are equipped with receivers that capture the energy transmitted by the MCs. The WRSN ensures reliable and controllable energy replenishment, facilitating the sustained operation of the sensor network. Aiming to charge more nodes and further improve survival rate, more and more researchers proposed partial charging algorithms. Xu et al. proposed a partial charging scheme to increase sensor survival rate. However, in their work, sensors are always replenished by a fixed unit power each time. Guo et al. proposed a partial charging scheme where sensor nodes are charged to their 80% energy level. To achieve the sum of charging rewards collected from all charged sensors by the mobile charger per tour maximized, Liang et al. determined the energy level for partial charging using the residual energy of the sensor nodes.
|
|
ICACT20250091 Slide.02
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
My presentation is organized by following sections.
|
|
ICACT20250091 Slide.01
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Hello, everyone! My name is Tingwei Li, from College of Mathematics and Informatics, South China Agricultural University, China. It's my honor to be here to share you with my paper. The title of my article is A DRL-Based Partial Charging Scheme for Multiple Mobile Chargers for Maximizing Survival Rate and Energy Usage Efficiency.
|
