ICACT20230243 Slide.25
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
That's all for my presentation. For any question to be raised, I'm quite willing to discuss them with you at any time. Thank you, every one.
|
|
ICACT20230243 Slide.24
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Finally, we summarize the content of the paper. We design a D2D-assisted network task offloading framework that introduces energy harvesting technology and DVFS technology, and uses multi-agent reinforcement learning to make decisions.
In follow-up research, controllable wireless energy transmission can be considered instead of random amount of green energy, so as to improve the stability of the system,
|
|
ICACT20230243 Slide.23
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The fifth figure is the impact of gamma value on performance. Gamma is the discount factor.
It quantifies how much importance we give for future rewards. when the gamma is too small, such as 0.9, it cannot be considered long-term enough. When the gamma value is too high,
such as 0.995, the returns of 100 steps will have bad performance.
|
|
ICACT20230243 Slide.22
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The fourth figure is the impact of energy abundance on performance. It can be seen from the figure that when the mean is low, the device is severely constrained by energy and the performance fluctuates greatly, while when the mean is greater than or equal to 7, the energy is sufficient. , resulting in little performance impact.
|
|
ICACT20230243 Slide.21
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The third figure shows the impact of active devices on algorithm performance. The more active devices, the greater the system load, but the idle devices and edge server devices remain unchanged, so the average task delay will increase.
|
|
ICACT20230243 Slide.20
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The second figure shows the impact of learning rate on algorithm performance, When the learning rate is too high, the neural network cannot converge due to the parameters oscillating out of the valid range. When the learning rate is too low, the algorithm is easy to converge to the local optimum which results suboptimal performance, and the convergence speed is slow.
|
|
ICACT20230243 Slide.19
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The next part is the experimental results. The first is the comparison between our proposed algorithm and other benchmark algorithms. The description of the benchmark algorithm is shown on the left, and the experimental results are shown on the right. It can be seen that our algorithm has excellent performance while fast convergence.
|
|
ICACT20230243 Slide.18
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The figure shows the multi-agent reinforcement learning network architecture, and the local observation is integrated into the global observation through communication between devices, which is used as network input for calculation.
The device makes decisions based on the input, and saves the transition into the replay memory for the subsequent network update stage. When the network is updated, the transition is sampled randomly, the gradient is calculated according to the policy gradient formula, and it is back-propagated to the neural network for update.
|
|
ICACT20230243 Slide.17
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
So We assign independent Actor and Critic networks to each device, and update the network in the same way as the Proximal Policy Optimization algorithm
The PPO algorithm ensures that each update is only performed within a small range through the clip operation to ensure the stability of the network.
|
|
ICACT20230243 Slide.16
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
In order to solve this problem, we need to consider the characteristics of the problem mentioned before: therefore, we utilize the reinforcement learning algorithm based on the principle of trial and error to cope with randomly changing system states and output decisions online.
Facing a huge action space, we refine the subject of decision-making into each device, so that the problem can be solved by using a multi-agent reinforcement learning algorithm.
If we do not use the multi-agent algorithm, the size of the action space will be exponentially related to the number of active devices, but after using the multi-agent algorithm, it will be linear
|
|
ICACT20230243 Slide.15
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Therefore, we first formulate the problem mathematically. We minimize the average task completion delay by making decisions about offload targets and CPU frequency. The three constraints are constraints on offloading decisions, CPU frequency selection, and battery power.
|
|
ICACT20230243 Slide.14
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
After confirming the model, we analyze the problem and find that the problem has the following characteristics:
The system states are time-varying and hard to predict precisely and even unpredictable.
The state of the system changes with time and is unpredictable, so we can only use online algorithms to solve.
Current decisions will have long-term effects in the future, so our algorithms must be far-sighted.
The objective function is non-convex and the control decisions are discrete with large action space.
|
|
ICACT20230243 Slide.13
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The delay we want to optimize is the sum of computation delay and transmission delay.
The update of the remaining energy of the device battery is based on the energy harvesting and the energy consumed in the current time frame, which cannot exceed the upper limit of the battery capacity.
The energy consumed in the current time frame depends on the offloading decision of the queue head task and the CPU frequency
|
|
ICACT20230243 Slide.12
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The transmission model is similar to the computation model, and it also uses an iterative method for computing according to the principle of first-in-first-out.
The transmission rate r is calculated according to Shannon's theorem, which mainly depends on the channel state,
|
|
ICACT20230243 Slide.11
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
For the computation delay, we use δ to represent the waiting time, which is the time from when a task enters the queue to when it starts executing; Use l to represent the completion time, that is, the time required for the task to enter the computing queue and be completed
|
|
ICACT20230243 Slide.10
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Each task is represented by task size and task complexity, Task complexity represents the number of CPU cycles required to process one bit of data.
Each device has a battery and computing queue, where the computing queue is used to represent the tasks performed on the device, and for active devices, it also includes a transmission queue to represent the tasks to be transmitted. Note that edge server does not have battery and is not bound by battery energy.
As shown in the figure, if the offloading decision is minus 1, it means local execution; if the offloading decision is 0, it is offloaded to the edge server; otherwise, it is offloaded to the corresponding idle device.
|
|
ICACT20230243 Slide.09
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Next we represent our system model, first the device model, where we first divide the devices into active devices and idle devices. Each time frame t, the mobile device captures a random amount of energy and saves it in the battery, and the active device generates tasks with a certain probability.
|
|
ICACT20230243 Slide.08
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The main contributions of this paper are as follows: For model, we introduce Dynamic voltage and frequency scaling and energy harvest technology into the task offloading framework in D2D-assisted MEC network to minimize the average task delay under strict energy constraints.
For algorithm: we use a multi-agent reinforcement learning algorithm based on MAPPO to deal with the multi-dimensional discrete action spaces and complex cooperative relationships between devices.
And we have proved the effectiveness and superiority of our algorithm through sufficient experiments.
|
|
ICACT20230243 Slide.07
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Both energy harvesting technology and DVFS technology are to prolong the life of equipment, and energy harvesting technology ensures the continuity of service by capturing green energy.
|
|
ICACT20230243 Slide.06
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Dynamic voltage and frequency scaling (DVFS) is the adjustment of power and speed settings on a computing device’s various processors, controller chips and peripheral devices to optimize resource allotment for tasks and maximize power saving when those resources are not needed.
|
|
ICACT20230243 Slide.05
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The D2D technology can be regarded as a supplement to the mobile edge computing technology, which allows offloading the tasks that the edge server has no time to process to the idle devices in the system, thereby improving the utilization of system resources and reduce server load.
|
|
ICACT20230243 Slide.04
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
First, let’s introduce mobile edge computing, mobile edge computing can offload tasks to edge server closed to the end user and significantly reduce the service response delay.
|
|
ICACT20230243 Slide.03
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
First of all, with the increase of equipment and the demand for computing power of software services, the load in the network continues to increase, so the goal of this paper is to reduce the average delay of tasks under the combination of MEC framework, D2D technology, DVFS technology and energy harvesting technology,
|
|
ICACT20230243 Slide.02
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The following is the outline of this presentation, which is mainly divided into five parts
The first is the introduction part, which introduces the general principle of the technology used and the system model
Second, in the problem formulation part, the problem is expressed mathematically
The algorithm design part explains the main structure of the algorithm
The experimental analysis stage compares the algorithm with the benchmark algorithm and proves its effectiveness
The conclusion section summarizes the paper and gives directions for future research
|
|
ICACT20230243 Slide.01
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Hello, everyone, I am Xin Mi, and I am honored to share this paper with you.
In this paper, we will use the multi-agent reinforcement learning method to solve the problem of task offloading and CPU frequency selection of the D2D-assisted mobile edge computing system with the introduction of energy harvesting module and Dynamic voltage and frequency scaling technology.
|
|
ICACT20230243 Slide.00
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
![[Big slide for presentation]](/upload/2023/0243/Presentation/00/20230243.01.jpg){kind=link}