.jpg) |
Lets look at how we design the reward signal to guide the contextual bandit policy. The reward function is the most critical part of any reinforcement learning system because it defines what "optimal" means. For BRIGHT, optimal monitoring must balance three competing objectives: accuracy, efficiency, and load balancing. Our reward function is designed to maximize accuracy while minimizing cost and congestion. We penalize the system based on three components. First, the RMSE, or the root mean square error, of the traffic matrix estimate. We want this to be low. Second, the monitoring cost, which is the total number of OpenFlow messages sent. We want this to be low for efficiency. And third, the variance of the monitoring load across all switches. We want this to be low to prevent any single switch from becoming a bottleneck. By combining these three terms into a single negative reward, the policy is forced to find a sweet spot: a strategy that is accurate enough, but also highly efficient and fair in its resource usage. This multi-objective reward design is what allows BRIGHT to achieve high performance without causing control-plane congestion. Now that weve selected a template, we need to execute it using our spatio-temporal scheduler.
|
IEEE/ICACT20260358 Slide.10
[Big Slide]
[YouTube]