 ICACT20240248 Slide.00
[Big Slide]
|
Chrome  Click!! |
 |
Hey everyone! I am Hung Tse Chiang, and I come from the National Taiwan University of Science and Technology. I am excited to share our work with you all today. Our research is Pitching-Motion: Pose-Based Pitch Trajectory Overlay System. |
| ICACT20240248 Slide.01
[Big Slide]
|
Chrome Click!! |
 |
And this is today's outline. This presentation can be separated into the following sections: Introduction, Strike Zone Detection, Baseball Pitch Overlay, Experiment, and Conclusion.
|
| ICACT20240248 Slide.02
[Big Slide]
|
Chrome Click!! |
 |
In the first section, I will briefly introduce the problems the baseball system faces so you can understand more clearly why we are trying to solve the problem.
|
| ICACT20240248 Slide.03
[Big Slide]
|
Chrome Click!! |
 |
Present trajectory analysis systems integrate pitching trajectories with the electronic strike zone for clarity. The electronic strike zone is built by two high-speed cameras. The system can get the 3d position of the space. But with convenience, the electronic strike zone is fixed. The system doesn't need to measure the hitter's height. Then there is a problem behind it. In a baseball game, there will be a lot of different hitters. According to baseball rules, the strike zone might vary because of the hitter's height. So, the electronic strike zone used nowadays is against baseball rules, and our research has aimed to address this problem by designing a system that can detect the hitter's height and vary the strike zone. The implementation details will be covered in the next section. |
| ICACT20240248 Slide.04
[Big Slide]
|
Chrome Click!! |
 |
The second section will explain the details of Strike Zone Detection, including the baseball rules, how we detect hitter and confirm that hitter is ready to hit.
|
| ICACT20240248 Slide.05
[Big Slide]
|
Chrome Click!! |
 |
First, start by figuring out what the strike zone is exactly. So, we can check out the Major League Baseball (MLB) website, they have shown the rules of the strike zone. According to them, the strike zone should change depending on how tall the hitter is, specifically considering their shoulders, waist, and knees. So, the lower part of the strike zone matches the height of the hitter's knees. And the upper boundary lines up with the midpoint between a hitter's shoulders and waist. Finally, the width of the strike zone is determined by the width of the home plate.
|
| ICACT20240248 Slide.06
[Big Slide]
|
Chrome Click!! |
 |
This research uses YOLO-Pose to detect human poses in images. We can obtain the hitter's strike zone by identifying critical skeletal points in the human body.
This is a TV broadcast screenshot of baseball game, and we recognize the human body poses skeletal points, including pitcher, hitter, catcher, and umpire. In the TV broadcast screenshot, since the pitcher's posture faces away from the camera, the nose can be used to exclude the pitcher's skeletal point. Considering that the hitter usually exists in the outermost region among the hitter, catcher, and umpire, this research utilizes the distances between skeletal points to determine those at the outermost edge. By comparing the heights of the detected outermost individuals' nose or ear coordinates, it is possible to identify which skeletal point belongs to the hitter.
|
| ICACT20240248 Slide.07
[Big Slide]
|
Chrome Click!! |
 |
After determining the hitter's skeletal point, including the hitter's shoulders, the waist, and knees, the next step is to identify the timing to draw the strike zone, and this step is crucial.
Many hitters tend to lift their front knee while hitting, which will affect the strike zone by the baseball rules. If we choose hitter-hit timing to generate a strike zone, this can result in a narrower strike zone, making it more challenging to determine strikes accurately. The differences in knee positioning can lead to variations in drawing the strike zone. This inconsistency could cause unfairness in the game. Therefore, we need to determine which timing is more suitable based on their standing position and the lowest point of their knee.
In this research, we divide the process before the pitcher releases the baseball into two key moments, namely, the Release Frame and the Prepare Frame. The Release Frame is when the pitcher is about to release or has just released the baseball, and the hitter has already started preparing for the hitting. We use YOLOv7 to determine the exact timing of the Release Frame. When the pitcher is preparing to release the ball, he will grab the ball in his hand so YOLOv7 can not detect it. Once YOLOv7 detects the ball, it means that the ball is out of the pitcher's hand, so we can rely on this phenomenon and determine when it is Release Frame.
|
| ICACT20240248 Slide.08
[Big Slide]
|
Chrome Click!! |
 |
The Prepare Frame is defined in the 20 frames preceding the Release Frame. We will find which frame where the hitter's knee position is the lowest in those 20 frames.
This figure represents the process of finding the Prepare Frame. First, we determine the Release Frame. We will look back for 20 frames and search for which frame has the lowest knee position. When the hitter is a right-handed hitter, we will check his left knee and vice versa. This Prepare Frame moment represents the hitter in a standing position and provides more accurate information about the positions of the hitter's shoulders, waist, and their knees. Once the Prepare Frame is detected, the upper and lower boundaries of the strike zone are defined. The width of the strike zone is determined manually by marking the width of the home plate. This strike zone, customized based on the hitter's height, provides a more accurate basis for determining a good pitch.
|
| ICACT20240248 Slide.09
[Big Slide]
|
Chrome Click!! |
 |
The third section will introduce the algorithm that detects baseball positions. We can find all ball trajectories in the same game with the algorithm. Once we get the trajectories, we can visualize them and provide a tool to baseball players and coaches that they can use as an analysis tool.
|
| ICACT20240248 Slide.10
[Big Slide]
|
Chrome Click!! |
 |
Because detecting the ball's position will have noise caused by camera or image processing, or the chosen object detected algorithm could go undetected to the ball. We use the Kalman Filter algorithm to solve this problem. Kalman Filter can deal with this noise problem and help us to get a more accurate ball's position. The operation of the Kalman filter involves estimating the system's state by combining the system model and observed data. It consists of two main phases, which are the prediction phase and the update phase. In the prediction phase, the current state is predicted based on the previous moment's state as a reference. The predicted state is corrected in the update phase based on the current observed values. Through this recursive process, the accuracy of the estimation continuously improves. In this figure, we represent the Kalman Filter's main idea. First, we have the previous state called x k minus one. Next, we use the system model equation to predict the current state called x k negative. Then, we are in the update phase. We will use the current measurement to update our state so we can get a more accurate state called x k. This is the basic principle of the Kalman Filter, and the equation of Kalman Filter will be introduced in the next page. |
| ICACT20240248 Slide.11
[Big Slide]
|
Chrome Click!! |
 |
These are Kalman Filter equations. In those equations, we can focus on the covariance; this is a measure of the estimated uncertainty of the prediction state. Next, we can look at the Kalman Gain; the Kalman gain is the weight given to the measurements and predictions. We use Kalman gain to determine which is more reliable. With a high gain, the filter places more weight on the most recent measurements and thus conforms to them. With a low gain, the filter conforms to the predictions more closely. |
| ICACT20240248 Slide.12
[Big Slide]
|
Chrome Click!! |
 |
This section will introduce the experiment results, including the dataset we use and how we measure our system's accuracy and performance.
|
| ICACT20240248 Slide.13
[Big Slide]
|
Chrome Click!! |
 |
The dataset utilized in this research is MLB-YouTube, which includes 20 baseball game videos from the 2017 MLB postseason. In this research, we extracted videos featuring successful pitches and batters executing hitting actions. So, the dataset we use is one hundred one videos, and we crop those videos into fourteenth thousand three hundred and seventy-one. |
| ICACT20240248 Slide.14
[Big Slide]
|
Chrome Click!! |
 |
This research focuses on defining the strike zone using how the body is in a two-dimensional plane, similar to tasks where objects are identified in pictures. Just like the object detection task, we have decided to use Intersection over Union (IoU) as the metric to evaluate our research.
IoU's calculation involves two parameters: the first is the ground truth, which is manually annotated in the dataset. The second is the target range derived from the algorithm used in the experiment. IoU is computed by measuring the overlap between these target ranges and ground truth, then dividing it by the union to obtain the IoU value. This equation is how we calculate the IoU metric, so we estimate the area of overlap and divide it by the area of union. This metric reflects the correlation between the ground truth and predictions. This research also tested the accuracy of the electronic strike zone used nowadays. I will show you the results of these two metrics and compare them to see which one is more accurate.
|
| ICACT20240248 Slide.15
[Big Slide]
|
Chrome Click!! |
 |
The ground truth we used in this research is achieved by manually drawing the strike zone following baseball rules in the Prepare frame. Then, this manually drawn strike zone was compared with the strike zone computed using the Yolo-Pose and the electronic strike zones to calculate the IoU. The electronic strike zones used nowadays rely on synchronized images captured by two high-speed cameras, which are used to calculate the 3D coordinates of the strike zones through triangulation techniques. The compared result is shown in this table. We can see that Our Yolo-Pose strike zone is more accurate than the electronic strike zone used nowadays. Yolo-Pose strike zone's IoU is zero point eight five four one, and the electronic strike zone used nowadays is zero point seven four seven three. Yolo-Pose has a fourteenth point two nine percent improvement. |
| ICACT20240248 Slide.16
[Big Slide]
|
Chrome Click!! |
 |
This page is the result of the system. When we detect the pitcher release baseball, we can get the Release Frame. And then, we can use the previous frames to find which one is the Prepare Frame. The next step is using the Prepare Frame's hitter critical skeletal points to draw our strike zone; we can see this directly in the figure. We detect baseball trajectory using the Kalman Filter in whole games, and we can use the same pitcher's video clip in the same game to generate the overlapping trajectory. We chose one of the pitcher's clips and gave it a high weight so that it would not be more transparent than others. The other will be transparent so they can be overlapped to generate the baseball trajectory overlapping video. This overlapping of baseball trajectory can let baseball players and coaches use more convenience. |
| ICACT20240248 Slide.17
[Big Slide]
|
Chrome Click!! |
 |
The final section is the conclusion.
|
| ICACT20240248 Slide.18
[Big Slide]
|
Chrome Click!! |
 |
Our research proposes the YOLO-Pose Strike Zone, which addresses the issue of the Electronic Strike Zone not conforming to baseball rules in practical applications.The YOLO-Pose Strike Zone achieved an IoU score of 0.8541, marking a 14.29% improvement in accuracy over the Electronic Strike Zone.Considering viewer experience and practicality, our research's overlapping trajectories offer an intuitive way to visualize differences and their connection with the YOLO-Pose Strike Zone.
|
| ICACT20240248 Slide.19
[Big Slide]
|
Chrome Click!! |
 |
Thank you for listening to my presentation. I appreciate the opportunity to share my research with you all. Thank you.
|
