 ICACT20210265 Slide.01
[Big Slide]
|
Chrome  Click!! |
 |
Good morning, chairman, ladies and gentlemen,
It is my pleasure to show you our study. The title is “Robot for Ball Fetch-and-Carry with
Computer Vision in Deep Learning” The authors are Limin Su*, Chao-Jen Huang*, Mei Liu*, Bing-Yuh Lu*, Jidong Che*, Ximiao Wang*, Jiqi Feng*, Changyong Li*, Youhong Feng* with *Faculty of Automation, Guangdong University of Petrochemical Technology, and Yueliu Mo with Faculty of Business administration, Guangzhou College of Technology and Business, Guangdong, China.
|
| ICACT20210265 Slide.02
[Big Slide]
|
Chrome Click!! |
 |
The first part is abstract! |
| ICACT20210265 Slide.03
[Big Slide]
|
Chrome Click!! |
 |
What hashed tenses are! Employing the modern AI technique, a robot can serve as a cleaner to fetch the balls and make the tennis court in order. Based on the idea, we developed a robot for ball fetch-and-carry with computer vision in deep learning. The robot which functioned as fetch-and carry ball sever was implemented. The structure of this robot are (1) Raspberry Pi for processor; (2) a camera for computer vision; (3) a 4-wheel model car for carry; and (4) robot arms for the mechanism of fetching ball.
|
| ICACT20210265 Slide.04
[Big Slide]
|
Chrome Click!! |
 |
The computer vision in deep learning technology makes the robot pick the balls quickly. For the safety, the navigation and anti-collision modules were embedded in the robot. The results presented that the number of successful fetch-and-carry is in the range of 69 to 88 times in the competition of working in 100 minutes to pick the table tennis balls. Therefore, the technology can be developed to be a fetch-and-carry server for other balls in the near future.
|
| ICACT20210265 Slide.05
[Big Slide]
|
Chrome Click!! |
 |
Next, I will talk about the introduction of this study.
|
| ICACT20210265 Slide.06
[Big Slide]
|
Chrome Click!! |
 |
According to the "Thirteenth Five-Year Plan for the Development of the Sports Industry in China" issued by National Sports General Administration, China's total output value of the sports industry is more than 2.600 billion yuan in 2019, and will be expected to reach 3.5 trillion yuan by 2023. Behind China's developed sports industry, there is bound to be an urgent demand for sports equipment . Notably, with the development of science and technology, this demand of the sports will shift to a higher technological content in the world. The smarter devices will be increased and widely used in many sports. |
| ICACT20210265 Slide.07
[Big Slide]
|
Chrome Click!! |
 |
For educational purpose, many robot competitions have been token place for students’ training of the techniques of robotics. One of the competitions is to design a robot for fetch-and-carry balls. The educators and students need to focus on the design and implementation of programming microprocessors, sensors, robotics, navigation technology, computer vision, 3D printing and even cover design. Therefore, many institutes and companies support budgets and technologies to hold on the robotic competitions, such as SICK Robot Day, TDK Cup and so on. |
| ICACT20210265 Slide.08
[Big Slide]
|
Chrome Click!! |
 |
Furthermore, ICACT has promoted the modern technology such as IT, IoT and so on for many years. These modern technologies have been widely employed in many industries. Let’s check the sports industry. The problem is “What is the vision of the future sports industry?” Based on the aforementioned modern technologies, the proposed solutions include IT, IoT, AI, AR, VR, Robot and so on. The new ideas will change the limitations of playing, practicing, and industry of many sports. For examples, practicing golf in VR makes the game at home possible. It contains the components of IT, IoT and VR. Therefore, we proposed the robot for ball fetch-and-carry with computer vision in deep learning to pick the table tennis balls up.
|
| ICACT20210265 Slide.09
[Big Slide]
|
Chrome Click!! |
 |
Next, we introduce the methods. |
| ICACT20210265 Slide.10
[Big Slide]
|
Chrome Click!! |
 |
The functional block diagram of the robot for fetch-and-carry balls is presented in Fig. 2. The devices are introduced as follows,
A. Micro controller
Raspberry Pi systems-on-chip was employed to be the central processor of the robot. The Raspberry Pi 4, with a quad-core ARM Cortex-A72 processor, full gigabit Ethernet, and available with 2, 4 or 8GB of RAM. The video controller can generate standard modern TV resolutions, such as HD and Full HD [10-15].
B. Camera for computer vision
The computer vision is based on the deep learning of the artificial neuron network [16-19]. The number of the picture for training is 2000. Some of the training samples are presented in Fig. 3. The server is TensorFlow provided by Alibaba Cloud. The computer vision in the TesorFlow has been implemented in deep learning to analysis more than 10,000 pictures. The camera of the robot is responsible for taking pictures and obtaining images of the training target and then connecting with the server in the Internet to upload the pictures. Some of the training results are embedded in the Raspberry Pi to recognize the balls. However, if some cases can not recognize the ball, it will sent to the TensorFlow provided by Alibaba Cloud for recognition and training.
C. 4-wheel model car for carry
2 tank-type tracked wheels with the motor drive module were employed for the movable carrier of the electric heating radiator. The carrier was controlled by the IO port of STM32 G0. The key chip of motor drive module was L298N which was functioned by dual H-bridge. The range of power supply is 5 to 35V (dc). The maximum output current is 2A (the instantaneous peak current is 3A), and the maximum output power is 25W. The speeds of the wheels were tuned by pulse width modulation (PWM) technology where the duty cycles of the driving pulses were modulated by timers.
D. Robot arms for the mechanism of fetching ball
The robot arms for mechanism for fetching ball were designed by 3D printing [20-21]. The arms with several jokes and the notches can fetch the ball very stably. The camera is located on the centre of the beginning of arms to catch a ball using the computer vision. This design is aimed at the easier control of the robot arms.
|
| ICACT20210265 Slide.11
[Big Slide]
|
Chrome Click!! |
 |
The image processing is based on binary images. The contour and texture of the ball can be extracted the features. Compared with the database, the classification of the category of balls can be performed. The computation of the distances of the relative geometric values are decided by the pixels. Finally, it returns three sets of data of "category", "distance" and "X-Y axis".
|
| ICACT20210265 Slide.12
[Big Slide]
|
Chrome Click!! |
 |
For testing the performance of the robot, an experiment was design. The test is based on picking up table tennis balls. The test method is to place 100 table tennis balls in a specific area and record the number of picking up balls within the specified 10 minutes. The qualification rule is that the average picking speed is less than or equal to 8 seconds per ball, i.e. 75 fetches in 10 minutes. |
| ICACT20210265 Slide.13
[Big Slide]
|
Chrome Click!! |
 |
We move to the results of this study. |
| ICACT20210265 Slide.14
[Big Slide]
|
Chrome Click!! |
 |
It is notable to find a clear presentation in the program in PC for parameter settings in the controller terminal. The user-machine interface is user friendly to design a clear-cut and reasonable model for the robot.
The right side is the robot for fetch-and-carry ball. There are 4 parts, (a) camera; (b) Robot arms for the mechanism of fetching ball; (c) controller board; and (d) wheels.
|
| ICACT20210265 Slide.15
[Big Slide]
|
Chrome Click!! |
 |
The measurement is to count the successful picks of the ball during 10 minutes. The results were listed in Fig. 6. The maximum number of the picks is 88, and the minimum is 69. The qualification rule is that the average picking speed is less than or equal to 8 seconds per ball, i.e. 75 fetches in 10 minutes. Therefore, 18 tests were pass, but 2 test was fail. The average of successful picks of the ball during 10 minutes is 80.5 fetches. The standard deviation is 5.1 fetches. |
| ICACT20210265 Slide.16
[Big Slide]
[YouTube] |
Chrome Click!! |
 |
Now, please enjoy the demonstration and touch the robot.
|
| ICACT20210265 Slide.17
[Big Slide]
|
Chrome Click!! |
 |
The video show presented the surprisingly high performance and accuracy. It’s an amazing technology.
Now, the presentation moves to the section of discussion. |
| ICACT20210265 Slide.18
[Big Slide]
|
Chrome Click!! |
 |
The sports industry will develop soon, because the global population is increasing. Moreover, the population of elders is going to increase soon in many countries. The elders are much richer than ones before. Therefore, some of them will do exercises more. Modern parents wish their children to be wise and healthy. Therefore, the training of doing some exercises is necessary to their children. Consequently, the smart device for sports is a potential issue. |
| ICACT20210265 Slide.19
[Big Slide]
|
Chrome Click!! |
 |
However, this is a preliminary study for the ball fetch-and-carry robot. Some of the hardware and software should be improved,
The moving speed of the robot can be accelerated by employing the higher power motors.
The location of the camera should be adjusted to a higher position for a better view for fetching balls.
The geometric parameters of the robot arms should be newly designed to improve the successful rate of touched balls. In another word, to enlarge the size of the arms may be a better strategy.
|
| ICACT20210265 Slide.20
[Big Slide]
|
Chrome Click!! |
 |
Finally, we have the conclusion. |
| ICACT20210265 Slide.21
[Big Slide]
|
Chrome Click!! |
 |
A robot for ball fetch-and-carry was implemented. It can be improved for a better performance to fetch the balls efficiently. Because the sports market will develop soon, the commercial smart device for sports is a potential issue for the relative researches.
|
| ICACT20210265 Slide.22
[Big Slide]
|
Chrome Click!! |
 |
Thank you very much! |
