Meet Fujitsu’s AI Gymnastics Judges

The International Gymnastics Federation (FIG) recently approved the use of a “Judging Support System” developed by Fujitsu for a series of FIG gymnastics events in 2019. The system will be tested at the 2019 FIG World Cup Series, then officially launched for the 49th Artistic Gymnastics World Championships in Stuttgart, Germany this coming October. The goal is to get the system ready to score half the artistic gymnastics events at the 2020 Tokyo Olympics, and all such events at the Paris Olympics in 2024.

Unlike events such as running and swimming that rely on precise timekeeping technology, gymnastics events require judges to evaluate and score gymnasts’ performances on an artistic level. Gymnasts present their skills in routines which include a series of elements meeting the technical and composition requirements of the FIG Code of Points.

Leveraging AI and 3D sensing technologies, Fujitsu’s Judging Support System conducts real-time movement capture, analyses and scoring. Human judges sometimes struggle to score complicated and quick movements using the naked eye, and it’s hoped the system can overcome these and other human limitations so that evaluation will become more fair and accurate for all athletes.

Synced has prepared a brief Q&A for readers who would like to better understand the technology.

Q1. What type of sensors are used for data capture?

A1. Lidar sensors. Fujitsu’s MEMS solid-state Lidar has over 2 million pulses per second emission and a 15-meter detection range. Although such sensors are quite costly, Fujitsu predicts the system will bring in more than JP¥100 billion (~US$1 billion) profit over the next 10 years.

Q2. What are the core components in the hardware system?

A2. In addition to the 3D Lidar sensor, there is also a computer for sensor controlling, the cloud, digital front end (DFE) server, storage, load balancer, scoring application server and another computer for the judging system. The main computations are performed at the DFE server for joint recognition tasks, and at the scoring application server for scoring tasks based on joint positions.

Q3. What are the main modules in the judging support system?

A3. The key modules are movement sensing, joint position recognition and a database of gymnastics skill elements. A high-speed matching of captured data with previously stored data is the key to the system.

Q4. How does the judging system work? Any use of deep learning?

A4. The joint recognition module uses deep learning technology. The neural network model receives several multi-viewpoint depth images as input and outputs corresponding 3D joint position results. Joint recognition also requires certain adjustments and calibrations according to the human joint model, however the method behind this has not been published.

Fujitsu worked closely with the Japanese Gymnastics Association to build up the database with a set of elements for each skill difficulty. The project team collected over 800 elements for male gymnasts and more than 500 elements for female gymnasts — where the elements comprise a series of basic skills from top gymnasts. Again, details on the procedure and the possible use of deep learning remain cloaked in mystery.

For judging and scoring, the system does not apply an end-to-end model.

 

Q5. Does the system support Difficulty (D) or Execution (E) scoring?

A5. Both. The final score of each exercise is established by the summation of the D-Score and E-Score. Each time a gymnast completes an element with a certain skill level, he or she will be rewarded using a corresponding difficulty value (DV). Starting with a perfect 10.00 points, the E-score is lowered by 0.10 points for small faults, 0.30 points for medium faults, 0.50 points for large faults, and 1.00 point for very large faults.

Q6. In what events will the Judging Support System be implemented?

A6. The system will first used in the Men’s Pommel Horse and Vault events at the 2019 FIG World Cup Series. For the 2020 Tokyo Olympics, the system is scheduled to support Men’s Vault, Rings and Pommel Horse; as well as Women’s Vault and Balance Beam.

Q7. Who will benefit from the system?

A7. Fujitsu says that in addition to supporting judges in making decisions, the system can also assist athletes with training, and can help the audience better understand and enjoy gymnastics.

For example, the system can accurately inform athletes regarding their stability and the exact angle between their joints, so they can make appropriate adjustments and improvements. The system can also append quantitative indicators such as height and stability to competition streaming services, to guide the audience to a better appreciation of the sport.

Much about the Judging Support System’s tech remains unknown as FIG and Fujitsu have thus far not published any reports regarding system performance in joint recognition accuracy, scoring response time, real-time data capture or presentation feasibility, etc.

Can AI effectively assist or even replace human judges in gymnastics competitions? Fujitsu will test that question at the FIG World Cup, hoping to answer it positively at the international spotlight that is the Olympics.

 

Source: Synced China

---

Localization: Tingting Cao | Editor: Michael Sarazen