Introduction
This is based on the research paper titled “Learning Soccer Juggling Skills with Layer-wise Mixture-of-Experts,” which was presented at the SIGGRAPH ’22 Conference. The paper was authored by a group of researchers dedicated to the advancement of physics-based simulation and control for generating soccer juggling animations. The full paper can be accessed and read here. The credit for this insightful and innovative research goes to the authors of the paper. Their work contributes significantly to the field of reinforcement learning and its application in sports simulation.
In the world of sports, soccer juggling is a skill that requires a high level of control and precision. This skill involves keeping a soccer ball in the air by bouncing it off various parts of the body without letting it touch the ground. Recently, a group of researchers developed a system that uses physics-based simulation and control to generate soccer juggling animations. Their work, titled “Learning Soccer Juggling Skills with Layer-wise Mixture-of-Experts,” was presented at the SIGGRAPH ’22 Conference.
The researchers’ system is designed to easily specify different soccer juggling skills using either crude hand-designed pose sequences or motion capture data. Transitions between skills are introduced as directed edges in a control graph, and reinforcement learning (RL) is used to train control policies based on this graph. To support efficient and effective learning, the system employs a layer-wise mixture-of-experts architecture.
Methodology
The researchers designed a control graph that specifies various juggling skills and their transitions. These skills are learned via a random walk on the graph. The policy generates the action based on the upcoming control nodes and the simulation state. The policy is trained based on the reward feedback via RL. A simulation episode terminates if the constraints in the control node are violated, and the edge weight of the specific node will be updated to adjust the probability of traversing an edge during the random walk.
The researchers also introduced a layer-wise mixture-of-experts (MOE) architecture. A linear MOE layer consists of multiple linear layers (experts) that are used independently to construct different outputs. These outputs are blended together via the expert weights. A layer-wise MOE consists of multiple layers of linear MOE, and a common gating network is used to generate the expert weights for all linear MOE layers.
Results
The researchers found that different skills exhibit different gating patterns. They also observed that layer-wise MOE induces better specialization, which can reduce the interference effect between tasks. The slightly worse utilization is expected since the control graph is unbalanced.
The adaptive random walk supports the learning of challenging transitions. In all cases, training with adaptive random walk converges faster. Even without the adaptive random walk, the layer-wise MOE is better than alternatives with the adaptive random walk, further demonstrating the benefit of using the layer-wise MOE.
Conclusion
The researchers concluded that their system can perform a variety of full-body soccer juggling skills and the related transitions, including foot, knee, head, and chest juggling, as well as the around the world foot juggle. They also found that their learned policy can withstand perturbations equivalent to a moderate breeze. Surprisingly, they discovered that their policy is able to juggle novel shapes such as box, cylinder, and ellipsoid, with sizes similar to that of the soccer ball.
This research contributes to the field by proposing an overall method for learning difficult soccer juggling skills. It shows that a layer-wise mixture-of-experts architecture provides significant benefits for this multi-skill RL problem. The researchers also introduced an adaptive random walk training strategy in support of efficient learning.