Conditional Motion In-betweening
Jihoon Kim1 Taehyun Byun1 Seungyoun Shin2 Jungdam Won3 Sungjoon Choi1
Korea University1Dongguk University2
Meta AI3
Motion in-betweening is a process of generating intermediate poses between the start pose and the end pose, which are given by a user. The task has become especially challenging when the duration between the start and the end poses is long (i.e. sparse) because it is highly unconstrained. Although the state-of-the-art fully automated methods are capable of producing plausible motions given sparse keyframes, they often lack sufficient controllability to generate motions satisfying the contexts required in practical applications. In this paper, we focus on the methodology that can perform controllable motion in-betweening in two perspectives: pose-conditioned in-betweening and semantic-conditioned in-betweening. We also provide motion data augmentation strategy to improve the quality of pose-conditioned generation. The model generated by our framework outperforms the state-of-the-art motion in-betweening model while providing more controllability for users.
@article{
KIM2022108894,
title = {Conditional Motion In-betweening},
journal = {Pattern Recognition},
pages = {108894},
year = {2022},
issn = {0031-3203},
doi = {https://doi.org/10.1016/j.patcog.2022.108894},
url = {https://www.sciencedirect.com/science/article/pii/S0031320322003752},
author = {Jihoon Kim and Taehyun Byun and Seungyoun Shin and Jungdam Won and Sungjoon Choi},
keywords = {motion in-betweening, conditional motion generation, generative model, motion data augmentation},
abstract = {Motion in-betweening (MIB) is a process of generating intermediate skeletal movement between the given start and target poses while preserving the naturalness of the motion, such as periodic footstep motion while walking. Although state-of-the-art MIB methods are capable of producing plausible motions given sparse key-poses, they often lack the controllability to generate motions satisfying the semantic contexts required in practical applications. We focus on the method that can handle pose or semantic conditioned MIB tasks using a unified model. We also present a motion augmentation method to improve the quality of pose-conditioned motion generation via defining a distribution over smooth trajectories. Our proposed method outperforms the existing state-of-the-art MIB method in pose prediction errors while providing additional controllability. Our code and results are available on our project web page: https://jihoonerd.github.io/Conditional-Motion-In-Betweening}
}