DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimization.
Saved in:
| Title: | DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimization. |
|---|---|
| Authors: | Ponton, J. L.1 (AUTHOR), Pujol, E.1 (AUTHOR), Aristidou, A.2,3 (AUTHOR), Andujar, C.1 (AUTHOR), Pelechano, N.1 (AUTHOR) |
| Source: | Computer Graphics Forum. May2025, Vol. 44 Issue 2, p1-14. 14p. |
| Subjects: | Motion capture (Cinematography), Motion capture (Human mechanics), Time series analysis, Robot motion, Deep learning |
| Abstract: | High‐quality motion reconstruction that follows the user's movements can be achieved by high‐end mocap systems with many sensors. However, obtaining such animation quality with fewer input devices is gaining popularity as it brings mocap closer to the general public. The main challenges include the loss of end‐effector accuracy in learning‐based approaches, or the lack of naturalness and smoothness in IK‐based solutions. In addition, such systems are often finely tuned to a specific number of trackers and are highly sensitive to missing data, e.g., in scenarios where a sensor is occluded or malfunctions. In response to these challenges, we introduce DragPoser, a novel deep‐learning‐based motion reconstruction system that accurately represents hard and dynamic constraints, attaining real‐time high end‐effectors position accuracy. This is achieved through a pose optimization process within a structured latent space. Our system requires only one‐time training on a large human motion dataset, and then constraints can be dynamically defined as losses, while the pose is iteratively refined by computing the gradients of these losses within the latent space. To further enhance our approach, we incorporate a Temporal Predictor network, which employs a Transformer architecture to directly encode temporality within the latent space. This network ensures the pose optimization is confined to the manifold of valid poses and also leverages past pose data to predict temporally coherent poses. Results demonstrate that DragPoser surpasses both IK‐based and the latest data‐driven methods in achieving precise end‐effector positioning, while it produces natural poses and temporally coherent motion. In addition, our system showcases robustness against on‐the‐fly constraint modifications, and exhibits adaptability to various input configurations and changes. The complete source code, trained model, animation databases, and supplementary material used in this paper can be found at https://upc-virvig.github.io/DragPoser [ABSTRACT FROM AUTHOR] |
| Copyright of Computer Graphics Forum is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 186773088 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimization. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Ponton%2C+J%2E+L%2E%22">Ponton, J. L.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Pujol%2C+E%2E%22">Pujol, E.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Aristidou%2C+A%2E%22">Aristidou, A.</searchLink><relatesTo>2,3</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Andujar%2C+C%2E%22">Andujar, C.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Pelechano%2C+N%2E%22">Pelechano, N.</searchLink><relatesTo>1</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Computer+Graphics+Forum%22">Computer Graphics Forum</searchLink>. May2025, Vol. 44 Issue 2, p1-14. 14p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Motion+capture+%28Cinematography%29%22">Motion capture (Cinematography)</searchLink><br /><searchLink fieldCode="DE" term="%22Motion+capture+%28Human+mechanics%29%22">Motion capture (Human mechanics)</searchLink><br /><searchLink fieldCode="DE" term="%22Time+series+analysis%22">Time series analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Robot+motion%22">Robot motion</searchLink><br /><searchLink fieldCode="DE" term="%22Deep+learning%22">Deep learning</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: High‐quality motion reconstruction that follows the user's movements can be achieved by high‐end mocap systems with many sensors. However, obtaining such animation quality with fewer input devices is gaining popularity as it brings mocap closer to the general public. The main challenges include the loss of end‐effector accuracy in learning‐based approaches, or the lack of naturalness and smoothness in IK‐based solutions. In addition, such systems are often finely tuned to a specific number of trackers and are highly sensitive to missing data, e.g., in scenarios where a sensor is occluded or malfunctions. In response to these challenges, we introduce DragPoser, a novel deep‐learning‐based motion reconstruction system that accurately represents hard and dynamic constraints, attaining real‐time high end‐effectors position accuracy. This is achieved through a pose optimization process within a structured latent space. Our system requires only one‐time training on a large human motion dataset, and then constraints can be dynamically defined as losses, while the pose is iteratively refined by computing the gradients of these losses within the latent space. To further enhance our approach, we incorporate a Temporal Predictor network, which employs a Transformer architecture to directly encode temporality within the latent space. This network ensures the pose optimization is confined to the manifold of valid poses and also leverages past pose data to predict temporally coherent poses. Results demonstrate that DragPoser surpasses both IK‐based and the latest data‐driven methods in achieving precise end‐effector positioning, while it produces natural poses and temporally coherent motion. In addition, our system showcases robustness against on‐the‐fly constraint modifications, and exhibits adaptability to various input configurations and changes. The complete source code, trained model, animation databases, and supplementary material used in this paper can be found at https://upc-virvig.github.io/DragPoser [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Computer Graphics Forum is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=186773088 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1111/cgf.70026 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 14 StartPage: 1 Subjects: – SubjectFull: Motion capture (Cinematography) Type: general – SubjectFull: Motion capture (Human mechanics) Type: general – SubjectFull: Time series analysis Type: general – SubjectFull: Robot motion Type: general – SubjectFull: Deep learning Type: general Titles: – TitleFull: DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimization. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Ponton, J. L. – PersonEntity: Name: NameFull: Pujol, E. – PersonEntity: Name: NameFull: Aristidou, A. – PersonEntity: Name: NameFull: Andujar, C. – PersonEntity: Name: NameFull: Pelechano, N. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Text: May2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 01677055 Numbering: – Type: volume Value: 44 – Type: issue Value: 2 Titles: – TitleFull: Computer Graphics Forum Type: main |
| ResultId | 1 |
