Building a Training-Ready 3D Motion Data Platform

Overview

At Cinamon, I designed and introduced the initial version and architecture of the Motion Capture Processor (MCP), a shared platform for transforming production 3D motion assets into training-ready datasets.

Raw FBX and GLB assets were not directly usable by learning-based models. They arrived with different rig structures, coordinate systems, body configurations, motion conventions, and metadata quality. MCP established a consistent boundary between those heterogeneous assets and downstream model development.

The platform parses and filters source assets, normalizes or retargets their motion, canonicalizes spatial representations, validates structural and geometric integrity, and produces reproducible motion data for model training and evaluation. I also helped move this workflow from manually invoked processing to automated pipelines orchestrated with Dagster and Airflow across successive versions of the platform.

Ownership: Designed and introduced MCP’s initial version and architecture
Inputs: Heterogeneous FBX and GLB motion assets
Outputs: Validated motion representations, metadata, and reproducible dataset splits
Processing: Filtering, normalization, retargeting, canonicalization, validation, and conversion
Automation: Dagster- and Airflow-orchestrated pipelines for dependency management, scheduling, and operational visibility
Execution: Ray-based parallel processing with resumable workflows and Blender-based 3D operations
Consumers: Generative, retrieval, refinement, and animation models
Period: 2023–2026

Product Problem

A 3D motion file is a production asset, not a machine-learning sample.

Two files that appear to contain the same action can encode different skeleton hierarchies, joint orientations, coordinate conventions, frame rates, body proportions, or incomplete metadata. Feeding those differences directly into training silently changes the meaning of the data and makes failures difficult to trace.

Model teams therefore needed more than one-off conversion scripts. They needed a shared data contract that could repeatedly answer four questions:

Can this asset be parsed and interpreted consistently?
Is its skeleton, geometry, and motion structurally valid?
Can it be transformed into a representation compatible with model training?
Can the same processing result be reproduced, inspected, and reused by other model programs?

From 3D Assets to Model-Ready Data

flowchart TB
    A["FBX / GLB"] --> O["Dagster / Airflow"]
    O --> B["Parse & Filter"]
    B --> C["Normalize & Retarget"]
    C --> D["Canonicalize & Validate"]
    D --> E["Motion Dataset"]
    E --> F["Tensor (B, T, D)"]
    F --> G["Training & Evaluation"]
    H["Ray + Blender"] --> B
    H --> C
    H --> D

I designed MCP as a staged pipeline with explicit boundaries between source assets, 3D-domain processing, validation, and model-facing representations. Each stage produces inspectable artifacts and metadata rather than hiding all transformations inside a single opaque export step.

This separation made it possible to change a retargeting rule, validation criterion, or model representation without rebuilding the entire workflow. It also allowed data failures to be identified before they propagated into expensive model experiments.

Designing MCP from the Ground Up

The initial MCP architecture established the path from raw motion files to processed datasets that researchers could use consistently. I designed the core processing flow, introduced the first implementation, and connected its outputs to the training workflows used by the AI team.

The foundation included:

Parsing source assets and filtering them through explicit metadata criteria
Normalizing coordinate, skeleton, and motion conventions
Retargeting motion when the source and target body structures differed
Splitting long sequences into model-appropriate temporal windows
Producing deterministic training, validation, and test partitions
Recording processing results and failure reasons for inspection and recovery

The objective was not to support one model once. It was to make 3D motion data a reusable platform capability that different model programs could depend on.

Automating the Data Pipeline

Once the processing stages were established, I helped turn them from developer-operated commands into repeatable, observable workflows. The orchestration layer made dependencies explicit, scheduled data refreshes, exposed stage-level failures, and allowed completed outputs to be reused instead of restarting an entire dataset job.

The platform used Dagster and Airflow at different points in its evolution rather than treating them as one permanent stack. After comparing Airflow, Prefect, and Dagster, we selected Dagster for an asset-oriented pipeline with lower operational overhead for the initial use case. Dagster assets connected source-repository updates, metadata filtering, motion preprocessing, dataset splitting, and motion-text preparation as one dependency-aware workflow.

Airflow DAGs were also introduced and later refactored for scheduled operational preprocessing. These workflows incorporated human and VLM-generated annotations, applied error and warning gates before data entered training, refreshed processed datasets, and recorded pipeline statistics for review.

The orchestration and processing layers remained deliberately separate:

Dagster and Airflow: Workflow dependencies, scheduling, run status, and failure visibility
MCP: 3D-domain parsing, conversion, validation, and model-facing data contracts
Ray: Parallel execution across independent motion assets
Blender: Rig-aware transformations and visual or geometric validation

This separation allowed the team to change orchestration choices as operating requirements evolved without rewriting the core 3D processing logic.

A Tensor Contract for Motion Models

MCP bridges two representations that serve different purposes: articulated 3D assets for animation systems and dense tensors for learning systems.

As the platform evolved, model-facing motion could be encoded as a batch-first tensor with shape (B, T, D):

B — batch: multiple motion sequences processed together
T — time: the ordered frames of each sequence
D — features: flattened joint positions, rotations, and optional motion signals

The feature contract can include root and joint transforms, velocity, contact, or temporal deltas, while sequence lengths and masks preserve variable-duration motion. Rotation representations can be adapted for model needs and decoded back into animation-domain forms for geometric or visual validation.

This contract allows training code to consume normalized tensors without reimplementing FBX/GLB parsing, rig interpretation, padding, or coordinate conversion for every experiment. Just as importantly, decoded motion can be checked again in 3D space rather than evaluated only as arrays of numbers.

Data Quality as an Engineering Contract

Validation was treated as part of dataset construction, not as cleanup after training.

MCP checks whether a processed sample satisfies structural expectations and rejects or records invalid results before they enter a dataset. Geometric inspection complements numerical checks because issues such as incorrect facing direction, implausible root placement, broken retargeting, or discontinuous motion may only become obvious when reconstructed in 3D.

The validation path includes:

Required skeleton and joint-structure checks
NaN and malformed-value detection
Coordinate and initial-pose canonicalization
Geometric outlier and motion-integrity checks
Blender-based rendering or inspection for domain-visible failures
Processing reports that keep rejected assets and failure reasons traceable

By making these rules explicit, MCP turns data quality into a contract shared by data processing, model research, and downstream animation systems.

Scalable and Reproducible Execution

3D asset processing combines file I/O, CPU work, and Blender-dependent operations. MCP uses Ray-based parallel execution to distribute independent assets while bounding work in flight, recording progress, and resuming interrupted runs without discarding completed results.

Reproducible configuration and deterministic dataset splits keep model comparisons grounded in the same processed data. Researchers can trace a training sample back to its source and processing metadata, while pipeline changes can be evaluated without relying on undocumented local scripts.

The result is a workflow designed for operational reliability as well as throughput:

Automated and scheduled preprocessing workflows
Explicit stage dependencies and failure visibility
Parallel asset processing with controlled concurrency
Resume and recovery for interrupted dataset jobs
Deterministic train, validation, and test splits
Inspectable metadata and processing reports
Reusable outputs shared across model and product initiatives

Adoption and Evolution

MCP became a shared motion-data foundation rather than remaining tied to its first training workflow. It supported generative motion, retrieval, pose-to-pose transformation, refinement, and animation-related initiatives that otherwise would have needed separate preprocessing implementations.

The platform later evolved to support broader body representations, Torch-based motion operations, additional retargeting paths, and closer integration with model training and production systems. That evolution preserved the architectural principle established at the beginning: 3D-domain complexity should be handled behind a validated, reusable interface instead of leaking into every model codebase.

This project demonstrates my ability to identify a cross-team infrastructure gap, design the initial system from first principles, and turn domain-specific 3D assets into reliable inputs for learning-based products.

Technical Summary

3D data: FBX, GLB, articulated skeletons, coordinate systems, retargeting, and canonicalization
Model representation: PyTorch tensors, variable-length sequences, masks, joint positions and rotations, normalization, and padding
Orchestration: Dagster, Airflow, asset- and DAG-based dependencies, scheduling, run visibility, and automated dataset refreshes
Processing: Python, Ray, Blender, structured metadata, windowing, and deterministic dataset splits
Validation: Structural checks, geometric validation, visual inspection, failure reporting, and reproducible processing
Platform impact: Shared data contracts for generative, retrieval, refinement, and animation models