Scaling Generative Models from Research to Production

Overview

When I joined Cinamon, motion data was already being collected actively, but there was no AI model-training pipeline to turn those assets into trainable inputs. I began at the foundation: converting heterogeneous 3D motion data into validated tensor representations that models could consume.

From that foundation, I contributed across the full research-to-production lifecycle—defining problems grounded in 3D motion data, designing models, training and evaluating them, and delivering validated models into products. As the team grew, my role expanded to shaping research directions, setting priorities, reviewing technical decisions, and mentoring researchers and engineers.

• Contribution: Built the motion-model training foundation and contributed end to end—from problem definition and model design through training, evaluation, and product delivery
• Model scope: Text-to-motion generation, retrieval, pose-to-pose transformation, refinement, and stylization
• Lifecycle scope: 3D motion-data processing, distributed training and continued fine-tuning, evaluation, model export, and inference serving
• Leadership: Research direction, priority setting, technical review, and mentoring as the team scaled
• Collaboration: AI researchers, engineers, motion-capture operators, animation-domain specialists, and production-service teams
• Period: 2023–2026

Product Problem

The product challenge began with translating each desired capability into a well-defined modeling task. For every feature, we had to determine which motion data could support the target behavior, how that data should be represented for learning, and which model architecture was appropriate for the required level of control and output quality.

These decisions were inseparable from runtime constraints. A feature that required real-time interaction imposed different limits on model complexity, sequence length, and inference latency than a workflow that could run asynchronously. The intended execution model therefore influenced not only the serving architecture, but also the data representation and model design from the beginning.

Evaluation and engine integration were equally important parts of the problem. Research metrics alone could not determine whether generated motion was visually coherent, geometrically valid, or usable in an animation product. Each task required its own combination of quantitative evaluation, rendered inspection, and runtime validation. We also had to choose an export and integration approach that preserved model behavior while satisfying the animation engine’s compatibility and performance requirements.

The core product problem was to connect these decisions—task definition, data selection, representation, model architecture, execution constraints, evaluation, and engine delivery—within a repeatable lifecycle. This allowed new research ideas to be assessed not only by model quality, but also by whether they could become reliable product capabilities.

A Reusable Model Lifecycle

flowchart TD
    A["3D Assets<br/>(FBX, GLB, etc.)"] --> B["Ingestion & Normalization"]
    B --> C["Task-Specific Data Filtering"]
    C --> D{"Data Ready?"}
    D -->|"Issues Found"| E["Operator Feedback & Status Update"]
    E --> B
    D -->|"Ready"| F["Model Training & Fine-tuning"]
    F --> G["Quantitative & Visual Evaluation"]
    G --> H["Model Export"]
    H --> I["Model Serving<br/>(ONNX / TorchServe / Triton)"]
    I --> J["Production Animation Systems"]
    G --> F

The lifecycle began by ingesting and normalizing heterogeneous 3D assets, then filtering them according to the needs of each modeling task. Data readiness was tracked explicitly: when structural or geometric issues were found, the pipeline updated the asset status and returned actionable feedback to motion-capture operators before the affected data could proceed to training.

Once the data was ready, models moved through training and fine-tuning, quantitative and visual evaluation, export, serving, and integration with production animation systems. Evaluation findings fed back into model development, while explicit contracts between stages kept the lifecycle repeatable without tying it to a single implementation technology.

Building the Data Foundation

I initiated and built the first architecture of a motion-capture processing pipeline that converts FBX and GLB assets into training-ready representations.

The pipeline established reusable processing and validation steps for motion assets, including skeleton and coordinate normalization, retargeting, metadata preparation, and visual or geometric inspection. Ray and Blender were incorporated to distribute processing and validate results in the same domain in which downstream animation failures would become visible.

The resulting data foundation was reused across multiple model and product initiatives instead of rebuilding preprocessing logic for each experiment.

Key responsibilities included:

• Designing the asset-processing architecture
• Converting heterogeneous 3D assets into consistent training representations
• Connecting Ray-based parallel processing with Blender-based validation
• Establishing reusable interfaces between processed data, model training, and downstream systems
• Investigating failure cases before they propagated into training or production inference

Text-to-Motion: From Human Annotation to VLM-Assisted Automation

I contributed across the full Text-to-Motion lifecycle, from data collection and quality control to model training, evaluation, serving integration, and engineering handoff. Early datasets were collected through Amazon Mechanical Turk (MTurk), where I operated annotation batches and reviewed worker quality.

As vision-language models advanced, I helped introduce a VLM-assisted annotation pipeline. Motion clips were rendered into sequence and frame-level views, converted into structured text annotations by vision-language models, and routed to targeted human review when directional or interaction details required verification.

• Human annotation: MTurk batch operation and quality control
• Automation: VLM-assisted motion annotation with targeted human review
• Model development: Training, evaluation, and failure analysis
• Delivery: Internal serving, runtime integration, documentation, and engineering handoff

Selected Capabilities

Model Research and Distributed Training

I led and shaped research directions across text-conditioned motion generation, cross-modal retrieval, pose-to-pose modeling, motion refinement, and stylization. Rather than standardizing on a single modeling paradigm, we evaluated and combined generative, retrieval-based, and refinement-based approaches according to each task’s requirements. These included diffusion/score-based models for high-fidelity continuous synthesis and autoregressive Transformers for real-time sequential rollouts, with choices guided by physical fidelity, controllability, sequence stability, and runtime constraints.

For diffusion/score-based research, I pursued both shared and task-specific model strategies. I trained a foundation motion model and researched and applied inference-time guidance and noise scheduling to adapt a single foundation model effectively across tasks. These techniques allowed the generation process to respond dynamically to new spatial and temporal constraints, including obstacle avoidance and motion in-betweening, without modifying the base model weights. Separately, when latency, memory, or runtime constraints were more stringent, I designed and trained lightweight models tailored to the target deployment environment.

To accelerate model iteration and scale training as motion datasets grew, I established a distributed training foundation for multi-GPU and multi-node environments. This infrastructure served two primary purposes: synchronized data parallelism increased the effective batch capacity and improved training stability, while efficient use of compute resources across multiple GPUs and nodes reduced experiment turnaround time. I also introduced and operationalized a framework that allowed long-running training jobs to continue reliably. Structuring the experiment environment made monitoring easier and each run’s configuration readily inspectable, improving reproducibility across architecture exploration and continued fine-tuning. Together, these capabilities enabled rapid exploration of foundation-model adaptations and dedicated lightweight architectures, while supporting consistent, large-scale model evaluation against rigorous product criteria.

The core research and training scope included:

• Generative modeling & conditioning: Diffusion/score-based models and autoregressive Transformers adapted to diverse motion representations, sequence lengths, and multi-modal control constraints
• Sampling-time strategies & control: Inference-time guidance and noise scheduling to adapt a foundation model to dynamic constraints across tasks without weight updates
• Distributed scaling & training efficiency: Synchronized data parallelism and efficient compute utilization across multi-GPU and multi-node environments to expand batch capacity and reduce experiment turnaround time
• Training reliability & experiment reproducibility: Reliable execution of long-running training jobs and structured experiment environments for easier monitoring, inspectable run settings, and reproducible model exploration
• Foundation and lightweight model strategy: Principled trade-offs and continued adaptation between expressive foundation motion models and purpose-built lightweight architectures for real-time deployment

Evaluation as a Product Contract

Quantitative research metrics supported systematic comparisons across model variants, loss objectives, and checkpoint progressions. To complement these metrics with domain judgment, I established a user-study environment where in-house motion experts could visually evaluate rendered 3D motion and record their quality assessments. Integrating this expert feedback into the evaluation cycle created a continuous model-quality improvement loop grounded in the standards of practitioners who worked directly with motion.

The core evaluation framework included:

• Quantitative model comparison: Using benchmark metrics to compare model variants, loss objectives, and checkpoint progressions under consistent criteria
• Domain-expert user studies: A dedicated environment for in-house motion experts to visually review rendered 3D motion and record quality judgments
• Product quality assessment: Combining quantitative comparisons with domain-expert evaluations to assess motion quality for downstream animation
• Continuous quality improvement: Incorporating expert feedback into ongoing model evaluation and iteration

From Checkpoint to Production Inference

I delivered models through two production paths: direct integration into animation engines and API-based inference services. For engine integration, I exported models to ONNX, addressed Opset and runtime compatibility, and validated the exported artifacts against target-engine constraints. For service-based integration, I provided TorchServe and Triton serving options so downstream teams could access model inference through APIs.

Production readiness also required teams to evaluate models before adopting them. I built an internal test environment where users could exercise inference workflows and run performance tests using deployable model artifacts. To support integration and handoff, I provided example code, model and API specifications, implementation documentation, and direct technical support for the teams responsible for using the models.

The production-delivery work included:

• Engine-ready model export: ONNX artifacts with Opset and runtime compatibility for direct engine integration
• API-based model serving: TorchServe and Triton serving options for service-based inference
• Internal test environment: Functional and performance testing of deployable model artifacts before integration
• Engineering handoff and support: Example code, model and API specifications, implementation documentation, and technical support for consuming teams

Leadership and Impact

As the AI team expanded, my role as AI Lead centered on helping researchers, engineers, motion specialists, and product teams make aligned decisions. I worked with the team to clarify research priorities, review model and system designs, and coordinate research programs by aligning responsibilities with each team member’s strengths and expertise. I also supported researchers and engineers through experimentation and productionization.

Together, we established a shared foundation spanning data preparation, model development, evaluation, and inference delivery. Reusing this lifecycle allowed teams to build on prior work across model and product initiatives, reduced repeated setup, and kept research iteration grounded in production constraints. The resulting processes, documentation, and technical support also gave new team members and partner teams a clearer path from experimentation to delivery.

The impact came from a more connected way of working: research decisions incorporated product and domain context earlier, while production teams received models through more consistent evaluation and delivery practices.

Technical Summary

• Model research: Generative models, diffusion/score-based models, autoregressive Transformers, sampling-time guidance, foundation-model adaptation, and task-specific lightweight models
• Distributed training: Multi-GPU and multi-node training
• Experiment management: Reproducible configurations, experiment monitoring, ablation studies, and baseline comparisons
• Data and evaluation: 3D assets (FBX, GLB) processing and normalization, retargeting, VLM-assisted annotation, and Blender (bpy)
• Production delivery: ONNX export and Opset compatibility for direct engine integration, TorchServe and Triton API serving, internal performance testing, and engineering handoff through example code, specifications, documentation, and technical support