Scaling Generative Models from Research to Production

Overview

At Cinamon, I led key parts of the lifecycle that turns generative-model research into production capabilities. Generative character motion was the primary application domain, but the engineering problem was broader: building a repeatable path from heterogeneous source data and evolving research models to validated, versioned, and deployable inference artifacts.

My work connected 3D data processing, model research, distributed training, evaluation, export, and serving. I also set research directions across text-to-motion generation, motion retrieval, pose-to-pose modeling, refinement, and stylization while reviewing technical decisions and mentoring researchers and engineers.

• Contribution: Initiated the shared motion-data foundation and led key parts of model research, training, evaluation, and production delivery
• Model scope: Text-to-motion generation, retrieval, pose-to-pose transformation, refinement, and stylization
• Lifecycle scope: Data ingestion, distributed validation, training and continued fine-tuning, evaluation, model export, and inference serving
• Collaboration: AI researchers, engineers, animation-domain specialists, and production-service teams
• Period: 2023–2026

Product Problem

A production generative model is more than a model architecture or a successful research checkpoint.

Training data arrived through heterogeneous FBX and GLB assets with different skeletons, coordinate systems, motion conventions, and quality levels. A model trained on inconsistent data could produce plausible benchmark results while failing on the geometric and runtime constraints of an animation product.

Research models also changed quickly. New representations, objectives, datasets, and model variants had to be evaluated without breaking reproducibility. Once a model was selected, its behavior still had to survive export, runtime compatibility constraints, inference optimization, and integration with downstream systems.

The core problem was therefore to establish a shared model lifecycle that could repeatedly answer four questions:

1. Is the source data structurally and geometrically valid?
2. Can the model be trained and resumed reproducibly?
3. Does it improve the target behavior under measurable evaluation?
4. Can the resulting model be exported and consumed reliably in production?

A Reusable Model Lifecycle

flowchart LR
    A["FBX / GLB Assets"] --> B["Ingestion & Normalization"]
    B --> C["Ray / Blender Validation"]
    C --> D["Model Training & Fine-tuning"]
    D --> E["Quantitative & Visual Evaluation"]
    E --> F["ONNX Export & Runtime Validation"]
    F --> G["Triton Serving"]
    G --> H["Production Animation Systems"]
    E --> D

Rather than treating these stages as separate research and engineering tasks, I worked to connect them through explicit data, model, evaluation, and serving contracts. This made it possible to improve individual models while preserving a repeatable path to production.

Building the Data Foundation

I initiated and built the first architecture of a motion-capture processing pipeline that converts FBX and GLB assets into training-ready representations.

The pipeline established reusable processing and validation steps for motion assets, including skeleton and coordinate normalization, retargeting, metadata preparation, and visual or geometric inspection. Ray and Blender were incorporated to distribute processing and validate results in the same domain in which downstream animation failures would become visible.

The resulting data foundation was reused across multiple model and product initiatives instead of rebuilding preprocessing logic for each experiment.

Key responsibilities included:

• Designing the initial asset-processing architecture
• Converting heterogeneous 3D assets into consistent training representations
• Connecting Ray-based parallel processing with Blender-based validation
• Establishing reusable interfaces between processed data, model training, and downstream systems
• Investigating failure cases before they propagated into training or production inference

Text-to-Motion: From Human Annotation to VLM-Assisted Automation

I contributed across the full Text-to-Motion lifecycle, from data collection and quality control to model training, evaluation, serving integration, and engineering handoff. Early datasets were collected through Amazon Mechanical Turk (MTurk), where I operated annotation batches and reviewed worker quality.

As the data program evolved, I helped introduce a VLM-assisted annotation pipeline. Motion clips were rendered into sequence and frame-level views, converted into structured text annotations by vision-language models, and routed to targeted human review when directional or interaction details required verification.

The resulting data supported model retraining, evaluation, TorchServe-based internal serving, runtime integration, documentation, and handoff to the engineering team.

• Human annotation: MTurk batch operation and quality control
• Automation: VLM-assisted motion annotation with targeted human review
• Model development: Training, evaluation, and failure analysis
• Delivery: Internal serving, runtime integration, documentation, and engineering handoff

Selected Capabilities

Model Research and Distributed Training

I set or reviewed research directions across text-conditioned motion generation, retrieval, pose-to-pose modeling, motion refinement, and stylization.

The model stack used Python, PyTorch, Lightning, Hydra, OmegaConf, and CUDA. Training and continued fine-tuning included diffusion-based models, variable-length motion representations, geometric objectives, and conditioning strategies for controllable generation.

I operated distributed training with Lightning DDP and NCCL, including checkpoint-based restart and recovery. The goal was not only to complete large experiments, but to make experiments reproducible enough that model changes could be evaluated, compared, and eventually delivered.

The work included:

• Distributed multi-GPU and multi-node training
• Checkpoint restart and recovery for long-running experiments
• Continued fine-tuning of pretrained diffusion models for new data and objectives
• Reproducible configuration and experiment management
• Comparison of generative, retrieval-based, and refinement-based approaches
• Technical review and mentoring across model programs

Evaluation as a Product Contract

Model evaluation combined research metrics with visual and geometric validation.

Quantitative results were used to compare models, objectives, and checkpoints, while rendered motion made structural failures visible: foot sliding, discontinuities, unstable transitions, incorrect contact, or behavior that violated downstream animation constraints.

Evaluation was treated as a contract between research and production rather than a final reporting step. A model had to demonstrate improvement under measurable criteria and remain valid after export and runtime integration.

This included:

• Defining task-specific success criteria and baselines
• Comparing model variants and failure modes
• Combining quantitative evaluation with rendered inspection
• Validating exported behavior against research checkpoints
• Feeding evaluation findings back into data, objectives, and model design

From Checkpoint to Production Inference

I connected research checkpoints to production delivery through ONNX export and Triton serving.

Export required more than serializing model weights. Variable-length inputs, runtime and opset compatibility, unsupported operations, and downstream engine constraints had to be handled explicitly. Exported models were tested as deployable artifacts rather than assumed to behave identically to their research implementations.

The production-delivery work included:

• ONNX export for model variants and variable-length inputs
• Runtime and compatibility validation
• Tests for exported-model behavior
• Triton-based model serving
• Inference optimization
• Clear contracts for downstream consumers

This closed the gap between a promising research prototype and a model that production systems could invoke repeatedly and reliably.

Leadership and Impact

As AI Lead, I connected research priorities with product and infrastructure requirements. I reviewed model and system designs, allocated work across research programs, and mentored researchers and engineers through experimentation and productionization.

The main outcome was a shared foundation spanning data preparation, model development, evaluation, and inference delivery. It allowed multiple model and product initiatives to reuse the same lifecycle while keeping research iteration connected to production constraints.

This work demonstrates a broader capability beyond character motion: turning evolving generative-model research into measurable, deployable, and maintainable production systems.

Technical Summary

• Model research: Python, PyTorch, Lightning, diffusion models, text conditioning, variable-length modeling, geometric objectives, continued fine-tuning
• Distributed training: DDP, NCCL, CUDA, torch.distributed, checkpoint restart and recovery
• Experiment management: Hydra, OmegaConf, reproducible configuration, baseline comparison, failure analysis
• Data and validation: FBX, GLB, retargeting, Ray, Blender, visual and geometric validation
• Production delivery: ONNX, Triton, runtime compatibility, inference optimization, downstream model contracts