Building a Production Multimodal Agent Platform

Overview

CineV Studio is an AI-powered animation studio that enables creators to produce animated content with image and video generation models.

At Cinamon, I helped validate and introduce an LLM-agent-based system that later became a core part of CineV’s production workflow. My contributions began with a LangGraph-based proof of concept that treated the LLM as a workflow agent: it inspects structured state, selects tools, and decides how the pipeline should advance instead of following a fixed sequence. I also contributed across CineV’s core AI capabilities, including productizing the multimodal generation and orchestration paths that connect editable 3D previz to image and video generation.

• Contribution: Agentic architecture proposal and implementation, contributions across CineV’s core AI capabilities, and productization of multimodal generation and orchestration
• Module scope: Story and shot planning, agent tooling, editable 3D previz, multimodal generation, evaluation, and durable execution
• Collaboration: AI team, Web Service team, and DevOps team
• Period: February 2026–present

Product Problem

Turning a story into a producible visual sequence is not a collection of independent model calls. Scripts, characters, locations, props, shot intent, and visual references must retain the same meaning and identity as work moves across language, vision, image, and video models.

Prompt-only generation also gives creators limited control over pose, spatial relationships, and camera composition. Even when an individual image looks plausible, character, location, or prop identity may drift across shots, while the intended staging may be lost when each model call reinterprets the scene independently.

The product therefore had two tightly connected core problems to solve. First, it needed to preserve narrative meaning and asset identity through structured narrative state and identity-bound visual references. Second, it needed to make creative intent spatially controllable. Editable 3D previz makes that intent explicit, allowing creators to adjust character poses, proxies, motion, and cameras before those decisions guide downstream image and video models.

Making this workflow reliable as a production service introduced an additional engineering challenge: long-running execution, model variability, provider failures, retries, and user revisions all had to be handled without destabilizing the product. I contributed to resilient execution paths, monitoring, and distributed tracing that capture actionable errors across workflow and model boundaries. The resulting operational records made recurring failure patterns visible and supported continuous improvements to product stability.

Core Module Contributions & Collaboration

I proposed Cinamon’s initial LangGraph-based proof of concept for agentic orchestration and worked with the team to develop it. The key shift was to let the LLM act as an agent that evaluates the current state, calls the tools it needs, and determines the next pipeline transition rather than serving as one fixed step in a predetermined chain.

Building on that proof of concept, I contributed to the initial end-to-end implementation of the story-planning agent system. My work included graph-based planning flows, typed state transitions, tool-enabled access to project and spatial context, and the generation of grounded scene and shot specifications. The system was subsequently expanded and adopted as a core production capability.

I also contributed to designing the core multimodal generation and durable-orchestration paths that connect those decisions to reusable visual references, model providers, persistent assets, and long-running execution. The reference-binding layer keeps characters, locations, and props associated with their intended visual evidence across shots.

My contributions also extended beyond the LLM Agent. I worked on the 3D asset and motion foundation used by the previz layer: rigged assets, pose and motion libraries, camera state, spatial relationships, and reusable references that translate narrative and visual state into controllable spatial guidance.

I worked across functions on these core implementation paths and helped define the semantic, API, and event contracts needed for product integration. The surrounding product experience and service interfaces were developed with the Web Service team, while deployment and production operations were coordinated with the DevOps team.

Agent and Previz System Design

flowchart TB
    A["Story Intent + Reusable Assets"] --> B["Agentic Planning<br/>LangGraph · Tool Use"]
    B --> C["Narrative & Visual State<br/>Identity · Shot Intent"]
    C --> D["Editable 3D Previz<br/>Pose · Motion · Camera"]
    U["Creator Edits"] --> D
    D --> E["Image & Video Generation<br/>in CineV Studio"]

The core workflow carries story meaning and reusable asset identity into an editable spatial representation before image and video generation. The production details behind each stage are described below:

• Agentic story planning: LangChain and LangGraph compose typed state graphs for story, scene, action, and shot planning. At each bounded transition, the LLM evaluates current state, selects tools, and determines how the workflow should proceed.
• Agent tooling: Explicit tools query project assets, available actions, map state, and spatial relationships instead of asking the model to reconstruct grounded context from prompts.
• Structured narrative state: Story entities, relationships, shot intent, visual references, and model decisions are maintained in explicit schemas, preserving their meaning across planning, 3D previz, and media generation.
• Identity-bound visual references: Characters, locations, and props remain associated with the intended visual evidence in a stable order, preserving identity as shots move across model and provider boundaries.
• Editable 3D control layer: Structured output becomes poses, proxies, motion, and camera state that creators can inspect and revise before generation.
• Multimodal generation boundary: Visual references and shot intent are translated into provider-independent image and video requests, then executed as durable workflows with persistent results.

Editable 3D Previz for Controllability

The 3D previz is not only a visualization of agent output. It is an editable intermediate representation between narrative planning and generative media. Creators can select or infer a pose, place character and environment proxies, adjust motion and camera composition, and then use the resulting shot as a visual reference for image and video generation.

The narrative and visual state defines what must remain consistent: story meaning, character identity, reusable assets, and shot intent. The 3D previz defines how that intent is staged and filmed, giving creators a direct way to inspect and revise spatial decisions before generation.

This approach extends controllability beyond prompt wording. Spatial intent remains explicit, creator revisions are preserved as structured state, and the same 3D and motion assets can be reused across shots instead of being reinterpreted independently by each model call.

Evaluation & Operations

I helped establish and maintain an internal development deployment separate from production so the AI team could build and test agent, previz, and generation ideas without waiting for full product integration. Promising capabilities could be validated early and moved into the product through a repeatable adoption path, shortening the feedback loop and accelerating feature development.

I also contributed to building a shared monitoring and tracing environment that made failures and quality regressions visible across the product workflow. Centralizing those records reduced the time spent reproducing problems, made recurring patterns easier to identify, and created a continuous loop for validating fixes and improving service reliability.

AI tools were used throughout implementation, debugging, evaluation, and iteration. I helped organize the project and its development workflows to be LLM/VLM-friendly, making relevant context and feedback easy for both engineers and AI tools to inspect and reuse. This reduced repeated context reconstruction and enabled a much faster development cadence from idea to tested capability.

Productization & Impact

• Connected story input, reusable character, location, prop, and motion assets, and image/video storyboards as one coherent production workflow.
• Preserved narrative and visual continuity across shots through structured narrative state and identity-bound visual references.
• Turned editable 3D poses, proxies, motion, and cameras into a control surface that makes narrative intent spatially explicit for downstream generation.
• Helped introduce LangGraph-based agentic orchestration through a working internal proof of concept in which the LLM selects tools and controls pipeline progression.
• Contributed to the initial end-to-end implementation of the story-planning agent system, helping establish the foundation that later became a core production capability.
• Helped establish a separate internal development deployment and a repeatable experiment-to-adoption loop, allowing the team to rapidly build and test candidate capabilities, promote meaningful results into the product, and accelerate the pace of product improvement.

Technology

• Agent: Python, LangChain, LangGraph, Pydantic, typed state, structured outputs, tool calling, bounded retries, and reflection
• 3D and motion: Rigged 3D assets, pose and motion libraries, camera and spatial state, and editable 3D proxies
• Generation and execution: LLM/VLM and image/video provider adapters, FastAPI, Inngest, MongoDB, PyMongo, and object storage
• Evaluation and observability: Internal development deployment, regression evaluation, LangSmith, OpenTelemetry, Datadog, and pytest
• Deployment: Docker, Kubernetes, Helm, and ArgoCD in collaboration with the DevOps team

Public Evidence

CineV’s public materials show the downstream product experience: creators develop stories through a storyboard-first workflow, reuse characters and other project assets, direct shots in 3D, and generate media within a persistent project rather than treating each output as an isolated prompt.

Visit CineV · CineV Getting Started