🎬 Look-Before-Move

Look-Before-Move is a narrative-grounded camera planning pipeline for Blender scenes. Given a story/script directory, it builds scene context, searches camera viewpoints, plans motion, renders shot clips, and evaluates the generated result with segment-level cinematic metrics.

The released repository contains code only. Scene assets, story datasets, generated videos, model checkpoints, paper figures, logs, and benchmark outputs are intentionally excluded.

✨ Overview

The system is organized as four executable stages:

1. Director: parses story inputs, builds scene context, shot contracts, and blocking plans.
2. Cinematographer: performs candidate viewpoint generation, validation, Monte Carlo/quality search, VLM reflection, semantic height adjustment, and trajectory grounding.
3. VideoEngineer: converts camera plans into executable Blender trajectories and renders clips.
4. Editor: assembles rendered clips and emits the final edit manifest.

Engine/run_full_pipeline.py runs these stages end to end. Evaluation/ contains CineStoryEval, the metric pipeline used for SP, IC, and TQ reporting.

🧱 Repository Structure

Look-Before-Move/
|-- Director/                 # Story parsing, scene context, shot contracts, blocking
|-- Cinematographer/          # Viewpoint search, candidate validation, trajectory planning
|-- VideoEngineer/            # Blender rendering and clip handoff generation
|-- Editor/                   # Timeline assembly
|-- Engine/                   # End-to-end pipeline runner
|-- Evaluation/               # CineStoryEval metric code and Qwen helper scripts
|-- tools/                    # Batch runners, audit tools, ablation summaries
|-- config/                   # Example runtime configuration
|-- requirements.txt          # Python dependencies for code + evaluation
`-- .env.example              # Environment variable template

🚀 Quick Start

1) Requirements

• Windows or Linux with Python 3.11+.
• Blender 4.5 is recommended. The code calls Blender in background mode and uses bpy inside Blender subprocesses.
• NVIDIA GPU is recommended for quality mode and local Qwen evaluation.
• FFmpeg should be available on PATH for robust video handling.

Install the Python environment:

git clone https://github.com/EngineeringAI-LAB/Look-Before-Move.git
cd Look-Before-Move

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

If your CUDA/PyTorch version differs, edit the first lines of requirements.txt or install a PyTorch wheel that matches your driver before installing the remaining packages.

2) Configuration

Copy the environment template and edit local paths/secrets:

Copy-Item .env.example .env
notepad .env

Important variables:

• STORYBLENDER_BLENDER_EXE: absolute path to blender.exe, or leave unset if blender is on PATH.
• ANYLLM_API_KEY, ANYLLM_API_BASE, ANYLLM_PROVIDER: OpenAI-compatible or AnyLLM-compatible VLM endpoint used by Director/Cinematographer.
• STORYBLENDER_VISION_MODEL: vision model name for VLM reflection and selection.
• LBM_SCRIPTS_ROOT: directory containing story asset folders.
• CINESTORY_QWEN_MODEL: local path or Hugging Face model id for the evaluator. Default: Qwen/Qwen2.5-VL-7B-Instruct.
• CINESTORY_QWEN_CACHE_DIR: local Hugging Face cache for Qwen weights. Default: Evaluation/models/huggingface.
• CINESTORY_YOLO_WEIGHTS: local YOLO weight path. Default: Evaluation/models/yolo/yolo11x.pt.

The code also supports config/runtime_config.json; use config/runtime_config.example.json as a template. Do not commit real API keys.

🧠 Qwen and Evaluator Weights

CineStoryEval can run event-alignment scoring with a local Qwen video-language model. The evaluator loads with local_files_only=True, so weights must be prefetched before evaluation.

Install Qwen dependencies:

.\Evaluation\scripts\install_qwen_local.ps1

Download Qwen weights:

.\Evaluation\scripts\prefetch_qwen_model.ps1 `
  -ModelId "Qwen/Qwen2.5-VL-7B-Instruct" `
  -CacheDir ".\Evaluation\models\huggingface"

Equivalent Hugging Face CLI command:

huggingface-cli download Qwen/Qwen2.5-VL-7B-Instruct `
  --cache-dir .\Evaluation\models\huggingface

YOLO weights are required for strict subject detection. Place yolo11x.pt at:

Evaluation/models/yolo/yolo11x.pt

or set:

$env:CINESTORY_YOLO_WEIGHTS = "D:\path\to\yolo11x.pt"

Model directories are ignored by git.

📁 Input Data Layout

Each story directory should contain the Blender scene, story text, formatted models, layout scripts, and related assets. A typical directory looks like:

scripts/
`-- The Godfather/
    |-- story.txt
    |-- formatted_model/
    |-- layout_script/
    |-- animated_models/
    |-- supplementary_assets/
    `-- *.blend

The repository does not include these assets.

🎥 Run One Story

$env:STORYBLENDER_BLENDER_EXE = "<path-to-blender>\blender.exe"
$env:ANYLLM_API_KEY = "<your-api-key>"
$env:ANYLLM_API_BASE = "https://your-openai-compatible-endpoint"
$env:ANYLLM_PROVIDER = "openai"

.\.venv\Scripts\python.exe Engine\run_full_pipeline.py `
  --demo-root "<path-to-scripts>\The Godfather" `
  --run-id "the_godfather_quality" `
  --camera-quality quality

Useful quality options are exposed by Cinematographer/cinematographer_stage.py, including:

• --camera-quality fast|quality
• --run-pre-continuity-story-judge
• --disable-vlm-reflection
• --disable-trajectory-grounding
• --disable-semantic-height-adjust

🏃 Run All Stories in Quality Mode

Set the story root and run the batch orchestrator:

$env:LBM_SCRIPTS_ROOT = "<path-to-scripts>"
$env:LBM_PYTHON_EXE = ".\.venv\Scripts\python.exe"

.\.venv\Scripts\python.exe tools\run_all_scripts_parallel_generation.py `
  --ablation quality `
  --run-tag full_quality `
  --fixed-stories-from ".\missing_manifest_for_auto_discovery.json" `
  --generation-workers 2 `
  --evaluation-workers 1 `
  --llm-max-workers-per-story 2 `
  --eval-min-free-vram-gb 18

When --fixed-stories-from does not exist, the script discovers all directories under LBM_SCRIPTS_ROOT except demo.

📊 Run Evaluation

Build benchmark input from a video handoff:

.\Evaluation\scripts\build_benchmark_input.ps1 `
  -VideoHandoff "VideoEngineer\output\<run_id>\<story>\outputs\video_handoff_v1.json" `
  -StoryName "the_godfather" `
  -Output "Evaluation\config\benchmark_input_the_godfather.json"

Evaluate with local Qwen:

.\.venv\Scripts\python.exe Evaluation\run_cinestory_eval.py `
  --benchmark-input Evaluation\config\benchmark_input_the_godfather.json `
  --output-root Evaluation\output\the_godfather `
  --reference-root Evaluation\reference `
  --story-name the_godfather `
  --vlm-backend qwen_local

Use --vlm-backend none for a no-VLM smoke test.

📦 Outputs

Generated artifacts are written under stage-local output/ directories and are ignored by git:

• Director/output/.../outputs/director_handoff_v1.json
• Cinematographer/output/.../outputs/camera_handoff_v1.json
• VideoEngineer/output/.../outputs/video_handoff_v1.json
• Editor/output/.../exports/final_edit_v1.mp4
• Evaluation/output/.../cinestory_report_v1.json

📚 Citation

@misc{bian2026lookbeforemovenarrativegroundedworldvisual,
      title={Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds}, 
      author={Jiaming Bian and Bingliang Li and Yuehao Wu and Pichao Wang and Zhi Wang and Hailan Ma and Huadong Mo and Zhenhong Sun},
      year={2026},
      eprint={2606.26964},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2606.26964}, 
}