MotionCtrl: A Unified and Flexible Motion Controller for Video Generation

1. Clone this repo into custom_nodes directory of ComfyUI location

2. Run pip install -r requirements.txt

3. Download the weights of MotionCtrl motionctrl.pth and put it to ComfyUI/models/checkpoints

Four nodes Load Motionctrl Checkpoint & Motionctrl Cond & Motionctrl Sample Simple & Load Motion Camera Preset & Load Motion Traj Preset & Select Image Indices &Motionctrl Sample

Motion Traj Tool Generate motion trajectories

Motion Camera Tool Generate motion camera points

base workflow

https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_motionctrl_base.json

unofficial implementation "MotionCtrl deployed on AnimateDiff" workflow:

https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_motionctrl.json

1. Generate LVDM/VideoCrafter Video
2. Select Images->Scribble
3. Use AnimateDiff Scribble SparseCtrl