Now, you have the power packed all in one tool to animate, edit, or generate videos from scratch. VACE can be just simplify your entire workflow. Developed by ALI-VILAB, VACE (Video Anything Creation Engine) is a unified model designed to handle nearly every video editing and generation task you can think of. More can be found on their research paper.
And now, two powerful models from this framework are available on Hugging Face that are currently supported:
The model is capable to do ReferenceToVideo generation, VideoToVideo Editing, Masked VideoToVideo editing, and task Composition.
Installation
1. Setup ComfyUI. Older user need to update ComfyUI using the Manager.
2Then install missing nodes by selecting "Custom nodes manager" option.
3. Install and update the Wan custom nodes by Kijai.
4. Download the VACE Wan 1.3B model from Hugging face and save it into "ComfyUI/models/diffusion_models" folder. The 14 billion not yet released. We will update whenever it will be available. You will get much better précised and refined results than 1.3billion model if your VRAM handles.
5. Download Vae, text encoder and clip models from Kijai's repository explained in Wan Kijai's setup.
6. Restart ComfyUI and refresh it to take effect.
Workflow
1. After installing Wan custom nodes, you will find the workflow inside "ComfyUI/custom_nodes/ComfyUI-WanVideoWrapper/example_workflows" folder.
2. Drag and drop into ComfyUI.
3. Load Vace Wan2.1 model into model loader node.
6. Load your reference video in load Video node. Upload a video file here by clicking "choose video to upload". The node will then load and process frames from this video.
Force Rate-Forces the FPS (frames per second). If set to 0, the original FPS of the video is preserved. Useful if you want to override the frame rate.
Custom Width- If set to non-zero, the video frames will be resized to this width. Leave at 0 to keep the original width.
Custom Height - Same as above, but for height. Set to a custom height or leave 0 for original.
Frame_load_cap Maximum number of frames to load from the video. If your video has 1000 frames but you set this to 300, only the first 300 frames will be loaded.
skip_first_frames Skips the first N frames before loading begins. Useful to ignore intros or unwanted initial footage.
select_every_nth Load every nth frame. For example, setting this to 2 will load every second frame, effectively halving the frame count (good for faster processing).
7. Load target image in load image reference node.
To get the first frame of your target video just use Save image node(by searching) and connect it to load image reference node then run it once. You can also use other ControlNets like flux fill or redux etc.
Strength - Control ranges from 0 (minimum)-1.000 (max)
Vace start percent- This means when you want vace controlnet to apply on your video frame.
Vace end percent- This will control when you want vace controlnet to apply on your end video frame.
This means control starts from 0.00 (0%) to 1.00 (100%). For instance- If you set vace end percent to 0.4 means your video will be effected from 0 to 40% and the rest 60% will be not effected.
9. Add relevant prompt into prompt box.
10. Finally, run the workflow by clicking on Run button.
){kind=link}