Now, you have the power packed all in one tool to animate, edit, or generate videos from scratch. VACE can be just simplify your entire workflow. Developed by ALI-VILAB, VACE (Video Anything Creation Engine) is a unified model designed to handle nearly every video editing and generation task you can think of. More can be found on their research paper.
And now, two powerful models from this framework are available on Hugging Face that are currently supported:
(a) VACE-Wan2.1-1.3B-Preview, WAN VACE 14B and Wan Vace 1.3B
(b) VACE-LTX-Video-0.9
The model is capable to do ReferenceToVideo generation, VideoToVideo Editing, Masked VideoToVideo editing, and task Composition.
Installation
1. First, install and setup ComfyUI. Older user need to update ComfyUI using the Manager.
2. Then, install missing nodes by selecting "Custom nodes manager" option.
3. Install and update the Wan custom nodes by Kijai.
4. Download any of the WAN VACE models from Hugging face repository:
| Model | Description |
|---|---|
| (a) Wan2.1-VACE-14B | Official release generates more detailed results |
| (b) Wan2.1-VACE-1.3B | Official release takes less inference time with low quality |
| (c) WAN2.1-VACE1.3B/14B | Quantized by Kijai |
| (d) Wan2.1-VACE-1.3B preview | Unofficial release for faster generation |
| (e) Wan2.1-VACE-14B-GGUF | GGUF Custom node by City96, installation required |
Save it into "ComfyUI/models/diffusion_models" folder. If using Wan Vace GGUF variant, then save it into your "ComfyUI/models/unet" folder.
You will get much better promising and refined results in 14Billion model than 1.3billion model if your VRAM handles.
5. You also need to download VAE, text encoder and clip models from Hugging Face repository already explained. If you already installed the WAN basic setup then its not required.
6. Restart ComfyUI and refresh it to take effect.
Workflow
1. After installing Wan custom nodes, you will find the workflow inside "ComfyUI/custom_nodes/ComfyUI-WanVideoWrapper/example_workflows" folder. If using the GGUF variant then get the GGUF workflow from Hugging face repository.
2. Drag and drop into ComfyUI. Upload your target image and reference video into loader node.
3. Load Vace Wan2.1 model into model loader node. Use the Sage Attention mode for faster inference if you have not installed then just choose the sdpa mode.
6. Load your reference video in load Video node. Upload a video file here by clicking "choose video to upload". The node will then load and process frames from this video.
Force Rate-Forces the FPS (frames per second). If set to 0, the original FPS of the video is preserved. Useful if you want to override the frame rate.
Custom Width- If set to non-zero, the video frames will be resized to this width. Leave at 0 to keep the original width.
Custom Height - Same as above, but for height. Set to a custom height or leave 0 for original.
Frame_load_cap - Maximum number of frames to load from the video. If your video has 1000 frames but you set this to 300, only the first 300 frames will be loaded.
skip_first_frames - Skips the first N frames before loading begins. Useful to ignore intros or unwanted initial footage.
select_every_nth - Load every nth frame. For example, setting this to 2 will load every second frame, effectively halving the frame count (good for faster processing).
7. Load target image in load image reference node.
To get the first frame of your target video just use Save image node(by searching) and connect it to load image reference node then run it once. You can also use other ControlNets like flux fill or redux etc.
Strength - Control ranges from 0 (minimum)-1.000 (max)
Vace start percent- This means when you want vace controlnet to apply on your video frame.
Vace end percent- This will control when you want vace controlnet to apply on your end video frame.
This means control starts from 0.00 (0%) to 1.00 (100%). For instance- If you set vace end percent to 0.4 means your video will be effected from 0 to 40% and the rest 60% will be not effected.
9. Add relevant prompt into prompt box.
10. Finally, run the workflow by clicking on Run button.
){kind=link}