 |
| Canny illustration |
 |
| Depth illustration |
 |
| Pose illustration |
 |
| Soft Edge illustration |
First of all, get the Qwen basic workflow setup in ComfyUI.
There are control nets and LoRAs from community that has been described below:
A. ControlNets By InstantX
It is the unified ControlNet supporting canny, depth, pose and soft edge for Qwen Image. The ControlNet is built with 5 double blocks taken from the pre-trained transformer layers.
Its been trained from scratch on a 10M high-quality dataset (mix of general and human images) with training setup: Steps: 50K , Resolution: 1328×1328, Precision: BFloat16 , Batch size: 64 Learning rate: 4e-5 and Text drop ratio: 0.10 values.
(a) Download Qwen Image ControlNet Union (Qwen-Image-InstantX-ControlNet-Union.safetensors) . Then, save this to your ComfyUI/models/controlnet folder.
(b) Download Lotus Depth V1 (lotus-depth-d-v1-1.safetensors) . Then, save this to your ComfyUI/models/diffusion_models folder.
(c) Download VAE (vae-ft-mse-840000-ema-pruned.safetensors). You can also use any SD1.5 VAE for this. Now, save this to your ComfyUI/models/vae folder.
B. Qwen Image DiffSynth ControlNet Model Patches
This is just the patch models that mimics like the Control nets(depth, canny and inpaint) but not the actual controlnet model.
Download these models:
(a) Qwen image canny (qwen_image_canny_diffsynth_controlnet.safetensors)
(b) Qwen image depth (qwen_image_depth_diffsynth_controlnet.safetensors)
(c) Qwen image inpaint (qwen_image_inpaint_diffsynth_controlnet.safetensors)
Save these into your ComfyUI/models/model_patches folder. If not available then just create new folder as model_pathes.
C. Qwen Image Union ControlNet LoRA
It supports canny, depth, pose, lineart, softedge, normal, openpose features.
-Qwen Image Union LoRA (qwen_image_union_diffsynth_lora.safetensors)
Put this into ComfyUI/models/loras folder.
You need comfyui_controlnet_aux nodes for preprocessing as the native nodes not supported. This custom node can be installed from Manager.
Workflow
1. All the workflows can be found from our Hugging Face repository.
(a) Qwen Image Controlnet Patch (Qwen_Image_Controlnet_Patch.json) for depth, canny, inpaint features.
(b) Qwen Image Controlnet by InstantX (Qwen_Image_InstantX_Controlnet.json) for canny, depth, pose and soft edge features.
(c) Qwen Image Union Control Lora (Qwen_Image_Union_Control_Lora.json)
2. Drag and drop into ComfyUI.
 ){kind=link}