HiDream E1.1 Image Editing on low VRAMs in ComfyUI

Image editing models often struggle when it comes to following complex instructions. Some models over-edit and ruin parts that should not change, while others fail to accurately apply the desired modification. And when you throw in tasks like style transfer, text edits, or object manipulation, most models fall short. HiDream-E1, open-sourced (MIT licensed) editing based model built on the top of HiDream I1 fix that. 

It brings a balanced approach executing instructions precisely while keeping the rest of the image untouched. The newer HiDream-E1.1 takes it a step further, delivering even better visual quality and support for dynamic resolutions.

There are two version released - 

(a) HiDream E1.1 (released on July 16) handles 1 megapixel resolution with more precised generation

(b) HiDream E1 (released on April 28) handles 768 pixel resolution. 

Some of the working samples provided below for clear understanding:

Working Sample-1

Working Sample-2

Working Sample-3

Working Sample-4

HiDream-E1 was tested on two strong benchmarks: EmuEdit and ReasonEdit. 

(a) EmuEdit-  3589 samples across 10 task types 

(b)ReasonEdit- 197 samples focused on tough instructions that require reasoning Evaluation used GPT-4o to rate edits on two things (0-10 scale)

It led in tasks like global edits, text manipulation, color adjustment, style transfer, object removal, and more. Interested people can find more details into their research paper.

Table of Contents:

Installation

New user need to install ComfyUI. If you already using ComfyUI, then you need to update it from the Manager section by clicking on "Update All" option to avoid any missing nodes error.

Native Support

Official support already provided by ComfyUI, but it requires higher VRAMs. If you have low end GPU, you can download the FP8 or GGUF model variant.

1. Download any HiDream E1/ HiDreamE1.1(bf16) or HiDreamE1.1(fp8) model from Hugging Face repository and save it into your ComfyUI/models/diffusion_models folder.

Choose the variant that suits your use case and system specification:

Filename	Size	Precision	Variant	Description
hidream_e1_1_bf16.safetensors	34.2 GB	bf16	E1_1	HiDream E1.1 full version
hidream_e1_full_bf16.safetensors	34.2 GB	bf16	E1 Full	HiDream E1 full version
hidream_e1_1_bf16-fp8.safetensors	17.1 GB	fp8	E1.1	HiDream E1.1 optimized for low VRAMs

2. Download Text encoders  ( clip_l_hidream.safetensors, clip_g_hidream.safetensors, t5xxl_fp8_e4m3fn_scaled.safetensors, llama_3.1_8b_instruct_fp8_scaled.safetensors)

and save it into your ComfyUI/models/text_encoders folder.

3. Download VAE and save it into ComfyUI/models/vae folder but its not needed if you already using Flux/SD3.5  as this is the same VAE model.

4. Restart and refresh your ComfyUI to take effect.

GGUF(For Low VRAMs)

1. You need to setup ComfyUI custom node ComfyUI-GGUF by city96. If you already setup and using any other GGUF model, then its not required. 

You can follow our quantized model tutorial to know more about GGUF models.

2. Download the GGUF HiDreamE1.1 from Quantstack's Hugging face repository and save this into your ComfyUI/models/unet folder. There are are multiple variant to choose from Q2(for faster inference with low quality generation) to Q8(for best quality for higher VRAMs).

3. Text Encoders and VAE setup will be same as described above.

4. Restart and refresh your ComfyUI to take effect.

Workflow

1. Workflow can be found inside your ComfyUI inside template section.  Just click and open the HidreamE1.1 edit workflow. You can also download the workflow from our Hugging face repository. (HiDream-E1.png or HiDream-E1.1.png)

2. Drag and drop into ComfyUI. 

Upload your image as input.

Load HiDream Model into Load diffusion model node. 

Also load text encoders and VAE.

Add relevant Prompts into prompt box.

Test 1- Human Photography : 

Input Image

We uploaded a girl's image wearing white shirt and added prompt into positive prompt box. Here, our objective is to change the shirt color and printing style. Lets see how this performs.

Prompt: Change shirt into red and black striped style 

Output Image

Here, you can see the prompt has been followed well but the color has been vibrant and much glossy. 

Test 2 - Product Photography: 

Input Image

Prompt: Convert the perfume bottle to product photography where its on a stone with water droplets and green leaves, soft morning light, natural setting, clean branding

Output Image

Because the model has been tested with GPT4o model, for best results you should use GPT4 based prompting.

If you are looking for precise, instruction-based image editing then HiDream-E1 delivers. And if you want improved quality and flexibility, the newly released HiDream-E1.1 improves upon E1 with dynamic resolution support and sharper, more accurate results. 

The model sometime struggles to handle the consistency in terms of lightings that's need to be corrected in future updates.