Closed
Description
Describe the bug
When we've pre-generated prompt embeds, the expectation is that we can reuse these indefinitely and that encode_prompts will not try to access the non-existent text_encoder.
Reproduction
from diffusers import StableDiffusionPipeline
from compel import Compel
from transformers import CLIPTextModel
model_id = "runwayml/stable-diffusion-v1-5"
text_encoder = CLIPTextModel.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(
args.pretrained_model_name_or_path,
subfolder="tokenizer",
revision=args.revision,
variant=args.variant,
use_fast=False,
)
compel = Compel(tokenizer=tokenizer, text_encoder=text_encoder)
pipeline = StableDiffusionPipeline.from_pretrained(model_id, text_encoder=None, tokenizer=None)
prompts = ["a cat playing with a ball++ in the forest", "a dog playing with a ball in the forest"]
prompt_embeds = compel(prompts)
images = pipeline(prompt_embeds=prompt_embeds).images
images[0].save("image0.jpg")
images[1].save("image1.jpg")Logs
we hit NoneType has no attribute 'modules':
if (
isinstance(self, LoraLoaderMixin)
and USE_PEFT_BACKEND
):
# Retrieve the original scale by scaling back the LoRA layers
unscale_lora_layers(self.text_encoder, lora_scale)but it is resolved with the following change:
if (
isinstance(self, LoraLoaderMixin)
and USE_PEFT_BACKEND
and self.text_encoder is not None
):
# Retrieve the original scale by scaling back the LoRA layers
unscale_lora_layers(self.text_encoder, lora_scale)System Info
Doesn't run on MacOS:
FileNotFoundError: [Errno 2] Unable to read files /etc/os-release, /usr/lib/os-release
However, I'm running today's Diffusers release on Pytorch 2.4.