Fix a bug in the state dict judgment in ip_adapter.py.

[zhangp365]

What does this PR do?

When the ipadapter state dict is in a different order, the current code will raise an exception. Updating the conditional statement will fix this issue.

@yiyixuxu

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[hlky]

Thanks! 🤗

Minimal reproduction

from diffusers import StableDiffusionPipeline
from transformers import CLIPVisionModelWithProjection

image_encoder = CLIPVisionModelWithProjection.from_pretrained(
  "openai/clip-vit-large-patch14"
)

pipeline = StableDiffusionPipeline.from_pretrained(
  "stable-diffusion-v1-5/stable-diffusion-v1-5",
  image_encoder=image_encoder,
  variant="fp16",
  safety_checker=None,
)

ip_adapter = {"image_proj": {}, "ip_adapter": {}}
try:
  pipeline.load_ip_adapter(ip_adapter, "", "")
except KeyError:
  pass

# raises ValueError on main
ip_adapter = {"ip_adapter": {}, "image_proj": {}}
try:
  pipeline.load_ip_adapter(ip_adapter, "", "")
except KeyError:
  pass

Note: KeyError is expected in this reproduction because we're passing an empty state dict.

[zhangp365]

I didn't understand the "Update branch" button before, and I clicked it. Sorry about that.

[yiyixuxu]

thanks @zhangp365 @hlky