Remove conversion to RGB

[yelboudouri]

What does this PR do?

I removed the conversion to RGB in src/diffusers/utils/loading_utils.py. I spent hours trying to figure out why my RGBA images were getting a black background upon loading. I then realized that this function internally converts images to RGB, which should not be the case because this function is only responsible for loading the image. It's up to the user to decide which mode they want to convert their images to after loading them.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[patrickvonplaten]

Hey @yelboudouri,

This change would be a breaking change - could you add a parameter: "return_type='....'" to the function that allows to load RGBA images?

[yelboudouri]

I'm thinking of a way to solve this problem without breaking the code and also solving it once and for all because adding a return_type parameter is not sufficient. Image.convert() has more parameters than just the mode to convert to. So, I'm thinking about adding a parameter convert_method defaulting to None, and then if the convert method is provided, apply it to the image.

The updated code will be something like:

from typing import Union
import os
import requests
import PIL.Image
import PIL.ImageOps

def load_image(image: Union[str, PIL.Image.Image], convert_method=None) -> PIL.Image.Image:
    """
    Loads `image` to a PIL Image.

    Args:
        image (`str` or `PIL.Image.Image`):
            The image to convert to the PIL Image format.
        convert_method (`function`, optional):
            The conversion method for converting the image to RGB. Default is None.

    Returns:
        `PIL.Image.Image`:
            A PIL Image.
    """
    if isinstance(image, str):
        if image.startswith("http://") or image.startswith("https://"):
            image = PIL.Image.open(requests.get(image, stream=True).raw)
        elif os.path.isfile(image):
            image = PIL.Image.open(image)
        else:
            raise ValueError(
                f"Incorrect path or URL. URLs must start with `http://` or `https://`, and {image} is not a valid path."
            )
    elif isinstance(image, PIL.Image.Image):
        pass
    else:
        raise ValueError(
            "Incorrect format used for the image. Should be a URL linking to an image, a local path, or a PIL image."
        )
    
    image = PIL.ImageOps.exif_transpose(image)
    
    # Check if a custom convert method is provided
    if convert_method is not None:
        image = convert_method(image)
    else:
        image = image.convert('RGB')
    
    return image

And when using it:

def custom_convert_method(img):
    return img.convert('L', colors=8)

loaded_image = load_image("image.jpg", convert_method=custom_convert_method)

What do you think @patrickvonplaten ?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Super! Thanks for this!

[sayakpaul]

Let's get the code quality issues fixed.

[DN6]

LGTM 👍🏽 Let's just fix code QC issues.

[yelboudouri]

Can't get that QC to pass, it says 'Import block is unsorted or unformatted,' but the imports are already sorted.

[yiyixuxu]

@yelboudouri did you run make style

[yelboudouri]

@yiyixuxu forgot that 🤦‍♂️ Now it should be all good.

[sayakpaul]

Thank you for your contributions!

[Laisky]

This PR has caused a breaking change. Does this mean that load_image will no longer support PIL.Image? This clearly contradicts the function's comments and errors.

@yelboudouri @patrickvonplaten

fix: #6904