huggingface/diffusers

Expanded init fields in StableDiffusionPipeline cause incompatibilities with many/most inherited pipelines

開放

#6,969 建立於 2024年2月13日

 (22 則留言) (0 個反應) (1 位負責人)Python (4,562 個分叉)batch import
bugcontributions-welcomegood first issuehelp wantedwip

倉庫指標

星標
 (22,190 顆星)
PR 合併指標
 (平均合併 14天 10小時) (30 天內合併 101 個 PR)

描述

Describe the bug

class StableDiffusionPipeline has its init section as:

    def __init__(
        self,
        vae: AutoencoderKL,
        text_encoder: CLIPTextModel,
        tokenizer: CLIPTokenizer,
        unet: UNet2DConditionModel,
        scheduler: KarrasDiffusionSchedulers,
        safety_checker: StableDiffusionSafetyChecker,
        feature_extractor: CLIPImageProcessor,
        image_encoder: CLIPVisionModelWithProjection = None,
        requires_safety_checker: bool = True,
    ):

and here image_encoder was recently introduced thus changing the class signature
but most community pipelines do not include init field for recently included image_encoder and thus order or params is wrong

for example, examples/community/regional_prompting_stable_diffusion.py has this in its init:

        super().__init__(
            vae,
            text_encoder,
            tokenizer,
            unet,
            scheduler,
            safety_checker,
            feature_extractor,
            requires_safety_checker,
        )

which means bool value from requires_safety_checker is going to be passed as image_encoder and pipeline will fail during initialization like this:

> diffusers/pipelines/pipeline_utils.py:546 in _fetch_class_library_tuple
AttributeError: 'bool' object has no attribute '__module__'

this is a conceptual problem with changing master class signature while all inherited classes pass args list as simple list.
i don't see a simple solution as going back is bad and going forward requires updates to a lot of pipelines.

but at the very basic, at least add error handling to _fetch_class_library_tuple so invalid type does not cause entire solution to crash.

Reproduction

load nearly any community pipeline or any pipeline inherited from StableDiffusionPipeline, but not updated to use new signature in its super().__init__ call

Logs

No response

System Info

diffusers==0.26.3

Who can help?

@yiyixuxu @DN6 @sayakpaul @patrickvonplaten

貢獻者指南