[enhancement]: Z-Image IP Adapter

[iwr-redmond]

Is there an existing issue for this?

• I have searched the existing issues

Contact Details

No response

What should this feature add?

Research by @DragonDiffusionbyBoyo suggests that SD3 IP Adapters can be used with Z-Image Turbo after some code modifications.

The ComfyUI code designed for this purpose is MIT-licensed and available here and here. It may be possible to port this code to Invoke/Diffusers to round out the Z-Image Turbo functionality.

cc @Pfannkuchensack

Alternatives

No response

Additional Content

No response

[DragonDiffusionbyBoyo]

With regards to this, I have been trying to convert existing IP adaptors with limited results however I am also training an IPAdapter when I get time on the GPUS. This may or may not work. The underlaying structure with the conversions and modifications I have made should work according to the maths work. TBC