r/StableDiffusion Sep 10 '22

Prompt-to-Prompt Image Editing with Cross Attention Control in Stable Diffusion

215 Upvotes

View all comments

Show parent comments

6

u/bloc97 Sep 11 '22 edited Sep 11 '22

Great, that's exactly what the authors observed in the DDIM paper! If you don't mind, you are free to setup a quick demo with maybe one or two examples and push it to the github, that would be super cool for everyone to use!

Edit: And for the reason behind why 50 steps doesn't work as well, I guess maybe is that the forward process uses many tricks for acceleration while the inverse process was pretty much neglected and was not optimized (remember the first paper on diffusion models actually needed 1000 sampling steps too for good results), so you actually need to perform the diffusion correctly, for now (eg. 1000 steps).

6

u/Aqwis Sep 11 '22

Yeah, I'm generating a few examples now, and I'll post something in this subreddit and some code on Github later tonight. I didn't actually try your cross attention control code yet, I'll have to do that as well and see how all this fits together. :)

3

u/bloc97 Sep 11 '22

Sounds good, your inversion code can definitively be used standalone but it would be so cool to use it to edit an image!

3

u/ethereal_intellect Sep 11 '22

Wonder if the inversion code could be used to style transfer like in https://github.com/justinpinkney/stable-diffusion . Take clip1 embedding from image1, reconstruct the noise1, take image 2, find clip2, and recreate from noise1 to get a style2 result. Still only just read about it so haven't thought it through, but the reconstruction idea seemed very useful. I will think about it but i'm not sure i'm up to the task of coding it up/trying it out myself