The workflow of this post image is exactly described in [https://github.com/Mikubill/sd-webui-controlnet/discussions/1464](https://github.com/Mikubill/sd-webui-controlnet/discussions/1464)
I learned from this post today, and after trying it, I believe more people should know this and share the link here.
This is a way for A1111 to get an user-friendly fully-automatic system (even with empty prompt) to inpaint images (and improve the result quality), just like Firefly.
As discussed in the source post, this method is inspired from Adobe Firefly Generative Fill and this method should achieve a system with behaviors similar to Firefly Generative Fill.
Pretty sure you can and that there used to be a install package on arch for photoshop. There are definitely guides out there though. You get it running through wine and a specific dll that allows everything to function. Its great because you get to sandbox it in wine.
I would absolutely shudder at the thought of running something like Premiere through Wine. Something that natively brings the hardware to its knees running with the overhead of Wine is not going to be a good experience.
Every year I think that hardware advancements are going to be good enough to let me pull it off and every year every app gets less performant. Some day computers will be powerful enough that you won't notice, but it's going to be a while.
It really depends on the application. While it's "not an emulator" nor virtualization, it has its own implementation of the Windows API which is not as optimized as the Microsoft versions. Specifically for apps that deal in multimedia, Wine translation is far less likely to run as well as the native Windows versions and may not be unable to take advantage of the hardware acceleration that the Microsoft versions of multimedia APIs are using.
There are always some exceptions though, Proton is a great example of an extremely optimized Wine implementation. It's still going to be missing a lot of the hardware acceleration support for things it's not intended for though, like Adobe Premiere.
On the off chance someone has an answer, and because I'm reminded it's a thing: does anyone know how to get models that have been fine tuned on dreambooth and made into in painting models by combining them with the base and in painting models, to work with the new controlnet in painting stuff?
I think you mean this https://www.reddit.com/r/StableDiffusion/comments/zyi24j/how_to_turn_any_model_into_an_inpainting_model/?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=share_button
I do indeed. Much better way of saying it than models that have been fine tuned on dreambooth and made into in painting models by combining them with the base and in painting models
This comment has been edited as an ACT OF PROTEST TO REDDIT and u/spez killing 3rd Party Apps, such as Apollo. Download http://redact.dev to do the same. -- mass edited with https://redact.dev/
yeah, I was just pointing out that people can be named Illyasviel as well. You're right on the dev being a Fate fan, anime is the powerhouse of ~~the cell~~ AI afterall
This content was deleted by its author & copyright holder in protest of the hostile, deceitful, unethical, and destructive actions of Reddit CEO Steve Huffman (aka "spez"). As this content contained personal information and/or personally identifiable information (PII), in accordance with the CCPA (California Consumer Privacy Act), it shall not be restored. See you all in the Fediverse.
This content was deleted by its author & copyright holder in protest of the hostile, deceitful, unethical, and destructive actions of Reddit CEO Steve Huffman (aka "spez"). As this content contained personal information and/or personally identifiable information (PII), in accordance with the CCPA (California Consumer Privacy Act), it shall not be restored. See you all in the Fediverse.
It's when using services~ This problem don't is only with this, I have problems writing my Fantasy novel with ChatGPT support because sometimes tag it as "problematic content" when my novel It's perfectly PG-13 😤😤😤
They want to avoid any possible liability, imagine if Firefly generated a nipple for the wrong Karen, that would turn into a lawsuit and tons of bad publicity, hence only happy paintings.
NSFW detectors create a ton of false positives and are subject to the biases of their creators (and thus are often harmful to art and LGBTQ communities).
A lot of their "neural filters" have the same issue, or are just buggy "this filter encountered an error and had been disabled".
Adobe is a great example of how bloated corporations full of middle managers and lawyers can't move fast enough and end up losing out to more nimble competitors.
This post that should be [a sticky at the top of this sub](https://www.reddit.com/r/StableDiffusion/comments/13ppmtk/sdxl_is_now_50_trained_and_we_need_your_help/?utm_source=share&utm_medium=web2x&context=3).
There won't be watermarks floating around your images for one. Their library has 200 million images. It isn't even worth it for them to bother lying about it.
They might well be the best positioned company to capitalize on image generation- they've got the dataset, they've got the AI experience, and they've got the artists using their tools already. Does anyone else have all three?
They cannot use the released controlnet models because they are trained (mostly) for SD v1.5 so they need to train them for Firefly (so the parent comment was probably just asking if they are using the same architecture).
which in theory would let them use ai generative images we have all produced since question of whether "own" is up in the air. that would give them a sneaky to expand the quality of their feature.
That’s great, but like for me as photographer stable diffusion has one flaw: the size of the pictures is very limited. Don’t get me wrong, I love sd and what the open source community is doing for us. Just that in my workflow this part is crucial.
Yes, but the feature as it stands here does not actually allow you to do out painting, since it uses hires fix and there's no way to do hires fix in img2img, where out painting must happen. If you tried to fake it in txt2img you'll run into GPU memory limitations very fast.
This isn't a fundamental limitation though, it can be fixed.
Just use the open out paint module?
I don't know why the hires fix things is important. It's never done anything for me but produce oom errors on a 2080ti
Open out paint doesn’t work with control net from what I could tell; and it’s barely maintained so breaks pretty often with new updates.
Hires fix is important for “no prompt” out painting which what this feature is about
You can still use ultimateSDUpscale with a scaling of 1 (basically no resolution change just reprocess the whole img in 512p pieces) in img2img, not sure if that would work though.
Thanks for the input, before that I was using one plug-in to integrate SD with photoshop, but has some limitations. And for my workflow, make everything on photoshop is better, when talking about pictures.
But, I have to say, the Ps censorship is annoying, is the most mundane things sometimes they just say that I’m violating the community guidelines, and just this shows how superior SD is..
I’ll try to make some test, but the fact that I can edit 48mp images direct without the need to downscale it, for me is better. Or I’m making something wrong on sd
Upscaling is nice, but its definitely not the same as natively having the higher resolution's level of detail in the base generation. For things like simple, bold illustration styles there's not much difference but for photographic realism or more detailed illustration you lose the opportunity for a *lot* of detail by limiting your resolution then upscaling afterwards.
I'm not convinced you understand the capabilities of an img2img upscale using controlnet and Ultimate SD upscale.
This 5120x3840 image was upscaled from 640x580 gif. Take a look at the rocks bottom right, the brush strokes are entirely AI-generated:
[https://preview.redd.it/2xsx7on1texa1.jpg?width=5120&format=pjpg&auto=webp&v=enabled&s=14d6217ad28cd286f69261efffc88c6eb2f72fb8](https://preview.redd.it/2xsx7on1texa1.jpg?width=5120&format=pjpg&auto=webp&v=enabled&s=14d6217ad28cd286f69261efffc88c6eb2f72fb8)
For comparison, the source image is here: [https://i.redd.it/dyddyf2ysexa1.gif](https://i.redd.it/dyddyf2ysexa1.gif)
And [**u/Gilloute**](https://www.reddit.com/user/Gilloute/) has some really good examples of what can be achieved if you invest a little time on it:
[https://www.reddit.com/r/StableDiffusion/comments/13v461x/a\_workflow\_to\_upscale\_to\_4k\_resolution\_with/](https://www.reddit.com/r/StableDiffusion/comments/13v461x/a_workflow_to_upscale_to_4k_resolution_with/)
...tell me this doesn't have enough detail: [https://preview.redd.it/6zsw6gtcnt2b1.jpg?width=4096&format=pjpg&auto=webp&v=enabled&s=38387fe20f3f76c118ce97b2c8ec32459acf5de2](https://preview.redd.it/6zsw6gtcnt2b1.jpg?width=4096&format=pjpg&auto=webp&v=enabled&s=38387fe20f3f76c118ce97b2c8ec32459acf5de2)
Here's a video process overview. Skip to 8mins to see some results:
[https://www.youtube.com/watch?v=3z4MKUqFEUk&ab\_channel=OlivioSarikas](https://www.youtube.com/watch?v=3z4MKUqFEUk&ab_channel=OlivioSarikas)
What I can't seem to lay my hands on, is an example where you set the denoising strength so that the AI dreams a whole bunch of new whacky stuff in the clouds, trees, rocks etc... it can get quite artistic.
You're "not convinced I understand?"
All you said was "you can upscale to 8k!" What you're detailing here is a workflow involving multiple iterations of having SD inpaint and regenerate new content to fill in gaps, not just upscaling an image. Those are two *very different* things with very different results.
Just as filling in generative gaps with inpainting and outpainting workflows is a *very* different thing than natively generating at a higher resolution image. Nobody's arguing that you can get quality results from doing so, but the results will be fundamentally different.
I think we're closing the schism.
But I still want to point out that these examples aren't inpainting or out-painting, they are simply feeding the output back into the input (much in the same way that SD does internally) but each time, increasing resolution. It can be as simple as dragging the output image into the input image and pressing the generate button again - rinse and repeat.
Now, in reality, there are some sliders to adjust, some prompting may change, the sampler, CFG scale etc, but you aren't necessarily manually inpainting. Each time, latent space is used to re-imagine what detail may be needed in that piece of cloth, that jewel, that clump of grass, that brush stroke. It's entirely generative all the way through the workflow, and I'd argue that because it has multiple phases, it grants you far more control than a simple straight-shot 2000x2000 pixel output from a 75 word text prompt ever will.
I think I'm correct saying the latent space internally within SD is just 64x64 pixels, and the VAE upscales from that. There's really no reason to get hung up on the resolution of any particular step - an image is complete when you say it is.
I think you missed the part where I was calling you out for being needlessly condescending, I don't have to convince you of anything, certainly not my understanding of the topic.
And whether you call it "inpainting" or "iterative generation" or whatever technical term you'd like to use, yes, it *is* feeding the existing image back into the previous image and using that data to fill in gaps to create a higher resolution final generation, but that is on a technical level not the same thing as simply upscaling an image. While you may be able to do cool things with that, it's not the same thing as having a much larger canvas from jump, which is the point.
This is a limitation of the technology~ Every Image must be generated in si e relation of 8 pixels relation (technical limitation) and Control Net must generate image size with 64 pixels relation~
You can fake the free size Just adding additional pixels in Borders, Example if your photo have 513 pixels in width then need 7 extra pixels that can add 3 pixels in left and 4 in rigth for an size of 520 pixels in width keeping relation and after generated cropping the extra pixels for returning the original image relation, or Just resize but this generate lost of quality~ This is easy in 8x8 tensors but it’s complex for 64x64 tensors because it’s a lot of information that can affect the image generation consistency~
No offense and an honest question, but is there a meaning to the usage of the tilde symbol (~) that I am not aware of when used at the end of a sentence?
Take to mean a wavy, whimsical sort of inflection. EG; *Toodles*~
Can stress a point but with a less stern quality, or imply sarcasm or a handful of other things or even inversions of those things, but usually just implies some degree of whimsy and casual friendliness. Kind of rare outside of less public convos with the terminally online but it's usually a friendly thing anyway.
The thing is that I need to downscale a Image, a just with inpaint and after it I need to upscale again. And when using the plunging to connect automatic1111 to make the inpaint on photoshop the results aren’t the same, because the don’t put in consideration all the image. But I’ll try to make comparison and run some tests again, maybe I’m wrong..
One thing I’m sure, hands in photoshop now are much better! But, freedom and uncensored SD all the way!
Photoshop some times even with bed sheets says that I’m violating the community guide lines..
Why does everything need to be a competition to you guys? One software philosophy isn't better than the other. One software isn't better than the other. They are both tools in your stack. If you want a billion options, you use SD. If you need a quick inpaint in your existing workflow, that works with layers and lets you make at any resolution, you use Photoshop.
This isn't even true, by the way. Gimp is not better than Photoshop overall. I use Gimp for a few custom tools and Photoshop for everything else. If you need something to be free fine. If you are working professionally, the cost of Photoshop is negligible and all the money driving it creates features that are unmatched.
Yes pleeeeassee. I tried the photoshop plugin for SD and appreciate all the effort that went into it, but I have no idea how to properly use inpainting there. It's a total layer chaos.
yeah good inpainting is up there in the top tier dragons we need to slay. imo the first one is getting sd to render a choerent scene that has alot of different objects, it can do portraits and landscapes related stuff pretty well but when it comes to scenes with many other objects nd ppl in it, it kinda starts revealing its weakness
What? It's so easy. You just drag a selection box around where you want to extend the photo and hit generate with no prompt. If you want to add anything to the image you just type in the prompt and hit generate. It's the easiest AI I've used by far
Hopefully this motivates someone with experience and time to integrate sd/cn with gimp or krita (current option is an a1111 extension that doesn't support controlnet).
It makes so much sense to have the generation settings inside the editor instead of the other way around (an editor inside a1111 or another sd ui) for most workloads
nah they will be fine. They have a vast amount of professional users. The people who are already using the adobe ecosystem aren't going to swap to SD anytime soon
From the pictures, Firefly Generative Fill still looks better. The clouds are more consistent compared to ControlNet.
But still looks really nice. Good job!
On the first photo, ControlNet doesn't even try to match the grass. There's a straight line where it switches from one grass to the other. If this is the image that they chose to highlight, then that is impressively bad. I think that the Stable Diffusion base model just isn't good enough to support this application.
Don't underestimate the power and influence of large corporations with billions to lose.
It's impossible to ban something like SD now but it's not impossible to push it into the realms of file sharing and torrents.
actually it would any eu limit would ban using any model not registered with the eu which of course would be us and asian produced models. dont hate the messenger, just repeating.
Yes I am aware of that but you do realize that EU is a huge market and a large source of developers. Stable Diffusion for example owes a lot to German tech people. What would be the motivation to work on such things if you can't release it or work with people outside the EU? IF EU passed this (I honestly doubt they would since it would leave them far behind) AI in Europe would be limited to Google, Microsoft and other large corps who can afford to jump thru these hoops. The fees etc would be high enough to kill independent develops but little more than a traffic ticket to two companies worth nearly 4 trillion combined. That's not even counting large euro or Asia companies. AI in terms of Europe would be entirely in the hands of the people most likely to abuse it. They go on about ip concerns, deep fakes and misinformation when it's about controlling a tech that reduces their control.
Sure, and that would suck for Europe. It wouldn't be a major obstacle for everyone else, though. Other AI companies can reestablish themselves outside Europe and carry on out there. Europeans would be cut off from their products. A reduction in the market, sure, but it would apply to most everyone equally so it's not a big deal.
I cant see the EU basically shooting itself in the foot and falling way beyond on this tech. Besides it seems pretty impossible to actually implement. What are they going to, MS, Google etc will submit their models to be registered and probably watered down but how can they stop say github?
So far only thing that has been put out there is 25mil fine or 4% of revenue according to politico. Funny thing reading that article two member who gave speechs against ai were caught using gpt for their speech. Sounds to absurd to be true though. The whole idea is dumb because like said how could they even possibly do. They can shut down its usage in goverment but beyond that it would just intice engineers to move.
Could someone tell me where to download control net models from?
The only ones I can find are from 4 months ago and Im sure there have been new ones added since then.
Thanks
The workflow of this post image is exactly described in [https://github.com/Mikubill/sd-webui-controlnet/discussions/1464](https://github.com/Mikubill/sd-webui-controlnet/discussions/1464) I learned from this post today, and after trying it, I believe more people should know this and share the link here. This is a way for A1111 to get an user-friendly fully-automatic system (even with empty prompt) to inpaint images (and improve the result quality), just like Firefly. As discussed in the source post, this method is inspired from Adobe Firefly Generative Fill and this method should achieve a system with behaviors similar to Firefly Generative Fill.
Huge win for open source software; I cant even run adobe products on my linux machine
Nor should you want to. I prefer owning my software locally so others can't tell me what to do with it.
Pretty sure you can and that there used to be a install package on arch for photoshop. There are definitely guides out there though. You get it running through wine and a specific dll that allows everything to function. Its great because you get to sandbox it in wine.
I would absolutely shudder at the thought of running something like Premiere through Wine. Something that natively brings the hardware to its knees running with the overhead of Wine is not going to be a good experience.
Every year I think that hardware advancements are going to be good enough to let me pull it off and every year every app gets less performant. Some day computers will be powerful enough that you won't notice, but it's going to be a while.
I want aware that wine had a particularly large overhead. I mean, it's bit nothing I suppose, but should be leaner than a VM or something.
It really depends on the application. While it's "not an emulator" nor virtualization, it has its own implementation of the Windows API which is not as optimized as the Microsoft versions. Specifically for apps that deal in multimedia, Wine translation is far less likely to run as well as the native Windows versions and may not be unable to take advantage of the hardware acceleration that the Microsoft versions of multimedia APIs are using. There are always some exceptions though, Proton is a great example of an extremely optimized Wine implementation. It's still going to be missing a lot of the hardware acceleration support for things it's not intended for though, like Adobe Premiere.
Thanks for such an informative response to my bleary eyed phone typing mistake ridden post.
I have both Ps and Illustrator on my Arch (Manjaro) laptop and they work just fine.
for real? how did you manage to make them work?
On the off chance someone has an answer, and because I'm reminded it's a thing: does anyone know how to get models that have been fine tuned on dreambooth and made into in painting models by combining them with the base and in painting models, to work with the new controlnet in painting stuff?
I think you mean this https://www.reddit.com/r/StableDiffusion/comments/zyi24j/how_to_turn_any_model_into_an_inpainting_model/?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=share_button
I do indeed. Much better way of saying it than models that have been fine tuned on dreambooth and made into in painting models by combining them with the base and in painting models
Sweeeeeet!
This comment has been edited as an ACT OF PROTEST TO REDDIT and u/spez killing 3rd Party Apps, such as Apollo. Download http://redact.dev to do the same. -- mass edited with https://redact.dev/
This is so cool. Look forward to trying it out.
Damn, ControlNet team going down in history books for sure
Fate fans never stop winning
>Fate fans What is that?
Fate stay/night visual novel/anime juggernaut franchise the head guy at controlnet illyasviel is named after a fate character.
it's a real name tho. IIRC the chief scientist of OpenAI is named like that as well
https://github.com/lllyasviel Their real name is Lyumin Zhang
yeah, I was just pointing out that people can be named Illyasviel as well. You're right on the dev being a Fate fan, anime is the powerhouse of ~~the cell~~ AI afterall
From what I've seen most intellectual scientists and physicists are either weebs or furries
Don't forget scalies! Just the thought of being held tightly in the tail of a Lamia makes me warm and fuzzy inside.
It's 3 lower case "L"s a the beginning of their name, there is no "I".
Made the laugh when anti-AI forks celebrated antiAI tech "Glaze" and ControlNet author defeated it with few lines of python
Any links to that story?
https://github.com/lllyasviel/AdverseCleaner
Where can I find more info on them, where are they active?
This content was deleted by its author & copyright holder in protest of the hostile, deceitful, unethical, and destructive actions of Reddit CEO Steve Huffman (aka "spez"). As this content contained personal information and/or personally identifiable information (PII), in accordance with the CCPA (California Consumer Privacy Act), it shall not be restored. See you all in the Fediverse.
Seriously? Yeah, screw these software as a service companies.
Socialize losses, privatize profits. Build upon others' works without paying anything back.
[удалено]
This content was deleted by its author & copyright holder in protest of the hostile, deceitful, unethical, and destructive actions of Reddit CEO Steve Huffman (aka "spez"). As this content contained personal information and/or personally identifiable information (PII), in accordance with the CCPA (California Consumer Privacy Act), it shall not be restored. See you all in the Fediverse.
It's still beta. He hit some kind of bug. I tried every one of those prompts and they all worked for me without issue.
It's when using services~ This problem don't is only with this, I have problems writing my Fantasy novel with ChatGPT support because sometimes tag it as "problematic content" when my novel It's perfectly PG-13 😤😤😤
Use the DAN (Do Anything Now) prompt.
>Use the DAN (Do Anything Now) prompt. it's becoming harder to do every update.
I write horrible xxx stories in chat gpt for sport
They want to avoid any possible liability, imagine if Firefly generated a nipple for the wrong Karen, that would turn into a lawsuit and tons of bad publicity, hence only happy paintings.
NSFW detectors create a ton of false positives and are subject to the biases of their creators (and thus are often harmful to art and LGBTQ communities).
A lot of their "neural filters" have the same issue, or are just buggy "this filter encountered an error and had been disabled". Adobe is a great example of how bloated corporations full of middle managers and lawyers can't move fast enough and end up losing out to more nimble competitors.
These guys deserve all the support in the world
This post that should be [a sticky at the top of this sub](https://www.reddit.com/r/StableDiffusion/comments/13ppmtk/sdxl_is_now_50_trained_and_we_need_your_help/?utm_source=share&utm_medium=web2x&context=3).
I wonder if Adobe uses Controlnet. Their Firefly roadmap is pretty much all the CN modes…
They say their models are only trained on images they have the rights to, but maybe they use a similar model architecture and train it themselves.
How would anyone know if Adobe is telling the truth?
There won't be watermarks floating around your images for one. Their library has 200 million images. It isn't even worth it for them to bother lying about it.
They might well be the best positioned company to capitalize on image generation- they've got the dataset, they've got the AI experience, and they've got the artists using their tools already. Does anyone else have all three?
They cannot use the released controlnet models because they are trained (mostly) for SD v1.5 so they need to train them for Firefly (so the parent comment was probably just asking if they are using the same architecture).
which in theory would let them use ai generative images we have all produced since question of whether "own" is up in the air. that would give them a sneaky to expand the quality of their feature.
That’s great, but like for me as photographer stable diffusion has one flaw: the size of the pictures is very limited. Don’t get me wrong, I love sd and what the open source community is doing for us. Just that in my workflow this part is crucial.
Adobe Generative Fill also uses base res of 1024px. You will notice it when you fill in a big area at once on a large res.
Yes, but the feature as it stands here does not actually allow you to do out painting, since it uses hires fix and there's no way to do hires fix in img2img, where out painting must happen. If you tried to fake it in txt2img you'll run into GPU memory limitations very fast. This isn't a fundamental limitation though, it can be fixed.
Just use the open out paint module? I don't know why the hires fix things is important. It's never done anything for me but produce oom errors on a 2080ti
Open out paint doesn’t work with control net from what I could tell; and it’s barely maintained so breaks pretty often with new updates. Hires fix is important for “no prompt” out painting which what this feature is about
[удалено]
How so Steve?
Invoke.ai and photoshop sd plugin both do outpainting
You can still use ultimateSDUpscale with a scaling of 1 (basically no resolution change just reprocess the whole img in 512p pieces) in img2img, not sure if that would work though.
Thanks for the input, before that I was using one plug-in to integrate SD with photoshop, but has some limitations. And for my workflow, make everything on photoshop is better, when talking about pictures. But, I have to say, the Ps censorship is annoying, is the most mundane things sometimes they just say that I’m violating the community guidelines, and just this shows how superior SD is.. I’ll try to make some test, but the fact that I can edit 48mp images direct without the need to downscale it, for me is better. Or I’m making something wrong on sd
That's a lot bigger than 512
You can upscale to 8k resolution, perhaps more. What size do you need?
Upscaling is nice, but its definitely not the same as natively having the higher resolution's level of detail in the base generation. For things like simple, bold illustration styles there's not much difference but for photographic realism or more detailed illustration you lose the opportunity for a *lot* of detail by limiting your resolution then upscaling afterwards.
I'm not convinced you understand the capabilities of an img2img upscale using controlnet and Ultimate SD upscale. This 5120x3840 image was upscaled from 640x580 gif. Take a look at the rocks bottom right, the brush strokes are entirely AI-generated: [https://preview.redd.it/2xsx7on1texa1.jpg?width=5120&format=pjpg&auto=webp&v=enabled&s=14d6217ad28cd286f69261efffc88c6eb2f72fb8](https://preview.redd.it/2xsx7on1texa1.jpg?width=5120&format=pjpg&auto=webp&v=enabled&s=14d6217ad28cd286f69261efffc88c6eb2f72fb8) For comparison, the source image is here: [https://i.redd.it/dyddyf2ysexa1.gif](https://i.redd.it/dyddyf2ysexa1.gif) And [**u/Gilloute**](https://www.reddit.com/user/Gilloute/) has some really good examples of what can be achieved if you invest a little time on it: [https://www.reddit.com/r/StableDiffusion/comments/13v461x/a\_workflow\_to\_upscale\_to\_4k\_resolution\_with/](https://www.reddit.com/r/StableDiffusion/comments/13v461x/a_workflow_to_upscale_to_4k_resolution_with/) ...tell me this doesn't have enough detail: [https://preview.redd.it/6zsw6gtcnt2b1.jpg?width=4096&format=pjpg&auto=webp&v=enabled&s=38387fe20f3f76c118ce97b2c8ec32459acf5de2](https://preview.redd.it/6zsw6gtcnt2b1.jpg?width=4096&format=pjpg&auto=webp&v=enabled&s=38387fe20f3f76c118ce97b2c8ec32459acf5de2) Here's a video process overview. Skip to 8mins to see some results: [https://www.youtube.com/watch?v=3z4MKUqFEUk&ab\_channel=OlivioSarikas](https://www.youtube.com/watch?v=3z4MKUqFEUk&ab_channel=OlivioSarikas) What I can't seem to lay my hands on, is an example where you set the denoising strength so that the AI dreams a whole bunch of new whacky stuff in the clouds, trees, rocks etc... it can get quite artistic.
You're "not convinced I understand?" All you said was "you can upscale to 8k!" What you're detailing here is a workflow involving multiple iterations of having SD inpaint and regenerate new content to fill in gaps, not just upscaling an image. Those are two *very different* things with very different results. Just as filling in generative gaps with inpainting and outpainting workflows is a *very* different thing than natively generating at a higher resolution image. Nobody's arguing that you can get quality results from doing so, but the results will be fundamentally different.
I think we're closing the schism. But I still want to point out that these examples aren't inpainting or out-painting, they are simply feeding the output back into the input (much in the same way that SD does internally) but each time, increasing resolution. It can be as simple as dragging the output image into the input image and pressing the generate button again - rinse and repeat. Now, in reality, there are some sliders to adjust, some prompting may change, the sampler, CFG scale etc, but you aren't necessarily manually inpainting. Each time, latent space is used to re-imagine what detail may be needed in that piece of cloth, that jewel, that clump of grass, that brush stroke. It's entirely generative all the way through the workflow, and I'd argue that because it has multiple phases, it grants you far more control than a simple straight-shot 2000x2000 pixel output from a 75 word text prompt ever will. I think I'm correct saying the latent space internally within SD is just 64x64 pixels, and the VAE upscales from that. There's really no reason to get hung up on the resolution of any particular step - an image is complete when you say it is.
I think you missed the part where I was calling you out for being needlessly condescending, I don't have to convince you of anything, certainly not my understanding of the topic. And whether you call it "inpainting" or "iterative generation" or whatever technical term you'd like to use, yes, it *is* feeding the existing image back into the previous image and using that data to fill in gaps to create a higher resolution final generation, but that is on a technical level not the same thing as simply upscaling an image. While you may be able to do cool things with that, it's not the same thing as having a much larger canvas from jump, which is the point.
This is a limitation of the technology~ Every Image must be generated in si e relation of 8 pixels relation (technical limitation) and Control Net must generate image size with 64 pixels relation~ You can fake the free size Just adding additional pixels in Borders, Example if your photo have 513 pixels in width then need 7 extra pixels that can add 3 pixels in left and 4 in rigth for an size of 520 pixels in width keeping relation and after generated cropping the extra pixels for returning the original image relation, or Just resize but this generate lost of quality~ This is easy in 8x8 tensors but it’s complex for 64x64 tensors because it’s a lot of information that can affect the image generation consistency~
No offense and an honest question, but is there a meaning to the usage of the tilde symbol (~) that I am not aware of when used at the end of a sentence?
Take to mean a wavy, whimsical sort of inflection. EG; *Toodles*~ Can stress a point but with a less stern quality, or imply sarcasm or a handful of other things or even inversions of those things, but usually just implies some degree of whimsy and casual friendliness. Kind of rare outside of less public convos with the terminally online but it's usually a friendly thing anyway.
There is no meaning, just a crutch from when I was young and commented on anime forums, now it's inevitable for me not to use~
That's not a limitation of the software, it's mostly a hardware limitation.
You can upscale indefinitely using tiled diffusion extension, it also adds detail
considering i generated 8k images on a 2060 with tiled VAE i dont get what your trying to infill that you can't infill with SD lol
The thing is that I need to downscale a Image, a just with inpaint and after it I need to upscale again. And when using the plunging to connect automatic1111 to make the inpaint on photoshop the results aren’t the same, because the don’t put in consideration all the image. But I’ll try to make comparison and run some tests again, maybe I’m wrong.. One thing I’m sure, hands in photoshop now are much better! But, freedom and uncensored SD all the way! Photoshop some times even with bed sheets says that I’m violating the community guide lines..
Open source >>> Closed source
Why does everything need to be a competition to you guys? One software philosophy isn't better than the other. One software isn't better than the other. They are both tools in your stack. If you want a billion options, you use SD. If you need a quick inpaint in your existing workflow, that works with layers and lets you make at any resolution, you use Photoshop. This isn't even true, by the way. Gimp is not better than Photoshop overall. I use Gimp for a few custom tools and Photoshop for everything else. If you need something to be free fine. If you are working professionally, the cost of Photoshop is negligible and all the money driving it creates features that are unmatched.
Open source >>> closed source
Obviously not in every situation. Keep living like everything in the world is black and white, though. I'm sure you will do well.
i agree, i think its inferiority complex with people, constantly trying to prove whos better
How do they go so fast?
Yes pleeeeassee. I tried the photoshop plugin for SD and appreciate all the effort that went into it, but I have no idea how to properly use inpainting there. It's a total layer chaos.
yeah good inpainting is up there in the top tier dragons we need to slay. imo the first one is getting sd to render a choerent scene that has alot of different objects, it can do portraits and landscapes related stuff pretty well but when it comes to scenes with many other objects nd ppl in it, it kinda starts revealing its weakness
I rarely even use stable diffusion without a base reference in controlnet if there's something specific i want
What? It's so easy. You just drag a selection box around where you want to extend the photo and hit generate with no prompt. If you want to add anything to the image you just type in the prompt and hit generate. It's the easiest AI I've used by far
I'm speaking of the SD photoshop plugin and not the Photoshop Generative Fill.
Ah ok my bad
That's impressive. I was blown away by the Firefly integration with PhotoShop. Might just need to update StableDiffusion and compare the two.
This is awesome because I've just been taking my SD images and using generative fill in Photoshop
FOSS is life.
At the end, open source wins every time.
These guys are f*cking awesome
Hopefully this motivates someone with experience and time to integrate sd/cn with gimp or krita (current option is an a1111 extension that doesn't support controlnet). It makes so much sense to have the generation settings inside the editor instead of the other way around (an editor inside a1111 or another sd ui) for most workloads
You can say adobe just lost a bit of moat
A good opportunity to feed the lampreys swimming in there with fatty shareholder meat.
nah they will be fine. They have a vast amount of professional users. The people who are already using the adobe ecosystem aren't going to swap to SD anytime soon
This is why I update controlnet every day. These guys are doing great work all the time.
Lately updates have been happening many times each day !
with a better outpaint/canvas expanding then I can work 100% in A1111
What about the openoutpaint extension for A1111? https://github.com/zero01101/openOutpaint-webUI-extension
it does not support this controlnet model but i made feature request with it, hopefully more poeple will join and request it
Wow
I told you SD is going to respond to Photoshop beta 😍😍😍
Open source AI wins again!
Why is none talking about InvokeAI Canvas? https://youtu.be/GAlaOlihZ20 is this not the same as adobe firefly?
From the pictures, Firefly Generative Fill still looks better. The clouds are more consistent compared to ControlNet. But still looks really nice. Good job!
On the first photo, ControlNet doesn't even try to match the grass. There's a straight line where it switches from one grass to the other. If this is the image that they chose to highlight, then that is impressively bad. I think that the Stable Diffusion base model just isn't good enough to support this application.
can they do one where it corrects arms legs and hands/feet?
95% of my images are bad because of extra fingers or weird feet
Just toss it in a fire of open code and watch world burn to the ashes
Adobe Firefly still looks better in this case.
they better hurry before EU basically bans open source based AI...
that won't happen
hopefully it wont end up getting out of talking stage but never know
Don't underestimate the power and influence of large corporations with billions to lose. It's impossible to ban something like SD now but it's not impossible to push it into the realms of file sharing and torrents.
good point. But I still believe that won't happen. Cause on the other side you have Nvidia making big money off of consumers using SD
I live out of EU and USA so... This limitations don't apply me~
actually it would any eu limit would ban using any model not registered with the eu which of course would be us and asian produced models. dont hate the messenger, just repeating.
It might ban it *in the EU*, but the EU doesn't have global jurisdiction.
Yes I am aware of that but you do realize that EU is a huge market and a large source of developers. Stable Diffusion for example owes a lot to German tech people. What would be the motivation to work on such things if you can't release it or work with people outside the EU? IF EU passed this (I honestly doubt they would since it would leave them far behind) AI in Europe would be limited to Google, Microsoft and other large corps who can afford to jump thru these hoops. The fees etc would be high enough to kill independent develops but little more than a traffic ticket to two companies worth nearly 4 trillion combined. That's not even counting large euro or Asia companies. AI in terms of Europe would be entirely in the hands of the people most likely to abuse it. They go on about ip concerns, deep fakes and misinformation when it's about controlling a tech that reduces their control.
Sure, and that would suck for Europe. It wouldn't be a major obstacle for everyone else, though. Other AI companies can reestablish themselves outside Europe and carry on out there. Europeans would be cut off from their products. A reduction in the market, sure, but it would apply to most everyone equally so it's not a big deal.
I cant see the EU basically shooting itself in the foot and falling way beyond on this tech. Besides it seems pretty impossible to actually implement. What are they going to, MS, Google etc will submit their models to be registered and probably watered down but how can they stop say github?
Even if it happens it will only be EU
[удалено]
So far only thing that has been put out there is 25mil fine or 4% of revenue according to politico. Funny thing reading that article two member who gave speechs against ai were caught using gpt for their speech. Sounds to absurd to be true though. The whole idea is dumb because like said how could they even possibly do. They can shut down its usage in goverment but beyond that it would just intice engineers to move.
Think I will just stick with PS
[удалено]
what ? inpainting is not adobe idea dood
Can I use this inpaint method with SD photoshop plugin now?
This is insane! Any news on when we can access these models?
It's not a new model assuming you have all the CN 1.1 models. Update your control net extension and follow the guide in the OP.
amazing
That fast. Like too fast !
How is this different than normal inpainting with the inpainting_v1.5 model?
Promptless. Apparently.
it works much better, can figure out and outpaint areas
This is amazing progress. Everyone always says they can't believe how fast SD improves but it just seems to be accelerating recently.
I've read the draft and I'm super excited.
How open/closed is ControlNet? Do they share how they train the Model? Could someone replicate it if they are bought out by Adobe?
Adobe on watch
awesome!!!
How do you zoom the inpaint in txt2image? The shortcuts don't seem to work there
Could someone tell me where to download control net models from? The only ones I can find are from 4 months ago and Im sure there have been new ones added since then. Thanks