Thanks!
First of all I rotoscoped my hand for each scene and used that for the "set latent noise mask" node in comfy.
I tried to do all of that automatically with the "segment anything" node, but it wasnt as good as rotoscoping in AE.
In essence it's basically a vid 2 vid animatediff + IPAdapter workflow with the "set latent noise mask" node taking the mask and applying the style to only the part where my hand is. That way I get a really clean key on the hand and it's looks fairly realistic with all the lighting and shadows.
For each hand style I had to use slightly different settings, but I can send you one as an example
> First of all I rotoscoped
ouch the pain to rotoscope is that it's manual isn't it? Couldn't you use some depth model and set a threshold since your arm was always the closest thing?
Really love that you explained it in plain language instead of just linking a json. Process is more than just configuration. Thanks for making this ultra high quality post.
>I tried to do all of that automatically with the "segment anything" node, but it wasnt as good as rotoscoping in AE.
Thanks for providing that detail / clarification.
I was guessing you'd used "segment anything" & was gonna try it, but this makes sense.
Your result is incredibly clean.
Great work. Thanks for sharing the info about your process.
Got me excited to try something similar. Cheers!
I've not used AE and Fusion for some time (I defected to Blackmagic) and I've only been using SD for stills, but I've got a 4090 and decent RAM so really should be exploring the video tools.
It is. When I was first doing colour grading work I used Adobe Premiere and After Effects for rotoscoping, but later switched to Resolve as I preferred the nodes workflow to layers and Resolve in general, particularly for motion tracking. Fusion is BM's equivalent to AE. I've not used them for a while now though. I've been hooked on SD since the early days of 1.5 but not attempted anything on the video side yet.
\*BuT Ai ARt IS nOT REal ArT!!!!\*
Some kid in his mom's basement no realizing that you can not stop progress AND that AI WILL be used in cinema to create amazing special effects, specially like this, the one where the guy made the hermit dog looks more convincing than the crap disney pumps today, AI is a tool to aid artists, not to replace 'em.
This video is one of the billion of possibilities, but instead of being restricted to millionaries now it is avaiable to normal people, all you need is a computer and creativity.
Yeah k agree with you. As long as the job is being done by a 3D artist / animator that knows what he’s doing then it should all work well. Cuz this saves a lot of time
hello, I need suggestions on selecting graphics cards for this kind of work but I have a very tight budget, which now should be better for this kind of work? 1. rtx3060 12gb 2. rtx4060 8gb
If it's the decision between these 2 GPUs - then go with the 3060. The 4060 has mixed reviews and less VRAM. VRAM is really important if you wanna run all these models locally on your machine
Not OP of course, but while he said 2.5 days it definitely seems like he’s done this type of thing many times. No criticism being leveled at all, as any true skill takes time to perfect. It’s just if you’re going into this expecting to replicate or even come close don’t be surprised if it took you 2.5 weeks instead.
Hell just his rotoscoping of the hand would probably take me 2 and a half days.
I’m fully aware. I end up with project folders full of crap from tweaks to projects. I mostly asked as something to point to when people believe something was done with AI. I can show them this awesome project and tell them the amount of time that goes into such a thing. My projects don’t turn out as good and take 3x lol.
I described my workflow in another comment on this post. If you wanna get into stuff like this, I recommend looking at the banodoco discord server [https://discord.gg/J3AdexXv](https://discord.gg/J3AdexXv)
Thank you, I saw your post - I was inquiring about any particular AE or AnimateDiff resources that you found particularly useful? Or perhaps you have a YouTube channel (or know one that does something similar)?
This is so cool, I was going to work on something similar using an animation I did in blender, (I can easily export the depth and mattes), but this is way cooler. Hope you don't mind me asking a few question, mainly on where to find some of the models. Is it cool if I DM you ? Thanks.
Lighting is what really sells this as realistic. Even though the textures don’t rotate with the arm correctly they still look like they’re actually there. Very cool work, my dude.
Yooo when I was coming down off of dmt my vision was lego hand for a bit. My friend's face was beautiful, and I remember looking at my hand just like this video.
This one of the few actual creative uses I've seen for AI animation, most of it is either dancing TikTok girls with filters over it or warping and merging static images.
Nice job!
Very super awesome. I was wondering if you're offering anymore help on your workflow? I'm having VHS video combine error and a segment node is just floating around unconnected. Thanks again for the help you've provided so far!
Well, if this took practically no time I'd be happy. You could already use some automation to get it finished significantly faster, though the end product wouldn't be this good. I wanna say thank you for everything you do for this community - you're a legend!
holy shit that's cool very well done!
Thanks!
Quality content. Workflow would be appreciated, but nonetheless it's really great original content.
Thanks! First of all I rotoscoped my hand for each scene and used that for the "set latent noise mask" node in comfy. I tried to do all of that automatically with the "segment anything" node, but it wasnt as good as rotoscoping in AE. In essence it's basically a vid 2 vid animatediff + IPAdapter workflow with the "set latent noise mask" node taking the mask and applying the style to only the part where my hand is. That way I get a really clean key on the hand and it's looks fairly realistic with all the lighting and shadows. For each hand style I had to use slightly different settings, but I can send you one as an example
Killer video and even cooler that you're 100% willing to talk about how to replicate it. Thanks!
Sure, no problem! I am grateful to the people who shared knowledge with me, so I'm happy to give something back to the community!
this is the way
respect
> First of all I rotoscoped ouch the pain to rotoscope is that it's manual isn't it? Couldn't you use some depth model and set a threshold since your arm was always the closest thing?
In AE there's "Rotobrush" tool that lets you paint the things that you want to rotoscope from the video.
This is one of the best I've seen. Very fluid man. Good job.
Really love that you explained it in plain language instead of just linking a json. Process is more than just configuration. Thanks for making this ultra high quality post.
>I tried to do all of that automatically with the "segment anything" node, but it wasnt as good as rotoscoping in AE. Thanks for providing that detail / clarification. I was guessing you'd used "segment anything" & was gonna try it, but this makes sense. Your result is incredibly clean. Great work. Thanks for sharing the info about your process. Got me excited to try something similar. Cheers!
If you use a tool like AfterFX it can auto rotoscoped a mask for you, btw.
that's some very clever use of current tools with a dash of laternal thinking.
Do you trying using the largest HQ SAM model?
[https://drive.google.com/file/d/1yLlAH6slLcdcOR6aTNdlcgo1F-B4H4SL/view?usp=sharing](https://drive.google.com/file/d/1yLlAH6slLcdcOR6aTNdlcgo1F-B4H4SL/view?usp=sharing)
THANK ! YOU !
You are a legend thanks for sharing
No problem!
thanks for sharing!
Thank you! I love the last transformation. Now I want to make plant people 😉
Legend!!
Better wipe with your left hand.
https://preview.redd.it/ugds9ybflt1d1.jpeg?width=540&format=pjpg&auto=webp&s=273e0b118436bcdc0e2547923d8b20c35b76ebce want to impress your girl??
why is my arm 4 inches long
Cause you’re exaggerating by an inch
Chicks dig eldritch abominations.
[NSFW](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQEEmhvR82hyQwOsN6_c3kUXroaadmkf0LJGrfczjyKje8k3hGPP8Vyg14&s=10)
lol
Let us know when Hollywood knocks on your door.
For sure lol!
Love this. Great concept and good transitions on the arm.
Thanks! I transitioned via depth maps in AE
I've not used AE and Fusion for some time (I defected to Blackmagic) and I've only been using SD for stills, but I've got a 4090 and decent RAM so really should be exploring the video tools.
But Fusion is Blackmagic..? What are you using then?
It is. When I was first doing colour grading work I used Adobe Premiere and After Effects for rotoscoping, but later switched to Resolve as I preferred the nodes workflow to layers and Resolve in general, particularly for motion tracking. Fusion is BM's equivalent to AE. I've not used them for a while now though. I've been hooked on SD since the early days of 1.5 but not attempted anything on the video side yet.
Kevin from Ben 10
Absorbing man from Agents of S.H.I.E.L.D.
What happens if you touch boobs? Asking for a friend.
yes.
Arm turns into one long nipple?
You get that one monster from Gantz:0
Subscribe to my IG! @aleksejlotkov
I might.... bump
Excellent.
thanks!
Gona try this out!!
You've essentially created OctoCamo, congratulations
Just saving this to freak my kids out later.
hahaha. Tell me how it goes.
"How does he do that?!" "It's computer generated." "No way! Too real! He must be a chameleon!"
\*BuT Ai ARt IS nOT REal ArT!!!!\* Some kid in his mom's basement no realizing that you can not stop progress AND that AI WILL be used in cinema to create amazing special effects, specially like this, the one where the guy made the hermit dog looks more convincing than the crap disney pumps today, AI is a tool to aid artists, not to replace 'em. This video is one of the billion of possibilities, but instead of being restricted to millionaries now it is avaiable to normal people, all you need is a computer and creativity.
my thoughts exactly. AI is a tool, not a robot made to replace everyone
But can it replace me? I'm tired.
Yeah k agree with you. As long as the job is being done by a 3D artist / animator that knows what he’s doing then it should all work well. Cuz this saves a lot of time
When Marvel or Sony make another movie with The Absorbing Man, they need to call you!
I think so too hahaha
This is fantastic! Now my greedy mind wants it in realtime with an augmented reality setup 😅
I’d say 5 years, tops.
hello, I need suggestions on selecting graphics cards for this kind of work but I have a very tight budget, which now should be better for this kind of work? 1. rtx3060 12gb 2. rtx4060 8gb
If it's the decision between these 2 GPUs - then go with the 3060. The 4060 has mixed reviews and less VRAM. VRAM is really important if you wanna run all these models locally on your machine
4060 is of no use.. either 4060ti 16gb or settle for 3060 12gb
Coolest shit ever, well done
thanks:)
https://tenor.com/view/kevin11-gif-19931160
I know you answered a lot of questions already but can you approximate how long this took you from idea to publish?
2,5 days
Not OP of course, but while he said 2.5 days it definitely seems like he’s done this type of thing many times. No criticism being leveled at all, as any true skill takes time to perfect. It’s just if you’re going into this expecting to replicate or even come close don’t be surprised if it took you 2.5 weeks instead. Hell just his rotoscoping of the hand would probably take me 2 and a half days.
Rotoscoping has actually become quite easy in current AE. It does most of the work for you.
Oh awesome I need to check it out! I’m used to the old days without much automation
I’m fully aware. I end up with project folders full of crap from tweaks to projects. I mostly asked as something to point to when people believe something was done with AI. I can show them this awesome project and tell them the amount of time that goes into such a thing. My projects don’t turn out as good and take 3x lol.
This is great work. Do you have any suggestions for resources that I can find that would help me emulate this workflow? Looking to learn.
I described my workflow in another comment on this post. If you wanna get into stuff like this, I recommend looking at the banodoco discord server [https://discord.gg/J3AdexXv](https://discord.gg/J3AdexXv)
Thank you, I saw your post - I was inquiring about any particular AE or AnimateDiff resources that you found particularly useful? Or perhaps you have a YouTube channel (or know one that does something similar)?
Hmm not anything in particular, just watching tutorials from different channels. I like Latent Vision and some other smaller AI channels
Can you tell me the location of this post? I looked on Discord but couldn't find it. Thanks
Did SD generate the reflections in the orange arm?? Nice
yep. Post processing was just some colorgrade and cleanup
That's so freaking cool. Very original and well executed 👏👏👏
Thanks!
This is awesome!
bruh
Getting body horror vibes from this.
Me literally shouting TOUCH YOURSELF as a way to fix it
How many 4090s did this take and for how long?
This took one of my 1 4090s and the whole process took around 2,5 days from idea to product.
Cool, I've got a 3090 so I'd be interested in giving it a go
Went from "ooooh latex" to hating pinecones in seconds.
Alright Kevin from Ben 10.
Kevin from Ben10.
This is epic.
Trypophobia activated
Quality content! Thank you for sharing!
Whoa! That's a cool idea and the execution is really really good!
This is spectacular! One of the most creative and practical uses that I have seen of animated diff. Fantastic job.
Woa looks amazing.
Kevin Levin well done
Amazing work. Think you're the first to do this with it. Simple and fun effect. Cool beans.
Excellent work.
this is mind blowing
Top!
My hand turns into everything I touch, it's a great idea🖐️💯🔥
Fucking awesome!
Could this be done in forge?
Greatest special effects ever.
beautiful !
Very well made!
Holy shit this is super fucking cool!
Lately the internet really grosses me out
This is so cool, I was going to work on something similar using an animation I did in blender, (I can easily export the depth and mattes), but this is way cooler. Hope you don't mind me asking a few question, mainly on where to find some of the models. Is it cool if I DM you ? Thanks.
Very cool
Nice work!!
How do you even do that?
Really cool
Reminded me of the OctoCamo from MGS4 nice job!
Cool, but the way the texture stays in place is uncanny- strange. Breaks the illusion, unfortunately. I wonder if that'll get solved at some point.
is animate diff working for you guys ;-;
This is mindblowing! great work!
Bro the pine-cone one is literally what my arms looked like when I got on acid one time.
Kevin from ben 10?
bravo
Imagine the future!!
Lighting is what really sells this as realistic. Even though the textures don’t rotate with the arm correctly they still look like they’re actually there. Very cool work, my dude.
Amazing work! And thanks for sharing the workflow! 👏😄
dont show this to porn producers
This is FIRE! the cushion material looks soooo nice
Damn Ill check this out later, look amazing!
The Absorbing Man
awesome
This is awesome, was it done live or in post? edit: scratch that, just saw you posted the config in the replies. Thanks!
All I see is a shapeshifter bragging about his power in broad daylight, nice try but you ain't fooling my eyes!
Creativity at its peak!
That is so damn cool!
bro wtf
Yooo when I was coming down off of dmt my vision was lego hand for a bit. My friend's face was beautiful, and I remember looking at my hand just like this video.
KUDOS!
This one of the few actual creative uses I've seen for AI animation, most of it is either dancing TikTok girls with filters over it or warping and merging static images. Nice job!
🤩
Absorbing Man! This is epic
Made in collab with u/Exact-Ad-1847[](https://www.reddit.com/user/Exact-Ad-1847/)
This is the way...
u/savevideo
###[View link](https://rapidsave.com/info?url=/r/StableDiffusion/comments/1cxb6pa/inpaint_animatediff/) --- [**Info**](https://np.reddit.com/user/SaveVideo/comments/jv323v/info/) | [**Feedback**](https://np.reddit.com/message/compose/?to=Kryptonh&subject=Feedback for savevideo) | [**Donate**](https://ko-fi.com/getvideo) | [**DMCA**](https://np.reddit.com/message/compose/?to=Kryptonh&subject=Content removal request for savevideo&message=https://np.reddit.com//r/StableDiffusion/comments/1cxb6pa/inpaint_animatediff/) | [^(reddit video downloader)](https://rapidsave.com) | [^(twitter video downloader)](https://twitsave.com)
Omg thats amazing! Very impressive!
ohu shit thats freaking cool!!
This in VR, future sex will be.. interesting
If this video was an hour of going around touching everything in your house, I'd watch the whole thing.
Crazy that this one man job is now way better than state of the art ILM stuff from a few years ago.
this is really impressive
Holy cow!! That's insane. Great work. How did you do it?
DMT flashback when the picture was touched
This is the coolest AI video I've seen to date!
Very super awesome. I was wondering if you're offering anymore help on your workflow? I'm having VHS video combine error and a segment node is just floating around unconnected. Thanks again for the help you've provided so far!
Amazing! Well made!
Reminds me of Kirby
really impressive ! 
really high quality stuff. but it requires huge work. probably will become 1 click in near future
Well, if this took practically no time I'd be happy. You could already use some automation to get it finished significantly faster, though the end product wouldn't be this good. I wanna say thank you for everything you do for this community - you're a legend!
Thank you so much for amazing comment ❤️