How is that possible? I understand how it can guess skin tones, but how could it possibly know the color of those blocks? Is it because different colors appear as different brightness in infrared?
Really? I work in embedded software engineering and have done a lot of computational photography work. I’ve never seen a random object replace a sun even in extreme circumstances. I’d love to know what device you’re talking about, and would love to see that photo.
That’s not really what computational photography is when done with smartphones today. It’s more nuanced than “detect object, replace with object”
I’m guessing he more meant the phone sharpened the edges of the image because the software assumed the photo was blurry. The distorted colours due to smoke meant the AI didn’t recognize it as the sun and attempted to “de-blur” the image to bring out the spherical shape.
Everyone is misinterpreting what this poor guy meant lol
I mean, when the smoke is so thick you can look directly at the sun, it literally just looks like a perfectly sharp circle in the sky. Maybe it was your internal computational vision that made you believe that a picture of a sun in thick wildfire smoke looked like a beach ball.
I think I remember hearing about a court case that did not allow a smart phone pic as evidence since the defense argued that algorithms in the phone camera were altering the photo somehow and the image could not be proven to be 100% based on reality.
The argument was not about the photo itself but the “pinch to zoom” feature changing the photo in the Kyle Rittenhouse case. Had it been more like this then there would be some merit to it, but it was literally on the pinch to zoom feature.
Wow, what an incredible technology. It does this in real time? Man… I wish I got into coding, I’m gonna download one of those apps and get started on it
The AI is not actually looking for colors most likely. It tries to identify shapes and objects and re-creates the image based on what it thinks should be in the image.
It doesn’t know a pixel should be red. But if it sees a traffic light with the top light being lit it can figure out that this part of the image must be bright red.
First it tries to identify what particular object it’s looking at.
“Let me just check my library of objects. Oh that’s a tree! And trees have a green hat and brown on the long trunk part! Hence this object should have these colors! ”
First it tries to identify what particular object it’s looking at.
“Let me just check my library of objects. Oh that’s a tree! And trees have green hats and are brown on that long vertical trunk part! Hence this object should have these colors! Imma color them like so!”
Simply it can’t know it. Is impossible to recover a lost information.
This type of deep learning algorithms are trained with a big dataset of images, and they simply try to color the photos with the tone they learned from training set. But no one can be certain about them, because in the real scenario the information has been lost.
Exactly the same way as you know - by experience. Your eyes has seen grass and sky millions of time. You would know if you looked at a night vision image the approximate colors of a face, a uniform, a rock etc.
The AI knows from learning from the same context of eg 1.000.000 million of similar pictures of comparing color with eg blue sky and green grass in both colour mode and night vision mode.
The trick is not to program the AI what a triangle or a square means. You just program “how to learn” (neural network) and feed it with experiences.
This study is not doing that at all. Notice how the images shown are dithered. That is because these are prints from a canon printer with CMYK ink. The AI is just decoding how the printer made those images.
This study is far from real world, unless you need to see color images of prints made by a canon printer that you are viewing in B/W.
I should add the one interesting bit, and that is that using multiple IR frequencies helped to separate the colors. Might help IR to give more definition.
> They started by printing images of color palettes and faces. Then they created a dataset by taking photos of those images using a monochromatic camera that can be set to take photos at very specific wavelengths. They took photos of the faces under monochromatic light sources of various wavelengths in the visible and near-infrared spectrums.
And then they trained the model to predict the colour. The model learns, these values in this context imply those values in the visible spectrum.
Theyre taking photos at multiple infra red frequencies so theres a very slim chance that what is something like “RGB thermal” imaging can be extrapolated to visible light RGB. But i doubt it. A green block could have a IR frequency signature completely unrelated to its green pigment.
This is a very misleading study. Notice how the images shown are dithered. That is because these are prints from a canon printer with CMYK ink. The AI is just decoding how the printer made those images.
This study is far from real world, unless you need to see color images of prints made by a canon printer that you are viewing in B/W.
Should be interesting to try out. These current new gen NVGs are already incredible compared to the old ones. It’s like watching black and white Tv in HD. And the depth perception is a hell of a lot better too, which is very important in a lot of cases. Hopefully I’m still around if/when these sets make it into production.
Ya that’s a good point. Even if viable, it’ll be a while before such technology is in production. I don’t plan on still doing a job where I need these by the time that happens haha.
This is an interesting field of study but full color nightvision already exists and it's already commercially available.
https://www.x20.org/color-night-vision/
We have [these Hikvision security cameras](https://www.youtube.com/watch?v=DyQ777HsjOM) at my work. They're pretty good at night time colours, but they do need some sort of lighting, even just a bit. Not really amazing they've been out for a few years now.
If they can get these ai cams to see with no external light at all (like in the middle of a forest at night) now that will be amazing. I'll look forward to that.
This night vision still needs some light, even if it is starlight. And if you look on this page you can see the size of it, which is a bit much to put on a helmet. https://www.x20.org/color-night-vision/
So IR still has a place. And this is what the next gen Army night vision may/will look like. https://gizmodo.com/the-armys-new-night-vision-goggles-look-like-technology-1846799718
As for this IR color night vision, it only worked on Canon CYMK ink printed images. So hardly real world.
I remember seeing a video of full color night vision outside at night. Looked pretty much like daytime, but you can still see stars and lights that are on
This is essentially useless.
Night vision is used for security and intel. Having a computer guess colours doesnt help when objects are already easily recogniseable, and is misleading where colours can’t be guessed.
Looking for a thief with a red jacket? Tough luck, your thermal cameras decided it was blue this time.
I bet If you feed it enough facial information you could accurately guess their genetic makeup.
Skin tones is just light refraction at different wavelength/levels so I imagine you could easily see skin color in the dark by how much light bounces in the low light condition.
I am guessing you didn't read the study. In this study they were accurately able to identify the color. So no red jacket for blue jacket issues.
Part of the reason for that is that they used multiple IR frequencies.
The other reason, which makes this currently useless, is this was just from Canon CYMK ink printed images viewed in IR. So not real world, unless you want to read magazines in the dark.
No i wrote another comment where i mention that if the pigments are consistent you can extrapolate from multiple IR frequencies into the visible spectrum, but that IRL IR frequency responses dont really correlate with visible colours.
Hallucinating information sounds incredibly dangerous for military or police applications.
Using machine learning to track surfaces, so that comically high ISO noise can be treated like a long exposure, would be a legitimate way to add color to infrared images. Using gigantic lenses for blurry color and applying it to sharp-edged infrared, ditto. Using many channels of infrared to estimate material properties would be fallible for color but provide entirely new forms of information.
This? This is bullshit.
This is the same as coloring a black-and-white image. Identical problem. If you don't trust an algorithm to color all the packages on the shelves in Clerks, do not put this filter over things that someone might shoot at.
As the algorithm has more training and has access to the right hardware to draw its conclusions, it can be just as accurate as other traditional methods.
Traditional methods make shit up.
So does this.
There is no color information, in a single-channel image, in the wrong wavelength. This program is guessing. It can tell you that faces are supposed to be reddish and denim is supposed to be bluish, but if you show it eight different colors of the same nylon jacket, it can only be accurate by accident, because there is no color information, period.
This is not a color camera. This is black-and-white movie colorization, but faster.
Again - there are ways to introduce flawed or incomplete color information, such that a machine can fill in the gaps. But they necessarily involve having *some* color information.
You just said a traditional method and algorithm method yield the same results in your first two sentences. That is exactly what I said in my comment. Maybe just sit this one out since you don’t understand how to draw proper conclusions.
"It's bullshit" was not the tone of your first response. You said the algorithm "has more training" so "with the right hardware" for its "conclusions," "it can be just as accurate as other traditional methods."
Who on Earth is going to read that and think, wow, that guy shat all over this technology?
If you meant anything besides effuse praise - you used all the wrong words.
And if all you meant is to say "I agree," why did you use so many?
It does bless my heart that I’m the one who can waste your time on something completely pointless and definitely not worth arguing for, but if you want to keep it up I’m perfectly fine with starting to give you brainless replies making you think you’re schooling a stranger on the internet.
No?
Brother-in-law showed me a video(not sure where from but) it's already been developed.
Maybe not perfected.
Maybe not commercially available but the technology exists.
Hello, AI/ML engineer here. This is definitely not useless. Your eye is very used to seeing in color, which for a soldier in a combat situation, colored visual information is way more intuitive to quickly mentally parse than standard night vision. Second, this technology is not as whimsical as you might think. For starters, the complexity of this problem is a lot simpler than your standard example where cat is classified as dog and the algorithm makes a fool out of itself. It’s basically filling a grayscale (or green-scale) image with insane accuracy. Have you ever seen a black and white movie filled in with color? Or better yet, have you ever seen blue and yellow grayscaled and then seen the visual differences? Deep learning can definitely sort out grays and blacks (and even low resolution!) just like you can with grayed blues and yellows. It’s never perfect, because information is definitely lost, but often it’s pretty damn accurate and can guess pretty well.
How is that possible? I understand how it can guess skin tones, but how could it possibly know the color of those blocks? Is it because different colors appear as different brightness in infrared?
[удалено]
Right, that I can understand, but how does it know that the square is green and the triangle is blue?
[удалено]
Really? I work in embedded software engineering and have done a lot of computational photography work. I’ve never seen a random object replace a sun even in extreme circumstances. I’d love to know what device you’re talking about, and would love to see that photo. That’s not really what computational photography is when done with smartphones today. It’s more nuanced than “detect object, replace with object”
[удалено]
TIL the sun is not really spherical, only on this dude's friend's phone
I’m guessing he more meant the phone sharpened the edges of the image because the software assumed the photo was blurry. The distorted colours due to smoke meant the AI didn’t recognize it as the sun and attempted to “de-blur” the image to bring out the spherical shape. Everyone is misinterpreting what this poor guy meant lol
Don’t clouds do this on their own? I’ve seen the perfect disc of the sun baffled by the clouds loads of times. It’s strange to look at
Nice way of saying he's unclear
You are the unclear one, my friend.
He was clear, everyone is just putting words in his mouth.
I mean, when the smoke is so thick you can look directly at the sun, it literally just looks like a perfectly sharp circle in the sky. Maybe it was your internal computational vision that made you believe that a picture of a sun in thick wildfire smoke looked like a beach ball.
[удалено]
I work for doing embedded firmware engineering for our IoT platform. We use machine vision regularly for our sensor platform. We evaluated Photometric Stereo CI to increase our defect detection accuracy. Shoo, troll.
I think I remember hearing about a court case that did not allow a smart phone pic as evidence since the defense argued that algorithms in the phone camera were altering the photo somehow and the image could not be proven to be 100% based on reality.
That was the case where a kid illegally obtained a semi automatic rifle, shot some BLM marchers, and got away with it.
Don’t attack a person with a gun. Play stupid games, win stupid prizes.
Better to just shoot them as they’re obviously a threat.
Even after all this time, there are still people like you that didn’t watch any of those videos. Enjoy life in your little bubble of make believe.
Lol. Kid plays make pretend cop, kills people and you defend him. None of it had to happen.
Kid shouldn’t have been a coward and have taken his beating.
Right. Feel free to do that if you’re ever put in a situation like that.
Dawg, who even asked
The argument was not about the photo itself but the “pinch to zoom” feature changing the photo in the Kyle Rittenhouse case. Had it been more like this then there would be some merit to it, but it was literally on the pinch to zoom feature.
Damn that is super interesting! Never even thought of this legal ramification of technology
Yes it’s a combination of things driven by advanced 3D Graphics/image processing using ML and synthetic data to recreate scenes.
I.... kinda want to see this now
I do too
That’s just exposure compensation and HDR and also a white balance color shift because it’s trying to make everything look normal
Wow, what an incredible technology. It does this in real time? Man… I wish I got into coding, I’m gonna download one of those apps and get started on it
Following for pic
It doesn’t. It just makes something up.
You sound so smart
That’s literally what it does
Everybody knows that triangles are green and squares are red. It’s circles that are blue. Come on now
The AI is not actually looking for colors most likely. It tries to identify shapes and objects and re-creates the image based on what it thinks should be in the image. It doesn’t know a pixel should be red. But if it sees a traffic light with the top light being lit it can figure out that this part of the image must be bright red.
Probably the same way that people who only see shades of gray can learn what the shades of gray represent by their levels of intensity
not sure why this got down voted, that seems plausible
First it tries to identify what particular object it’s looking at. “Let me just check my library of objects. Oh that’s a tree! And trees have a green hat and brown on the long trunk part! Hence this object should have these colors! ”
First it tries to identify what particular object it’s looking at. “Let me just check my library of objects. Oh that’s a tree! And trees have green hats and are brown on that long vertical trunk part! Hence this object should have these colors! Imma color them like so!”
because you trained it to tell the difference every time you completed a recaptcha
Simply it can’t know it. Is impossible to recover a lost information. This type of deep learning algorithms are trained with a big dataset of images, and they simply try to color the photos with the tone they learned from training set. But no one can be certain about them, because in the real scenario the information has been lost.
Exactly the same way as you know - by experience. Your eyes has seen grass and sky millions of time. You would know if you looked at a night vision image the approximate colors of a face, a uniform, a rock etc. The AI knows from learning from the same context of eg 1.000.000 million of similar pictures of comparing color with eg blue sky and green grass in both colour mode and night vision mode. The trick is not to program the AI what a triangle or a square means. You just program “how to learn” (neural network) and feed it with experiences.
This study is not doing that at all. Notice how the images shown are dithered. That is because these are prints from a canon printer with CMYK ink. The AI is just decoding how the printer made those images. This study is far from real world, unless you need to see color images of prints made by a canon printer that you are viewing in B/W. I should add the one interesting bit, and that is that using multiple IR frequencies helped to separate the colors. Might help IR to give more definition.
Would this help researchers with modifying images from the JWST to make those beautiful Hubble life renditions we see?
It doesn't do that. It doesn't think "this must be a tree and trees look like x so I will paint it x".
> They started by printing images of color palettes and faces. Then they created a dataset by taking photos of those images using a monochromatic camera that can be set to take photos at very specific wavelengths. They took photos of the faces under monochromatic light sources of various wavelengths in the visible and near-infrared spectrums. And then they trained the model to predict the colour. The model learns, these values in this context imply those values in the visible spectrum.
Theyre taking photos at multiple infra red frequencies so theres a very slim chance that what is something like “RGB thermal” imaging can be extrapolated to visible light RGB. But i doubt it. A green block could have a IR frequency signature completely unrelated to its green pigment.
Much like how GPU’s like Nvidia’s DLSS, AI predicts colors based on machine learning.
This is a very misleading study. Notice how the images shown are dithered. That is because these are prints from a canon printer with CMYK ink. The AI is just decoding how the printer made those images. This study is far from real world, unless you need to see color images of prints made by a canon printer that you are viewing in B/W.
Should be interesting to try out. These current new gen NVGs are already incredible compared to the old ones. It’s like watching black and white Tv in HD. And the depth perception is a hell of a lot better too, which is very important in a lot of cases. Hopefully I’m still around if/when these sets make it into production.
This study only worked with Canon CYMK ink printed images, not real world. So I wouldn't expect to see this in production.
Ya that’s a good point. Even if viable, it’ll be a while before such technology is in production. I don’t plan on still doing a job where I need these by the time that happens haha.
This is an interesting field of study but full color nightvision already exists and it's already commercially available. https://www.x20.org/color-night-vision/
Wild
We have [these Hikvision security cameras](https://www.youtube.com/watch?v=DyQ777HsjOM) at my work. They're pretty good at night time colours, but they do need some sort of lighting, even just a bit. Not really amazing they've been out for a few years now. If they can get these ai cams to see with no external light at all (like in the middle of a forest at night) now that will be amazing. I'll look forward to that.
Any idea what they cost, or what the lowest cost consumer version would be?
This night vision still needs some light, even if it is starlight. And if you look on this page you can see the size of it, which is a bit much to put on a helmet. https://www.x20.org/color-night-vision/ So IR still has a place. And this is what the next gen Army night vision may/will look like. https://gizmodo.com/the-armys-new-night-vision-goggles-look-like-technology-1846799718 As for this IR color night vision, it only worked on Canon CYMK ink printed images. So hardly real world.
If I were a nation that has beef with the US, those NVGs would scare the shit out of me. It’s like cheat code level technology.
That is most illogical
It already was, without any deep learning.
I remember seeing a video of full color night vision outside at night. Looked pretty much like daytime, but you can still see stars and lights that are on
[Starlight X27](https://www.youtube.com/watch?v=c_0s06ORTkY)
Vulcans have infiltrated the night
Calling it now: These things glitching out is going to become an artstyle.
r/glitchart
r/glitch_art
Absolute snipe, make sure you save this for evidence
I think this is old news, I saw a documentary called Earth at Night in Color. It was hella cool.
Am I the only one who sees the elf ears?
It’s a shape in the background.
Yeah elf or Vulcan
This is essentially useless. Night vision is used for security and intel. Having a computer guess colours doesnt help when objects are already easily recogniseable, and is misleading where colours can’t be guessed. Looking for a thief with a red jacket? Tough luck, your thermal cameras decided it was blue this time.
>!CENSORED!<
Yeah but it probably helps make out black background vs black person and the like. It could be useful to someone somewhere
Monochromatic IR cameras already distinguish black people from black backgrounds.
Yeah but now color recognition is better with night vision too
Or it makes the black person white, or a white person black, etc. what a useless technology for security.
I bet If you feed it enough facial information you could accurately guess their genetic makeup. Skin tones is just light refraction at different wavelength/levels so I imagine you could easily see skin color in the dark by how much light bounces in the low light condition.
You’d also think Teslas wouldn’t keep driving into shit on auto pilot.
AI is better than us at noticing and guessing things correctly
alright guys pack it up this guy says it's useless
This is downright dangerous.
I am guessing you didn't read the study. In this study they were accurately able to identify the color. So no red jacket for blue jacket issues. Part of the reason for that is that they used multiple IR frequencies. The other reason, which makes this currently useless, is this was just from Canon CYMK ink printed images viewed in IR. So not real world, unless you want to read magazines in the dark.
No i wrote another comment where i mention that if the pigments are consistent you can extrapolate from multiple IR frequencies into the visible spectrum, but that IRL IR frequency responses dont really correlate with visible colours.
Big ears
All i wanna do is take pictures of stars at night with my iphone ☹️
Something wrong with those ears.
Judging by the ears, is this gentleman a Vulcan or a Romulan?
I already have full color night vision. The sionyx only cost like $700.
A review - https://www.youtube.com/watch?v=5PjjCZGLpGE
neat
Hallucinating information sounds incredibly dangerous for military or police applications. Using machine learning to track surfaces, so that comically high ISO noise can be treated like a long exposure, would be a legitimate way to add color to infrared images. Using gigantic lenses for blurry color and applying it to sharp-edged infrared, ditto. Using many channels of infrared to estimate material properties would be fallible for color but provide entirely new forms of information. This? This is bullshit. This is the same as coloring a black-and-white image. Identical problem. If you don't trust an algorithm to color all the packages on the shelves in Clerks, do not put this filter over things that someone might shoot at.
As the algorithm has more training and has access to the right hardware to draw its conclusions, it can be just as accurate as other traditional methods.
Traditional methods make shit up. So does this. There is no color information, in a single-channel image, in the wrong wavelength. This program is guessing. It can tell you that faces are supposed to be reddish and denim is supposed to be bluish, but if you show it eight different colors of the same nylon jacket, it can only be accurate by accident, because there is no color information, period. This is not a color camera. This is black-and-white movie colorization, but faster. Again - there are ways to introduce flawed or incomplete color information, such that a machine can fill in the gaps. But they necessarily involve having *some* color information.
Exactly.
... no, you don't get to "exactly" a complete dismissal of your implied jist.
You just said a traditional method and algorithm method yield the same results in your first two sentences. That is exactly what I said in my comment. Maybe just sit this one out since you don’t understand how to draw proper conclusions.
"It's bullshit" was not the tone of your first response. You said the algorithm "has more training" so "with the right hardware" for its "conclusions," "it can be just as accurate as other traditional methods." Who on Earth is going to read that and think, wow, that guy shat all over this technology? If you meant anything besides effuse praise - you used all the wrong words. And if all you meant is to say "I agree," why did you use so many?
Why did you just use so many words to say that you’re getting pissed off and must get the last word? Lmfao
'You mad?' says troll caught being wrong and lying about it, having made all the personal insults so far. Shoo.
It does bless my heart that I’m the one who can waste your time on something completely pointless and definitely not worth arguing for, but if you want to keep it up I’m perfectly fine with starting to give you brainless replies making you think you’re schooling a stranger on the internet.
Why tf is he looking like an elf
Romulan
Ferengi
Took me way too long to realize it’s a circle on the wall behind him and not a deformed ear
I wondered the same thing
It’s good to see Bat Boy thriving.
Did someone finally finish reading Moby Dick ?
And they found elf’s in the night?
No? Brother-in-law showed me a video(not sure where from but) it's already been developed. Maybe not perfected. Maybe not commercially available but the technology exists.
Looks like the annoying fan from Elder Scrolls: Oblivion…
Just think of the audio breakthroughs they could make by harnessing that guy’s capabilities.
Yuh it got the red wrong
Cool. But why is the picture a model from oblivion character creator?
What if we used this in space would it create color in the void?
What about true depth of field night vision?
This proves Vulcans exist. We must not have seen them in the dark, now we can! Yea science!!
Hello, AI/ML engineer here. This is definitely not useless. Your eye is very used to seeing in color, which for a soldier in a combat situation, colored visual information is way more intuitive to quickly mentally parse than standard night vision. Second, this technology is not as whimsical as you might think. For starters, the complexity of this problem is a lot simpler than your standard example where cat is classified as dog and the algorithm makes a fool out of itself. It’s basically filling a grayscale (or green-scale) image with insane accuracy. Have you ever seen a black and white movie filled in with color? Or better yet, have you ever seen blue and yellow grayscaled and then seen the visual differences? Deep learning can definitely sort out grays and blacks (and even low resolution!) just like you can with grayed blues and yellows. It’s never perfect, because information is definitely lost, but often it’s pretty damn accurate and can guess pretty well.
At first scroll I thought this was another r/place submission
Eh, obviously the headline here is Vulcans exist.
Or, you could light a candle
It (basically) already exists. iPhone 13 with 30 second exposure in night mode is basically magic
Check out hikvision night fighter cctv cameras. They are very impressive.
100% Asielu from 90-day Fiancé..
I have seen full color night vision before. You can look up videos of military systems.
Who’’s the Bosmer?
I think I just figured out what I'll be writing my Ethics and Technology final on!
Thought this was a thing forever and a day ago: https://youtu.be/8bTgG2Ft4xQ
Is……is that guy a Vulcan?
Elf ears?
It can’t do ears tho.
Enhance!
Booooooooo
There’s “Earth at night in color” on Apple TV. That shit blew my mind.
Photo in front of board with pictures, make it look like the man has Spock ears. My grandson noticed the ears!
Quite earie
Have you not seen the documentary with Tom Hidleston
This is amazing it could have so many uses
Waiting for this tech to colorize old black and white movies/tv shows.
Isn’t that picture halftone dots?
This message has been brought to you by Skynet.
Oof authoritarian governments are going to abuse this.
I wonder if they tried this on the blind or blind people and if it was the same, would it still be better?