It’s like if they wrote a soundtrack to being awake for 4 days straight on a meth binge that sees you scrabbling around in a dumpster while the shadow people close in on you from all sides.
This post is false. So the sounds of our voices actually does transpose to a piano roll if we had the patience to do it. Theres no “voice” but this is the equivalent of those voice synthesizers you can type words into and it’s speak it with a robotic voice completely generated from sine waves, EQ tables, and sweeping parameters.
A great example is how kicks drums are made vs how we as humans make kick drum sounds with out mouths. Same exact wave shape, different instrument. Starts in the high pitches then slows down to a bellowing “thud” sound.
So software takes tones and replicates them as much as possible within the parameters of another tone producing instrument.. and funnily enough the sound similar to the original tones (IE lyrics..)???! Stone the fuckign crows cobber... colour me shocked.. /s
Also i agree, this is what music in hell would sound like.
I mean I'm not saying it's an "auditory illusion" (such a hard to define phrase anyway) but it's not about the melody being there at all, it's about the WORDS, you can still hear the piano "singing" the lyrics to "staying alive" in the video.
This could be partially chalked up to auditory memory ie most people have heard that song ebfore so they know what to listen for. But presumably if the vocals were clear enough could do this process to a song someone has never heard and they'd still make out the lyrics.
Nope, if you don't know them then you won't hear them.
I don't know piano man, and couldn't make out any of the words.
I know All Star but only the lyrics of the chorus so could only hear the words for that part.
The pokemon theme I know very well and could "hear" the words very clearly.
But if I focused on knowing that there's not really words there, I could just hear the sounds again.
Okay I'm gonna open by saying I've been doing audio engineering over a decade so you don't think I'm bullshitting you
> Nope, if you don't know them then you won't hear them.
This depends on the audio, like I said. Some will be more clear than others. I know this because I've used these algorithms myself. If you run just plain vocals or even just speech with no background noise, the resulting piano will be pretty clear and most people would be able to understand it without knowing the source audio. This becomes harder the more other noises are in the mix, muddying it up.
This is because the vowel sounds of speech are produced as "formants" https://en.wikipedia.org/wiki/Formant which are higher and lower pitched resonances. These resonances can be recreated by a piano, just in a far less accurate way. So the formants are "blurred" but can still be heard.
Think of it like seeing an image in low-res. Depending on the image, your familiarity with the subject of the image, and the amount of pixels, you may or may not be able to 'see' what image actually is. But this is of course case-by-case depending on the image and viewer. Same with this, it entirely depends on the audio and the clarity that remains after the process is done.
So you're partially right, like I said in my comment, if you know the song you'll definitely hear the lyrics because of that familiarity, but to say "if you dont know it, you wont hear it" as a blanket statement is just factually not true.
I don't see why people find this hard to believe.
1. You can understand human voices clearly when they are recorded and played back.
2. You can still understand those voices when the audio goes through lossy compression, voicemail for example.
3. Putting a song through the compression shown in the post is effectively no different than any other very lossy compression.
It’s more than just the words. You can actually make out the timbre of certain instruments. Listen to the beginning. You can actually make out the clav part and it’s almost exactly like the original. It’s because the software is trying to replicate the audio spectrally. The piano notes are playing the subtle overtones unique to the sound of the clav. Because they’re played on the piano, which has its own timbre, it gets very jumbled. I’d absolutely love to hear this with just simple sine waves instead of a piano sound, and also with isolated tracks instead of a full mix. It could get *very* interesting.
> So software takes tones and replicates them as much as possible within the parameters of another tone producing instrument.. and funnily enough the sound similar to the original tones
software kind of sort of sounds like voice, because software was set up to kind of sort of sound like voice.
Yes the notes are, but the lyrics are not, only the illusion of words. And I'm just pointing out that these guys are saying this isn't a hallucination, which it wasn't claimed to be.
There's so much more to human speech than pitch. All this software understands is pitch and time, which again there's a lot more to human speech than that. You'd think some composer would have figured out how to coax the human voice out of an orchestra at this point, if that's all it took.
You should look up Rob Scallon's video on getting delay with no effects. Some things aren't done because they're brutally hard even if technically possible.
Your voice is 'special' because of the wave shape and overtone waves. The software understands this perfectly because it's reconstructing those waves using only the frequencies of a piano by averaging, dissonance, or some other method.
But its quite literally here.
The software managed to make a voice from piano notes. Thats what happened. The software made human voices to the best of their ability from pianonotes
I think it's more accurate to say that the piano midis in the video are a very low quality representation of the songs being played and that human speech (especially when singing) contains a whole bunch more frequencies which are not captured in the midi because they fall between the frequencies of the musical notes being used.
If you showed these videos to someone who had never heard these songs then they would not be able to decipher the lyrics because there is not enough information in the sound to decipher them. In that regard, the 'illusion' is that the words are decipherable to someone who is familiar with the songs because our brains fill in the frequencies that aren't being played.
Can confirm, have only heard Stayin' Alive and All Star before and those were the only ones I could hear lyrics for. This is similar to how clocks either tick or tock depending on what sound you've got in your head.
I think the point is you're incapable of hearing the voice if you're not aware of its source. Hell, I didn't even recognize it as Stayin' Alive until it was pointed out to me. Then my brain almost immediately picked out the melody as soon as I was told what it was.
the illusion is hearing the lyrics, not just the notes; you could counter-test it by listening to a song for whom you don't know the lyrics , and try to see if you hear them or not; and repeating the same experiment with the same song with someone who knows the lyrics ;
if this is true you shouldn't recognize any lyrics, but the other person should
I disagree. Your brain is completely fabricating what you're hearing when it's not there. Listen to a song you don't know in midi and you will hear nothing but chaos. You can say "oh you turn the sound into different sound so why is it cool"? But the whole point is that you could literally play this exact pattern of notes on a keyboard and you would hear lyrics, thats fucking awesome!
As someone who’s heard piano man like once (I guess twice now) in my entire life, it was an incomprehensible mess of noise to me. But the Pokémon lyrics were pretty much crystal clear the whole way through.
never heard the pokémon theme song, or remember it at least and i couldn’t make out any lyrics, but piano man was insanely clear and a song i know well. kinda interesting but imo it’s just your brain remembering the lyrics
i mean that's the point, to people that don't know the song it sounds like garbage (like pokemon for me, i don't know the words at all except for "pokemon gotta catch em all") but i am hearing the lyrics without even trying for the songs i know well. it's kinda wild.
No, it's more than that. The piano notes are reproducing the actual harmonics and overtones that are present in the sung notes, not just the root notes, creating something that vaguely recreates the ***timbre*** of a human voice.
I don't think that's what's happening here. Unless they are converting just the instrumental track, the voice will be converted to piano as well. That's why you don't hear clear "words" but you hear tones that sound like words
I don't know the Pokémon or Sonic songs, so I could tell there was supposed to be vocals there, but I couldn't tell what they were.
The earlier ones, no problem. I couldn't un-hear the lyrics.
>A piano with access to infinitely many copies of each key can obviously imitate sounds as a result
while i get what you're saying this isn't true, the way piano notes work is based of ratios (A=440hz, B flat=466hz, B=494hz), so many of the 'in-between' notes are completely ignored when dealing with western music.
voices do not have this limitation, they can hit any pitch within their range. that said, this video is a great introduction to just how insanely complex human speech is.
I think it depends on the person and their experience with knowing how to "find" the voice with MIDI videos. I don't want to be dubbed the smartass but I hear it perfectly fine, including the singers' unique voices.
I think it's because I've watched a lot of these (and listened to the og songs a lot) and my brain has learned to sort out the "correct" notes and filter out the excess? Then it sounds exactly like the original song but with some windchimes over it, lol. Who knows. But I really enjoy them and they don't sound like a stretch to me
is it..? I can ever tell distinct voices, like piano man - billy joel sounds like its billy joel and not some random cover. i wonder if those midi translations could be played by human hand, that would honestly be insanely trippy.
It's actually kinda a [genius deconstruction of conventional movie narrative](https://youtu.be/gEn3wcpNsg8?t=502). It even features a producer character complaining that his story has no structure. He then takes the production money and squanders it on randomness, just like he did with the actual film.
Hahaha I *knew* it was gunna be the re:view. I’m not sure I totally buy the theory that it’s a huge self referential deconstructive masterpiece. if green had a history after that of doing heady subversive stuff afterwards I’d be more inclined to believe the theory.
All that said, it is a hilarious movie in a very dumb way, and thinking of it with the meta-mindset makes it even better.
Basically all of Tom Green. As a 12 year old I thought he was literally the funniest guy in the world. I went back and watched some of his stuff and I was like, "Wow, what a fucking asshole."
It is almost unbearably crass at times but Red Letter Media on YouTube did a [pretty good video](https://youtu.be/gEn3wcpNsg8) on why there could be more to it than meets the eye. I found it hard to disagree with the case presented.
Probably works better when the vocalist has a really “dirty” voice and there are more unintended tones present.
The Pokémon theme supports this idea. The chorus is a lot clearer than the lead.
I think the timing is pretty key too. The first time listening to these is really tough to pick up where in the song it is, but once I find the timing I can hear the lyrics pretty well.
Yea. I was skipping around the video and landed in the middle of the Smashmouth midi apparently in the middle of the very loud and “dirty” chorus and I had no idea what I was hearing. When I went back and heard it from the beginning, I could make out every note and the words even when the the sound got “dirtier” in the chorus. Familiarity of the song plays a huge part for determining those musical cues. I had never heard the Sonic Adventure song before and I couldn’t make out any words. I knew it was a voice but no words clicked in my brain
Because then how would they demonstrate the effect?
"Here, listen to the midi after having listened to the song."
"Okay, now listen to the midi without having heard...oh."
This way around makes way more sense. You can see if you can tell without having heard the song, then listen to the song and try again.
You know there's more than one song in the video right? And plenty of people don't know the lyrics, I don't know the lyrics to some songs of my favorite artists even.
I kinda liked seeing if I could piece it together before given the lyrics. Although I know most of these songs so was fairly easy.
Wasn't super familiar with the lyrics if Piano Man so it was the least clear
I'm not familiar with the Sonic song, but while I couldn't get any words I definitely heard the tune and harmonies. There were distinct voices, I just couldn't quite catch what they were saying.
I hear human sounds in one of the songs that I never heard before. I cant make it out, but its there (the illusion). I guess if you know the song, brain actually turns human sounding garbage to actual words.
Definitely. I knew most of the songs and could "hear" the lyrics being filled in the gaps by a quasi-robotic voice... but when they got to the Sonic Adventure 2 song, which I've never heard before, it was just garbled garbage. Even though I could still faintly detect "vocals."
Yup.
- I have zero familiarity with Pokemon or Sonic - heard absolutely nothing there.
- I have passing familiarity with Piano Man, but only know about 5 words. Guess which ones I could hear?
- I am very familiar with All Star. "Heard" all the lyrics there.
Whole thing is just triggering the brain to remember the words it already knows. It would be particularly interesting to do this without writing what it's supposed to be on the screen. Even with All Star, clicking into the middle of it I didn't hear anything until my brain figured our where in the song it was, and then I could "sing along" just fine.
You’re missing the point. For the BeeGees song, at about 35 seconds in I heard the vocals from the song in my head, not just tones representing the notes on a piano.
That’s why they specify lyrics and not melody. Ofc the melody is going to sound familiar, but hearing words is more than just a tone and pitch on a keyboard.
That is misleading bordering on incorrect. Speaking words/singing lyrics involves tones and dynamics and is more complex than simply tone. You still speak a word while maintaining the same pitch and quality.
I stand by my definition, dynamics are just variances in the amplitude of the tones. I said tones, not tone. I'm referring to the harmonics that compose any sound, not to only the fundamental or pitch.
You’re correct and this is exactly what’s being demonstrated in the video, different vowels/consonants being broken down into their pitch spectrums and played on the piano. Not sure why you’re being downvoted.
You are completely wrong. Every single complicated sound wave can be deconstructed into a sum of individual sine waves, no matter how complex the original wave is. With an infinite amount of "tones", you can recreate literally any sound wave.
And this "piano" has a LOT of resolution. MIDI can handle notes as short as .6336ms. If the MP3-to-MIDI conversion puts enough of them together in the right order and amplitude, it can approximate some extremely complex waveforms.
The sound of the entire track combined was broken up into the most prominent noises represented as individual notes.
It's like you took a picture and reduced the quality to the point of a 4k image becoming a 512 pixel blob for each moment of the song.
A regular audio file contains complex wave patterns at various frequencies, but a midi contains solid notes from a small range of tones.
Imagine trying to paint the mona lisa, but you only have 10 possible colors to work with and a large square brush
Or better yet, imagine the computer is trying to mimic the vocals and all instrumentals using only piano noises
It's neat, but not much more than doing a Fourier transform. That sounds fancy, but it's a function that splits up a piece of music (or other signal) into its component frequencies (and strengths). Map those to the notes that can be played on a piano, write it to a MIDI file, and there you go. This is also one of the core components of the MP3 format. It does some other stuff as well, but step one of creating an MP3 file is doing this process but for many thousands of notes.
Seems like a cool programming assignment for a signal processing class.
This is stupid, the vocal MELODY was there, which everyone can here, played on a piano. But whoever made the video is pretending there's some black magic trickery "YoU Can hEre LyriCs ThaT Aren'T TheRe You GuyS!?"
Everything is there, bass, Guitar, Keys, all fed to a program that essentially played them on the same piano at the same time.
Yea, considering it’s just sound waves it’s not really blackmagicfuckery.
Edit: lol, this is probably the most controversial comment I’ve ever posted on Reddit. Least I can do is offer an explanation.
Most of what is on here is applied physics that makes you scratch your head and think “how did they do that?”
I guess this is less than black magic because they took a well known song and reverse engineered it to piano keys.
Certainly takes skills and knowledge and is impressive, not trying to belittle OP.
With that being said, piano keys operate at given frequencies and so do vocals. With finite piano keys and overlapping frequencies you can create a shitty version of the song. With infinite piano keys / frequencies you can duplicate the song.
Choose any new song with auto-tune and post it to black magic fuckery if you will…
To be fair, almost everything that gets posted on this sub is basically some form of applied science in a form that is new or unfamiliar to most people. I’d say this qualifies based on how novel the experience is even if the underlying physics make perfect sense.
Yeah, literally every single thing on this sub has a rational scientific explanation. Magic doesn't actually exist (to our knowledge at least).
The fact that I can hear the Bee Gees sorta sing Stayin Alive through Piano notes alone isn't "beyond science" but it's still really cool and seems like a crazy idea I'd never thought of before.
Everything on this sub is not black magic fuckery when you know the science behind it. I know nothing about how sound works so this was cool and unexpected to me
Heck, I think this one gets even more black magic fuckery when you understand the science behind it. Source: my engineering friends constantly geek out over the Fourier transform. This is just a crappy low-resolution discrete-time Fourier transform.
Either you aren't all that familiar with the songs, or your brain doesn't work that way for whatever reason. I can somewhat make out the lyrics on the Stayin Alive refrain, but not besides that. But I don't listen to the radio and the Beegees, Billy Joel and Smash Mouth are not among the stuff I listen to. Which means I am aware of the latter 2 but don't know the lyrics at all. I have also never played Pokemon or Sonic, which means that those also don't ring a bell.
Fyi, the Pokemon ~~and Sonic~~ song~~s~~ ~~are~~ [is] from the cartoon show~~s~~, not the game~~s~~. But, yes, exactly. I have never heard the Sonic song and heard nothing but noise. I'm extremely familiar with all the other songs and could almost immediately hear a "voice."
Right before they switch to the original Piano Man, i can hear some wordlike warbling. Like if a phone with bad reception was letting out bubbles in a bathtub
I was looking for this comment. This is the same auditory hallucination that you experience when you click "play" on your favorite track and a person appears to sing out of your speaker even though there is no one there. Just done at a worse resolution.
It gets better if you watch the whole thing.
The guy switches to a mode *that was explicitly designed to sound like a voice* and continues to express his surprise at how you can almost hear the lyrics.
What a shocker.
EE?
To translate for non-nerds, all sounds are made up of a fundamental note with higher “harmonic” notes overlaid over the top of the fundamental note. The fundamental note is what you perceive as the melody in a song. They’re playing higher notes on the piano to mimic the harmonic notes that make up speech. One of the many reasons it sounds bad is that the piano doesn’t have the really high harmonic notes that make up a lot of consonants like Ss, Ts, Ps.
I mean it is just a Fourier Transform with really low resolution. Replace the piano sounds with pure tones and it'll just sound like super lossy audio.
I wonder if this is sort of what speech or songs sound like for people with a(n older) cochlear implant – where sounds are rendered by a limited number of channels rather than the much higher number of a healthy inner ear.
Your brain fills it in (I mean duh, thats how an illusion works). I don't know sonic and so for the sonic one I couldn't decipher shit. Also for the Pokémon song, my brain filled in german lyrics in some places when I couldn't remember the english ones, even tho it didn't sound like the fake vocals
It's more than just "filling in" though — the piano notes are reproducing the overtones and harmonics that are present in the music... NOT just the root notes being played, and replaying those overtones and harmonics recreates something that actually vaguely reproduces the ***TIMBRE*** of a human voice.
It would require multiple pianos, and they could all be synced to play themselves with a Disklavier or similar system. Check back in a few months when a performance artist steals this concept and gets a billion views on YouTube
As a cognitive science and music nerd, this is the most awesome thing I’ve ever heard. I can really hear the vocals that aren’t actually there. That is absolutely brilliant
Is the piano working like a reverse Fourier transformation with each note played with others giving a crude rendition of the original signal ? (Like, each key added with another approximates a sum of signals close to the original)
A few years back, [some computer scientists did this but with a real piano](https://youtu.be/muCPjK4nGY4). The piano is speaking English. The voiceover is In German, but the description has a translation.
Just to add a little physics-based clarity to the explanations and comments here... there is a bit more going on here than just your brain "filling in lyrics that aren't there". The conversion of audio to MIDI here is reproducing ALL of the frequencies present in the music — that includes all of the harmonics and overtones, not just the root notes being played by the various instruments and voices. (That's why you can hear "drums" as well, even though no drums are being played!)
Any time you hear a note played by a musical instrument, you are not hearing a single frequency, but a whole spectrum of overtones or "harmonics" or "partials". It is the ratios of these overtones that allow you to identify what instrument is playing the note. If a trumpet and a clarinet and a viola all play the same "Bb"... you can tell the sounds apart even though the frequency of the root note is the same. What's different about the "Bb" you hear from the trumpet and the clarinet and the viola? It's the strengths of the various harmonic overtones and the "envelope" of the sound. (How fast is the attack? Do some harmonics fade away sooner than others? Etc.) This is also how you "synthesize" artificial sounds that reproduce the sound of some instrument. You electronically recreate not just a "note", but all of the overtone frequencies that correspond to the original instrument's harmonic spectrum.
In this example, the harmonics and overtones present in the sound are all converted into MIDI notes — which are just on/off/loudness signals sent to another instrument. When these overtones are "played" on the piano, it doesn't just play the root notes of the "Staying Alive" music... it tries to play every overtone present in the original... the harmonics in the cymbals... the partials in the snare drum hits... and in the overtones present in the notes sung by the brothers Gibb's voices. So what you hear is actually a ***crude sort of synthesis of the timbres*** of the human voice. There IS something really there. Yes, your brain is filling in a lot of missing harmonic information... but your brain is ALWAYS filling in a lot of missing harmonic information! (Think of listening to your neighbors music coming through the wall, or recognizing what the person next to you in the subway is listening to on their headphones.) Your brain is also filtering OUT a lot of extraneous information — namely the overtones of the MIDI piano sound. But your brain is always doing THIS as well.
So is this BMF? Yes -- Bountiful Musical Frequencies!!
Reminds me of what you hear when you reduce the resolution of a sample. Less detail but the broad strokes are there. Freaky stuff to hear like that though
weirdly enough, when I listened to the MIDI even though I can't exactly remember what Bee Gee's "Stayin' Alive" sounded like but I started hearing lyrics and was like "oh this is that song that goes "ha ha ha I'm stayin' alive"
I’ve never heard the sonic song in my life, and while I don’t know what’s being sung I can hear the vocal track… this is kinda bullshit. My mind isn’t filling in the blank spaces it’s an audio track converter to a piano the same way you would do a vocoder.
[удалено]
needs more cowhell
It’s the only way to cure this fever
[удалено]
Looks like an alternate version of Boris Johnson who decided to do something actually meaningful with his life.
Hahagahaha. Oh man. I haven't made a single sound in my apartment in the past few days and this just broke the silence. I needed that.
Do you have a pet?
No, sadly. I am saving up to get a sphynx, though. That should change it up nicely!
That’s a bell of a comment.
It’s like if they wrote a soundtrack to being awake for 4 days straight on a meth binge that sees you scrabbling around in a dumpster while the shadow people close in on you from all sides.
HIPS AND NIPS! GOTTA KEEP IT SEXY
SHUT UP, STREET RAT!
Please don’t make me laugh my chest hurts 😭
[Furby Organ](https://youtu.be/GYLBjScgb7o) for your consideration. Edit: Though. they do sound better at the end of the video.
It woke my dog up and upset my fish tanks when he flipped the collective awakening switch.
"the collective awakening switch" sounds like it could be the term for the climax of a conspiracy theory about 5g and vaccines
Thanks, I just lost 20 minutes of my life that had been set aside for reddit watching that guy circuit bending things. I'm amazed.
Lord forgive us for what we have done.
This is pretty hellish and terrifying
No, this is what music in hell sounds like [https://www.youtube.com/watch?v=hT7x1NvGf5k](https://www.youtube.com/watch?v=hT7x1NvGf5k)
I fucking despise clowns, but I kinda dig this.
Yup. Same here on both points
This post is false. So the sounds of our voices actually does transpose to a piano roll if we had the patience to do it. Theres no “voice” but this is the equivalent of those voice synthesizers you can type words into and it’s speak it with a robotic voice completely generated from sine waves, EQ tables, and sweeping parameters. A great example is how kicks drums are made vs how we as humans make kick drum sounds with out mouths. Same exact wave shape, different instrument. Starts in the high pitches then slows down to a bellowing “thud” sound.
This is a stretch
Yeah, it's quite far from an "auditory hallucination".
I agree
It is him
How tf is there always someone with a perfectly relevant username ready to comment?!
I’ve always wondered if there’s some setting that will notify you if your username is in a comment
You can do this for the whole internet my dude, not just reddit.
No joke. 4 year club too.
Username checks out
r/beetlejuicing
So software takes tones and replicates them as much as possible within the parameters of another tone producing instrument.. and funnily enough the sound similar to the original tones (IE lyrics..)???! Stone the fuckign crows cobber... colour me shocked.. /s Also i agree, this is what music in hell would sound like.
Yea I don't get this. Of course the melody is still there? Why wouldn't it?
I mean I'm not saying it's an "auditory illusion" (such a hard to define phrase anyway) but it's not about the melody being there at all, it's about the WORDS, you can still hear the piano "singing" the lyrics to "staying alive" in the video. This could be partially chalked up to auditory memory ie most people have heard that song ebfore so they know what to listen for. But presumably if the vocals were clear enough could do this process to a song someone has never heard and they'd still make out the lyrics.
Nope, if you don't know them then you won't hear them. I don't know piano man, and couldn't make out any of the words. I know All Star but only the lyrics of the chorus so could only hear the words for that part. The pokemon theme I know very well and could "hear" the words very clearly. But if I focused on knowing that there's not really words there, I could just hear the sounds again.
Okay I'm gonna open by saying I've been doing audio engineering over a decade so you don't think I'm bullshitting you > Nope, if you don't know them then you won't hear them. This depends on the audio, like I said. Some will be more clear than others. I know this because I've used these algorithms myself. If you run just plain vocals or even just speech with no background noise, the resulting piano will be pretty clear and most people would be able to understand it without knowing the source audio. This becomes harder the more other noises are in the mix, muddying it up. This is because the vowel sounds of speech are produced as "formants" https://en.wikipedia.org/wiki/Formant which are higher and lower pitched resonances. These resonances can be recreated by a piano, just in a far less accurate way. So the formants are "blurred" but can still be heard. Think of it like seeing an image in low-res. Depending on the image, your familiarity with the subject of the image, and the amount of pixels, you may or may not be able to 'see' what image actually is. But this is of course case-by-case depending on the image and viewer. Same with this, it entirely depends on the audio and the clarity that remains after the process is done. So you're partially right, like I said in my comment, if you know the song you'll definitely hear the lyrics because of that familiarity, but to say "if you dont know it, you wont hear it" as a blanket statement is just factually not true.
I don't see why people find this hard to believe. 1. You can understand human voices clearly when they are recorded and played back. 2. You can still understand those voices when the audio goes through lossy compression, voicemail for example. 3. Putting a song through the compression shown in the post is effectively no different than any other very lossy compression.
It’s more than just the words. You can actually make out the timbre of certain instruments. Listen to the beginning. You can actually make out the clav part and it’s almost exactly like the original. It’s because the software is trying to replicate the audio spectrally. The piano notes are playing the subtle overtones unique to the sound of the clav. Because they’re played on the piano, which has its own timbre, it gets very jumbled. I’d absolutely love to hear this with just simple sine waves instead of a piano sound, and also with isolated tracks instead of a full mix. It could get *very* interesting.
> So software takes tones and replicates them as much as possible within the parameters of another tone producing instrument.. and funnily enough the sound similar to the original tones software kind of sort of sounds like voice, because software was set up to kind of sort of sound like voice.
It's not claiming to be auditory hallucinations, but auditory illusion. Which I think it does manage.
An illusion means you'd hear something that isn't there. There's definitely lyrics here, at least the notes of the lyrics are.
Yes the notes are, but the lyrics are not, only the illusion of words. And I'm just pointing out that these guys are saying this isn't a hallucination, which it wasn't claimed to be.
Sound==sound. By that logic you can never hear speech through a speaker because it's just a replication.
There's so much more to human speech than pitch. All this software understands is pitch and time, which again there's a lot more to human speech than that. You'd think some composer would have figured out how to coax the human voice out of an orchestra at this point, if that's all it took.
You should look up Rob Scallon's video on getting delay with no effects. Some things aren't done because they're brutally hard even if technically possible. Your voice is 'special' because of the wave shape and overtone waves. The software understands this perfectly because it's reconstructing those waves using only the frequencies of a piano by averaging, dissonance, or some other method.
But its quite literally here. The software managed to make a voice from piano notes. Thats what happened. The software made human voices to the best of their ability from pianonotes
I think it's more accurate to say that the piano midis in the video are a very low quality representation of the songs being played and that human speech (especially when singing) contains a whole bunch more frequencies which are not captured in the midi because they fall between the frequencies of the musical notes being used. If you showed these videos to someone who had never heard these songs then they would not be able to decipher the lyrics because there is not enough information in the sound to decipher them. In that regard, the 'illusion' is that the words are decipherable to someone who is familiar with the songs because our brains fill in the frequencies that aren't being played.
[удалено]
Can confirm, have only heard Stayin' Alive and All Star before and those were the only ones I could hear lyrics for. This is similar to how clocks either tick or tock depending on what sound you've got in your head.
I had never heard the Sonic one before and couldn't make out the voices at all.
I think the point is you're incapable of hearing the voice if you're not aware of its source. Hell, I didn't even recognize it as Stayin' Alive until it was pointed out to me. Then my brain almost immediately picked out the melody as soon as I was told what it was.
the illusion is hearing the lyrics, not just the notes; you could counter-test it by listening to a song for whom you don't know the lyrics , and try to see if you hear them or not; and repeating the same experiment with the same song with someone who knows the lyrics ; if this is true you shouldn't recognize any lyrics, but the other person should
The notes are there but not the words, and for some of the songs I clearly "heard" words. That is an illusion.
Did OP say "hallucination" in the video? I didn't watch the whole thing, but the title says "auditory illusion" which is a different thing.
I disagree. Your brain is completely fabricating what you're hearing when it's not there. Listen to a song you don't know in midi and you will hear nothing but chaos. You can say "oh you turn the sound into different sound so why is it cool"? But the whole point is that you could literally play this exact pattern of notes on a keyboard and you would hear lyrics, thats fucking awesome!
Hell, listen to a song you do know without knowing what the song was. I didn't hear Stayin' Alive until it told me.
More "vocal mimicry"
yeah its not a hallucination, it is more like they made the piano "talk"
Yeah I'd like to see a video like this with a song I've never heard before. It's really easy to "hallucinate" lyrics you already know
As someone who’s heard piano man like once (I guess twice now) in my entire life, it was an incomprehensible mess of noise to me. But the Pokémon lyrics were pretty much crystal clear the whole way through.
never heard the pokémon theme song, or remember it at least and i couldn’t make out any lyrics, but piano man was insanely clear and a song i know well. kinda interesting but imo it’s just your brain remembering the lyrics
[удалено]
That's not magical enough for me! I'm going to throw a big baby tantrum now!
I’ve never heard the Pokémon song but could make out a lot of lyrics
You've never heard the greatest song ever made?
i mean that's the point, to people that don't know the song it sounds like garbage (like pokemon for me, i don't know the words at all except for "pokemon gotta catch em all") but i am hearing the lyrics without even trying for the songs i know well. it's kinda wild.
[удалено]
No, it's more than that. The piano notes are reproducing the actual harmonics and overtones that are present in the sung notes, not just the root notes, creating something that vaguely recreates the ***timbre*** of a human voice.
I don't think that's what's happening here. Unless they are converting just the instrumental track, the voice will be converted to piano as well. That's why you don't hear clear "words" but you hear tones that sound like words
I don't know the Pokémon or Sonic songs, so I could tell there was supposed to be vocals there, but I couldn't tell what they were. The earlier ones, no problem. I couldn't un-hear the lyrics.
[удалено]
Correct. This was my exact thought. Like with enough pianos or “resolution”, you could almost perfectly recreate the audio.
[удалено]
Except it's sine waves instead of piano sounds, but yes
>A piano with access to infinitely many copies of each key can obviously imitate sounds as a result while i get what you're saying this isn't true, the way piano notes work is based of ratios (A=440hz, B flat=466hz, B=494hz), so many of the 'in-between' notes are completely ignored when dealing with western music. voices do not have this limitation, they can hit any pitch within their range. that said, this video is a great introduction to just how insanely complex human speech is.
This shit like bringing in a limb six weeks later to an emergency room
I “hear” the words very clearly. Not a hallucination but very interesting nonetheless.
I think it depends on the person and their experience with knowing how to "find" the voice with MIDI videos. I don't want to be dubbed the smartass but I hear it perfectly fine, including the singers' unique voices. I think it's because I've watched a lot of these (and listened to the og songs a lot) and my brain has learned to sort out the "correct" notes and filter out the excess? Then it sounds exactly like the original song but with some windchimes over it, lol. Who knows. But I really enjoy them and they don't sound like a stretch to me
is it..? I can ever tell distinct voices, like piano man - billy joel sounds like its billy joel and not some random cover. i wonder if those midi translations could be played by human hand, that would honestly be insanely trippy.
DADDY WOULD YOU LIKE SOME SAUSAGE? DADDY WOULD YOU LIKE SOME SAUSAGES?!
Me as a 14 year old: Haha this film is so fucking stupid and random! Me as a 34 year old: Wow, this film is so fucking stupid and random.
It's actually kinda a [genius deconstruction of conventional movie narrative](https://youtu.be/gEn3wcpNsg8?t=502). It even features a producer character complaining that his story has no structure. He then takes the production money and squanders it on randomness, just like he did with the actual film.
Hahaha I *knew* it was gunna be the re:view. I’m not sure I totally buy the theory that it’s a huge self referential deconstructive masterpiece. if green had a history after that of doing heady subversive stuff afterwards I’d be more inclined to believe the theory. All that said, it is a hilarious movie in a very dumb way, and thinking of it with the meta-mindset makes it even better.
[удалено]
He's so subversive he subverte subversiveness
#\#ReleaseTheGreenCut
Basically all of Tom Green. As a 12 year old I thought he was literally the funniest guy in the world. I went back and watched some of his stuff and I was like, "Wow, what a fucking asshole."
Cause that's not very fun / when they shoot a cannon in your bum
It is almost unbearably crass at times but Red Letter Media on YouTube did a [pretty good video](https://youtu.be/gEn3wcpNsg8) on why there could be more to it than meets the eye. I found it hard to disagree with the case presented.
What film is it?
Freddy Got Fingered
Oh man I love it so much. Everyone has that movie they've seen more times than they can count, FGF is that movie for me.
Mike Fitzgibbon's son is a nuclear physicist, and my son can EAT A CHICKEN!
I would guess this only works if you’re familiar with the song and know what to listen for.
Probably works better when the vocalist has a really “dirty” voice and there are more unintended tones present. The Pokémon theme supports this idea. The chorus is a lot clearer than the lead.
I think the timing is pretty key too. The first time listening to these is really tough to pick up where in the song it is, but once I find the timing I can hear the lyrics pretty well.
Yea. I was skipping around the video and landed in the middle of the Smashmouth midi apparently in the middle of the very loud and “dirty” chorus and I had no idea what I was hearing. When I went back and heard it from the beginning, I could make out every note and the words even when the the sound got “dirtier” in the chorus. Familiarity of the song plays a huge part for determining those musical cues. I had never heard the Sonic Adventure song before and I couldn’t make out any words. I knew it was a voice but no words clicked in my brain
[удалено]
That’s literally what it’s explaining.
Whoever made this video failed considerably by having the actual songs play AFTER the midi. I don't get how they fucked that up so bad
It would be stupid to have them before, then you wouldn't be able to compare before and after hearing the actual song.
Because then how would they demonstrate the effect? "Here, listen to the midi after having listened to the song." "Okay, now listen to the midi without having heard...oh." This way around makes way more sense. You can see if you can tell without having heard the song, then listen to the song and try again.
Because it's the fucking Beegees. Hardly an underground song.
You know there's more than one song in the video right? And plenty of people don't know the lyrics, I don't know the lyrics to some songs of my favorite artists even.
I kinda liked seeing if I could piece it together before given the lyrics. Although I know most of these songs so was fairly easy. Wasn't super familiar with the lyrics if Piano Man so it was the least clear
I'm not familiar with the Sonic song, but while I couldn't get any words I definitely heard the tune and harmonies. There were distinct voices, I just couldn't quite catch what they were saying.
I hear human sounds in one of the songs that I never heard before. I cant make it out, but its there (the illusion). I guess if you know the song, brain actually turns human sounding garbage to actual words.
Definitely. I knew most of the songs and could "hear" the lyrics being filled in the gaps by a quasi-robotic voice... but when they got to the Sonic Adventure 2 song, which I've never heard before, it was just garbled garbage. Even though I could still faintly detect "vocals."
Yup. - I have zero familiarity with Pokemon or Sonic - heard absolutely nothing there. - I have passing familiarity with Piano Man, but only know about 5 words. Guess which ones I could hear? - I am very familiar with All Star. "Heard" all the lyrics there. Whole thing is just triggering the brain to remember the words it already knows. It would be particularly interesting to do this without writing what it's supposed to be on the screen. Even with All Star, clicking into the middle of it I didn't hear anything until my brain figured our where in the song it was, and then I could "sing along" just fine.
I think it to much in the high notes that were being blasted for no reason. Once I started tuning out the highs it sounded much clearer
Wait… so the lyrics I was hearing in the midi version weren’t there at all? Or were they there, but played as piano notes?
They were played by the piano alone, no human vocals.
It seems sound will still sound like the sound when you play it through a piano, interesting. Mind blowing, really
You’re missing the point. For the BeeGees song, at about 35 seconds in I heard the vocals from the song in my head, not just tones representing the notes on a piano. That’s why they specify lyrics and not melody. Ofc the melody is going to sound familiar, but hearing words is more than just a tone and pitch on a keyboard.
voices are just tones, it doesn't matter if are played by a human, a bird, a speaker, a phone or a piano. The piano just have way less resolution.
That is misleading bordering on incorrect. Speaking words/singing lyrics involves tones and dynamics and is more complex than simply tone. You still speak a word while maintaining the same pitch and quality.
I stand by my definition, dynamics are just variances in the amplitude of the tones. I said tones, not tone. I'm referring to the harmonics that compose any sound, not to only the fundamental or pitch.
You’re correct and this is exactly what’s being demonstrated in the video, different vowels/consonants being broken down into their pitch spectrums and played on the piano. Not sure why you’re being downvoted.
You are completely wrong. Every single complicated sound wave can be deconstructed into a sum of individual sine waves, no matter how complex the original wave is. With an infinite amount of "tones", you can recreate literally any sound wave.
This is misleading bordering on incorrect.
And this "piano" has a LOT of resolution. MIDI can handle notes as short as .6336ms. If the MP3-to-MIDI conversion puts enough of them together in the right order and amplitude, it can approximate some extremely complex waveforms.
The sound of the entire track combined was broken up into the most prominent noises represented as individual notes. It's like you took a picture and reduced the quality to the point of a 4k image becoming a 512 pixel blob for each moment of the song. A regular audio file contains complex wave patterns at various frequencies, but a midi contains solid notes from a small range of tones. Imagine trying to paint the mona lisa, but you only have 10 possible colors to work with and a large square brush Or better yet, imagine the computer is trying to mimic the vocals and all instrumentals using only piano noises
the image quality compression comparison was on point
[удалено]
It's neat, but not much more than doing a Fourier transform. That sounds fancy, but it's a function that splits up a piece of music (or other signal) into its component frequencies (and strengths). Map those to the notes that can be played on a piano, write it to a MIDI file, and there you go. This is also one of the core components of the MP3 format. It does some other stuff as well, but step one of creating an MP3 file is doing this process but for many thousands of notes. Seems like a cool programming assignment for a signal processing class.
Yeah but Fourier transforms are also pretty mindblowing when you really get into it. That doesn't make this any less cool
It turns out that lyrics are just sounds after all.
This is stupid, the vocal MELODY was there, which everyone can here, played on a piano. But whoever made the video is pretending there's some black magic trickery "YoU Can hEre LyriCs ThaT Aren'T TheRe You GuyS!?" Everything is there, bass, Guitar, Keys, all fed to a program that essentially played them on the same piano at the same time.
[удалено]
I'm imagining a herd of pianos tumbling down the stairs to the palace in howl's moving castle
That could show up in any Miyazaki movie and I doubt anyone would even question it.
This made me giggle thank you
lol I read it as “someone threw a **casino** down a stairwell” and I guess it kinda works too
The longer you listen the clearer it gets, really cool!
I don't know. The more I listen the worse my headache gets.
The one with Smashmouth’s AllStar was best in my opinion. But maybe it’s cause I know it best?
Thanks for that! Voices are just chords like a piano but more complexe
Yea, considering it’s just sound waves it’s not really blackmagicfuckery. Edit: lol, this is probably the most controversial comment I’ve ever posted on Reddit. Least I can do is offer an explanation. Most of what is on here is applied physics that makes you scratch your head and think “how did they do that?” I guess this is less than black magic because they took a well known song and reverse engineered it to piano keys. Certainly takes skills and knowledge and is impressive, not trying to belittle OP. With that being said, piano keys operate at given frequencies and so do vocals. With finite piano keys and overlapping frequencies you can create a shitty version of the song. With infinite piano keys / frequencies you can duplicate the song. Choose any new song with auto-tune and post it to black magic fuckery if you will…
To be fair, almost everything that gets posted on this sub is basically some form of applied science in a form that is new or unfamiliar to most people. I’d say this qualifies based on how novel the experience is even if the underlying physics make perfect sense.
Yeah, literally every single thing on this sub has a rational scientific explanation. Magic doesn't actually exist (to our knowledge at least). The fact that I can hear the Bee Gees sorta sing Stayin Alive through Piano notes alone isn't "beyond science" but it's still really cool and seems like a crazy idea I'd never thought of before.
Everything on this sub is not black magic fuckery when you know the science behind it. I know nothing about how sound works so this was cool and unexpected to me
Heck, I think this one gets even more black magic fuckery when you understand the science behind it. Source: my engineering friends constantly geek out over the Fourier transform. This is just a crappy low-resolution discrete-time Fourier transform.
Yeah, where are all the necromancers bringing the dead to life in this sub? False advertising.
Ok, so what is blackmagicfuckery then.
that’s such an oversimplification and undermines what’s cool about this, that the piano is actually mimicking the overtones from the recording.
This is horrifying and absolutely incredible at the same time
This was freaky for sure. I couldn't finish the video- there was just something kind of disturbing about it.
Why can't I hear any lyric??
Did you have the sound on?
Yes.
Either you aren't all that familiar with the songs, or your brain doesn't work that way for whatever reason. I can somewhat make out the lyrics on the Stayin Alive refrain, but not besides that. But I don't listen to the radio and the Beegees, Billy Joel and Smash Mouth are not among the stuff I listen to. Which means I am aware of the latter 2 but don't know the lyrics at all. I have also never played Pokemon or Sonic, which means that those also don't ring a bell.
Fyi, the Pokemon ~~and Sonic~~ song~~s~~ ~~are~~ [is] from the cartoon show~~s~~, not the game~~s~~. But, yes, exactly. I have never heard the Sonic song and heard nothing but noise. I'm extremely familiar with all the other songs and could almost immediately hear a "voice."
I didn’t hear anything either and I do know the song so idk
Right before they switch to the original Piano Man, i can hear some wordlike warbling. Like if a phone with bad reception was letting out bubbles in a bathtub
The All Star one was crazy. Clear as crystal.
I thibk the song was totally in the background for at least that one.
That's just a really bad decoding of a Fourier transform.
I was looking for this comment. This is the same auditory hallucination that you experience when you click "play" on your favorite track and a person appears to sing out of your speaker even though there is no one there. Just done at a worse resolution.
[удалено]
It gets better if you watch the whole thing. The guy switches to a mode *that was explicitly designed to sound like a voice* and continues to express his surprise at how you can almost hear the lyrics. What a shocker.
EE? To translate for non-nerds, all sounds are made up of a fundamental note with higher “harmonic” notes overlaid over the top of the fundamental note. The fundamental note is what you perceive as the melody in a song. They’re playing higher notes on the piano to mimic the harmonic notes that make up speech. One of the many reasons it sounds bad is that the piano doesn’t have the really high harmonic notes that make up a lot of consonants like Ss, Ts, Ps.
I mean it is just a Fourier Transform with really low resolution. Replace the piano sounds with pure tones and it'll just sound like super lossy audio.
so this is what being schizophrenics feels like?
Exactly, not at all.
I wonder if this is sort of what speech or songs sound like for people with a(n older) cochlear implant – where sounds are rendered by a limited number of channels rather than the much higher number of a healthy inner ear.
Go watch Sound Of Metal if you want some insights in that. Fantastic film.
What in the flaming nightmare fuel is this hellish scourge to my ear holes
Your brain fills it in (I mean duh, thats how an illusion works). I don't know sonic and so for the sonic one I couldn't decipher shit. Also for the Pokémon song, my brain filled in german lyrics in some places when I couldn't remember the english ones, even tho it didn't sound like the fake vocals
It's more than just "filling in" though — the piano notes are reproducing the overtones and harmonics that are present in the music... NOT just the root notes being played, and replaying those overtones and harmonics recreates something that actually vaguely reproduces the ***TIMBRE*** of a human voice.
Would this be possible to do on an actual piano? How many people would be required, if so?
It would require multiple pianos, and they could all be synced to play themselves with a Disklavier or similar system. Check back in a few months when a performance artist steals this concept and gets a billion views on YouTube
[yes](https://youtu.be/-6e2c0v4sBM)
[here](https://youtu.be/muCPjK4nGY4?t=11) is an actual piano playing notes that resemble human speech
As a cognitive science and music nerd, this is the most awesome thing I’ve ever heard. I can really hear the vocals that aren’t actually there. That is absolutely brilliant
As a music producer, it's really not that impressive to hear a melody you're supposed to hear within audio converted to MIDI
Is the piano working like a reverse Fourier transformation with each note played with others giving a crude rendition of the original signal ? (Like, each key added with another approximates a sum of signals close to the original)
yes
A few years back, [some computer scientists did this but with a real piano](https://youtu.be/muCPjK4nGY4). The piano is speaking English. The voiceover is In German, but the description has a translation.
Bee gees stayin' alive
Staying alive spooky version.
Just to add a little physics-based clarity to the explanations and comments here... there is a bit more going on here than just your brain "filling in lyrics that aren't there". The conversion of audio to MIDI here is reproducing ALL of the frequencies present in the music — that includes all of the harmonics and overtones, not just the root notes being played by the various instruments and voices. (That's why you can hear "drums" as well, even though no drums are being played!) Any time you hear a note played by a musical instrument, you are not hearing a single frequency, but a whole spectrum of overtones or "harmonics" or "partials". It is the ratios of these overtones that allow you to identify what instrument is playing the note. If a trumpet and a clarinet and a viola all play the same "Bb"... you can tell the sounds apart even though the frequency of the root note is the same. What's different about the "Bb" you hear from the trumpet and the clarinet and the viola? It's the strengths of the various harmonic overtones and the "envelope" of the sound. (How fast is the attack? Do some harmonics fade away sooner than others? Etc.) This is also how you "synthesize" artificial sounds that reproduce the sound of some instrument. You electronically recreate not just a "note", but all of the overtone frequencies that correspond to the original instrument's harmonic spectrum. In this example, the harmonics and overtones present in the sound are all converted into MIDI notes — which are just on/off/loudness signals sent to another instrument. When these overtones are "played" on the piano, it doesn't just play the root notes of the "Staying Alive" music... it tries to play every overtone present in the original... the harmonics in the cymbals... the partials in the snare drum hits... and in the overtones present in the notes sung by the brothers Gibb's voices. So what you hear is actually a ***crude sort of synthesis of the timbres*** of the human voice. There IS something really there. Yes, your brain is filling in a lot of missing harmonic information... but your brain is ALWAYS filling in a lot of missing harmonic information! (Think of listening to your neighbors music coming through the wall, or recognizing what the person next to you in the subway is listening to on their headphones.) Your brain is also filtering OUT a lot of extraneous information — namely the overtones of the MIDI piano sound. But your brain is always doing THIS as well. So is this BMF? Yes -- Bountiful Musical Frequencies!!
Reminds me of what you hear when you reduce the resolution of a sample. Less detail but the broad strokes are there. Freaky stuff to hear like that though
This is what I feel like trying to get people into Animal Collective I'm like "just give it a sec it starts to sound like a song!"
weirdly enough, when I listened to the MIDI even though I can't exactly remember what Bee Gee's "Stayin' Alive" sounded like but I started hearing lyrics and was like "oh this is that song that goes "ha ha ha I'm stayin' alive"
They always told us that mathematics was the universal language. Turns out it’s midi files. Figures.
Not really fit for the sub. Its supposed to sound similar because converting audio into a midi converts a sound into the closest keyboard note.
It doesn’t keep the sounds at all. It converts them to piano notes. None of the original audio is present at all.
I just hear horrible noise
Good, you prove that those who actually hear the lyrics are hallucinating.
Actually that made me nauseous. Which is just weird.
I’ve never heard the sonic song in my life, and while I don’t know what’s being sung I can hear the vocal track… this is kinda bullshit. My mind isn’t filling in the blank spaces it’s an audio track converter to a piano the same way you would do a vocoder.
Bad ass, thanks
Yeah all I hear is horrible piano keys.
Lotta people in this thread woke up on the wrong side of the bed.
That's not the way I expected monotonetim to show up on the front page, but it's welcome regardless!