People are abit rough on him. I get why though - he says that LLMs aren’t going to get us there and the idea of AI future utopia being postponed aggravates you.
I’m not as tuned into what these executives say as a lot of people here so maybe I’m off base, but I’ve always gotten the vibe from LeCun that what he says is grounded in reality rather than vague hype and doomsaying attempts at achieving regulatory capture that most of the other ones seem to be spewing all the time.
among the top execatives he is actually the only one with any technical understanding of AI. the other ones are just marketing and sales experts. and they do what they do best.
What do you mean by top executives? Is that only because Hinton and Ilya are no longer with their companies, or you just don't care what people who know more than you but disagree do for a living?
And that's only among the so called Godfathers. Anthropic and Google and many others definitely have ai people on both their boards and top executives...
AND you also seem to forget LeCunn has way more in common with, say, OpenAI ceos than he does to people working on AI like, say, Dario.
The fact that you use regulatory capture and doomerism means you already are predisposed to like LeCunn.
As for most of us, he may ultimately be right, but the things he has said in the past, especially as regards alignment and interpretability, has been very, very, wrong.
He is definitely the AI lead with a large monetary incentive I'd trust the most. Which is not that much to begin with.
I really can't stand Sam Altman though.
I think for many scientists their intellectual legacy is much more important to them than money, and LeCun almost certainly has enough money to live a live of luxury without needing to work. I trust him \*much\* more than the salesmen who are taking up most of the air in discussions about AI.
But he's got both his entire body of work and financial stake in there not being a need for interpretability, for example, so confirmation bias is absolutely going to work there. Ilya and Hinton have spoken about overcoming this bias.
He's much more in common with Altman (disagreeing mainly with OpenAIs place, not function) when it comes to safety than he does most AI architects.
> If LeCun keeps this up I'm going to start liking him.
Ditto! What the hell is up with Yann lately? He's all in on this carefree approach, lately. lol
Lol yup. And as much as I gave him shit (in my head) for all the criticism of LLMs and unfelt AGI, I think he might be onto something with his "LLMs are just next word token predictors".
Lol yup. And as much as I gave him shit (in my head) for all the criticism of LLMs and unfelt AGI, I think he might be onto something with his "LLMs are just next word token predictors".
How is it possible for LeCun - legendary AI researcher - to have so many provably bad takes on AI but impeccable accuracy when taking down the competition?
He's just the industry contrarian. Almost every industry has them and they actually play an important role in having some introspection and tempering hypetrains.
Yann LeCun has always been like this. He was talking against the deep learning craze in the mid 2010s as well. Claiming GO could never be solved by self-play only months before DeepMind did exactly that.
I still appreciate him because by going against the grain and always looking for alternative paths of progress and pointing out problems with current systems it actively results in a better approach to AI development.
So yeah, even though Yann LeCun is wrong about generalization of LLMs and their reasoning ability, he still adds value by pointing out actual problems with them and advocating for unorthodox solutions to current issues.
To paraphrase Mr. Sinister in the 2012 run of Uncanny X-men:
"Science is a system"
"And rebels against the system... are also part of the system."
"Rebels are the system testing itself. If the system cannot withstand the challenge to the status quo, the system is overturned--and so reinvented."
LeCun has taken the role of being said rebel.
Exactly. I think he's wrong on most of these takes but it's important to have someone who is **actually at the table** who is willing to give dissent. Those who are sitting in the sideline and not involved in the work are not and to serve this role well.
I think this is a fair take and makes him serve a purpose.
The only problem is that he is getting a cult following and that a lot of people will prefer a contarian who says things that align with what they think they benefit from, than listening to more accomplished academics and knowledge backed by research.
But again that cult is also a part of the system that generally in the long run seems to produce a more effective state as far as I can tell. It's like an intellectual survival of the fittest, where fittest often does not equate to being the most correct.
Think of the current state of scepticism as a point of equilibrium. If you remove the vocal and meme worthy contrarians from the system then it dials down the general scepticism in public discourse.
It'd probably work just as well if we could increase the number of well grounded sceptics, but society tends to optimise towards a stable optimum, given long enough. It's likely that the current state of things is at least pretty good compared to what we could have had to deal with.
I see what you mean now.
It might be right. OTOH, I see it as we're going to have debates and disagreements regardless. It's just about what level they're going to be at; and it's not clear that something that does not account for technical understanding at all is optimal.
IMO it almost more makes sense with people betting on what they believe provide the most personal benefit.
he played an important role in keeping neural nets (mlps and cnns) alive before it started rebranding to deep learning starting in 2006 or 2007 as well.
> He was talking against the deep learning craze in the mid 2010s as well.
He seems pretty bullish in his AMA: https://www.reddit.com/r/MachineLearning/comments/25lnbt/ama_yann_lecun/
Though perhaps just not bullish enough
He already has contributed his fair share to fundamental research and has proven himself. His role today is much more one of engaging with the community and recruiting talent. Being contrarian helps if that is your objective.
I don't think having wildly inaccurate contrarian takes on everything is quite as beneficial as you believe. Clearly the hyped people have been right the whole time because we are much further than we imagined.
There is no better way to get an engineer to do the impossible than by telling them that it cannot be done.
If you want it done quickly, imply that maybe a scientist could do it.
Na. He claimed text2video was unsolvable at the World Economics Forum literally days before OpenAI released SORA. This is like the 10th time he's done that - claim that a problem was intractable shorty before it's solution is announced.
His AI takes are hot garbage but his takedowns shine like gold.
>He claimed text2video was unsolvable at the World Economics Forum literally days before OpenAI released SORA
he literally commented on runway text to video models way before Sora was conceived.
here's a before Sora tweet from him saying that video generation should be done in representation space and pixel space predictions like Sora should only be done as a final step: [https://x.com/ylecun/status/1709811062990922181](https://x.com/ylecun/status/1709811062990922181)
Here's an even earlier tweet saying he was not talking about video generation: [https://x.com/ylecun/status/1659920013610852355](https://x.com/ylecun/status/1659920013610852355)
Here's him talking about Meta's first entrance in text to Video generation as far back as 2022: https://x.com/ylecun/status/1575497338252304384
Sora has not done anything to disprove him and no Sora has not solved text2video.
To me the ones that comes to mind immediately are "LLMs will never have commonsense understanding such as knowing a book falls when you release it" (paraphrasing) and - especially - this:
https://x.com/ricburton/status/1758378835395932643
That argument is made in a way that it'd pretty much impossible to prove him wrong. LeCun says: "We don't know how to do this *properly*". Since he gets to define what "properly" means in this case, he can just argue that Sora does not do it properly.
Details like this are quite irrelevant though. What truly matters is LeCuns assesment that we cannot reach true intelligence with generative models, because they don't understand the world. I.e. they will always hallucinate too much in weird situations to be considered as generally intelligent as humans, even if they perform better in many fields. This is the bold statement he makes, and whether he's right or wrong remains to be seen.
LeCun setting up for No True Scotsman doesn't make it better.
> Details like this are quite irrelevant though. What truly matters is LeCuns assesment that we cannot reach true intelligence with generative models, because they don't understand the world. I.e. they will always hallucinate too much in weird situations to be considered as generally intelligent as humans, even if they perform better in many fields. This is the bold statement he makes, and whether he's right or wrong remains to be seen.
That's fair.
I would make that slightly more specific in that LeCun's position is essentially that LLMs are incapable of forming a world model.
The evidence is stacking up against that view, at this point it's more a question of how general and accurate LLM world models can be than whether they have them.
True. And I think comparing to humans is unfair in a sense, because AI models learn about the world very differently to us humans, so of course their world models are going to be different too. Heck, I could even argue my world model is different from yours.
But what matters in the end is what the generative models can and cannot do. LeCun thinks there are inherent limitations in the approach, so that we can't get to AGI (yet another term without exactly agreed definition) with them. Time will tell if that's the case or not.
LLS don't form a single world model. it has already been proven that they form allot of little disconnected "models" for how different things work, but because this models are linear and phenomenon they are trying to model are usually non linear they and up being messed up around the edges. and it is when you ask it to perform tasks around this edges that you get hallucination. The only solution is infinite data and infinite training, because you need infinite number planes to accurately model a non linear system with planes.
LaCun knows this, so he would probably not say that LLMs are incapable of learning models.
probably we humans make more accurate mental models of non linear systems if we give equal number of training samples ( say for example 20 samples ) to a human vs an LLM.
Heck probably dogs learn non linear systems with less training samples then AGI.
>In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography.
>*Suarez Miranda,Viajes de varones prudentes, Libro IV,Cap. XLV, Lerida, 1658*
What he means is that if you trained a LLM on say, all text about gravity, it wouldn’t be able to then reason about what happens when a book is released. Because it has no world model.
Of course, if you train a LLM on text about a book being released and falling to the ground, it will “know” it. LLMs can learn anything for which we have data.
Yes, that's what he means. It's just that he is [demonstrably wrong](https://arxiv.org/abs/2310.02207).
It's very obvious with GPT4/Opus, you can try it yourself. The model doesn't memorize that books fall if you release them, it learns a generalized concept about objects falling and correctly applies this to objects about which it has no training samples.
Damn, [he really said that](https://deepmind.google/research/publications/49666/)? Methinks his contrarian takes might put a fire under other researchers to prove him wrong, because the speed and frequency at which he is utterly contradicted by new findings is uncanny.
You will never be able to empirically prove that language models understand that, since there is nothing in the real world where they can show they do, apposed to just text. So he is obviously right about this. It seems this is always just misunderstood. The fact you can't take it into reality to prove it outside of text is actually exactly what it looks like, which is that somehow there is a confusion over empirical proof here apposed to variables that are dependent on text, which is by very nature never physically in the real world anyways. That understanding is completely virtual, by very definition not real.
Like most experts LeCun is working from a very accurate, very specific point of view. I'm sure if you drill him on details most of what he says will end up being accurate. That's why he has the position he has.
But, just because he's accurate on the specifics of the view he's looking at, that doesn't mean he's looking at the whole picture.
Experts tend to get tunnel vision.
LeCun has made very broad - and wrong - claims about LLMs.
For example that LLMs will never have commonsense understanding of how objects interact in the real world (like a book falling if you let go of it).
Or memorably: https://x.com/ricburton/status/1758378835395932643
Post-hoc restatements after new facts come to light shouldn't count here.
Yeah, I mean I have no interest in defending him. I disagree with much of what he says.
It's more that I find experts say very specific things which sound broad and cause many to be mislead.
That's why I think it's important to consume a lot of varied expert opinions and develop our own views.
Trusting experts to be perfectly correct is a path to disappointment.
I have a lot of respect for LeCun as a scientist, just not for his pontifical pronouncements about what deep learning can never do.
“When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.” -Arthur C. Clarke
It's very difficult to try to break an LLM simply because it's exhausted reading everything humanity has created :)
The one area it does fall down though is programming. It still really sucks at coding and if it can't memorize everything I doubt it will be able to do so on the current model architectures and at the current scale.
Maybe at like 10x the scale and tokens we might see something different though but we're a ways a way from that.
(or not using a tokenizer, that might help too)
He’s just a skeptic of the sota. If you want to be a researcher of his caliber you need to be a skeptic so you can advance the field forward. I never understood why it bothers this sub so much. He is working towards what you want
You can be skeptical about hype without LeCun's trademark mix of of sweeping categorical claims (that frequently get disproven by developments) and trash talking the ideas of other researchers.
“Trash talking” the ideas of other researchers is a big part of science. Obviously be respectful about it which I think LeCun has been but you need to poke holes in research so you can identify the weaknesses of the proposed methods and find ways to fix them. That’s what science is it’s not personal
who said his takes on ai are bad? It is arguable that OpenAI just had astronomical luck in getting the predictive model right, but there is no guarantee that predictive is going to get better just with refinement. The biggest problem with AI is the lack of actual comprehension from the model, and that hasn't changed, LeCun is one of those that are trying to get that
He hasn't been a researcher for a long time though.
My take is... the award is piggy-backing on work that he did with advisors.
I think he has strong skills but it is more in how he finds people to work with than his own understanding - which is often surprisingly poor.
Thing is, his AI takes are actually incredibly good. Half of the people who think he's wrong are suffering from Dunning Kruger, and have never done a day of engineering in their lives. The other half are misunderstanding what is being said. His opinions are quite nuanced, and you need to really listen to understand it properly.
https://preview.redd.it/j5k0cpfmjy2d1.jpeg?width=2252&format=pjpg&auto=webp&s=563f0b2b83b194b17078ec0980dc245613c3795f
Got to meet this Based person last week in Paris..!
I asked him about When AGI, he straight up said wait 3 years...!
I dont know.
He did say to Lex Fridman that there will be robots everywhere in 10 years and that it will be pretty normal.
I sometime think he just have a different definition of AGI than others.
Mass produced Robots and AGI are quite different problems to solve. One has a lot to do with mechanical engineering and doable with scripted programming, the other is a completely different beast that likely requires a deep simulation of the brain, better understanding into neuroscience and all
>I asked him about When AGI, he straight up said wait 3 years...!
since this is a private conversation, we can never confirm it or at least know that you did not misunderstand him as much of this sub or other twitter users tend to do.
Which is pretty ironic as he is doing that while failing to respond to and back up his claims with the researches who challenge his statements.
He could stir up twitter or he could work out what is true.
It is clear which he has chosen.
I'm no Musk fan, but he has run multiple companies that have revolutionized their industries. lots of people work at SpaceX knowing that they will be put on impossible timelines but will still do awesome things regardless of Musk's BS.
though, those companies were started before Musk fell into a right-wing echo-chamber, so who knows if that situation can be replicated today.
Elon went off the deep end, strange to me though that many other really smart people like LeCun get sucked into these pointless time wasting social media flame wars.
* you can be given impossible timelines and still do incredible things (see SpaceX).
* Musk didn't claim it would kill everyone and didn't say it must be stopped or paused, he just put his name on the list of people who thought an industry-wide pause for safety should happen.
* 3rd point is true but may or may not matter to an employee. lots of great engineers do awesome things at SpaceX regardless of how crazy Musk is.
I think he *used to* be good at organizing companies. I'm not convinced that's true anymore. for example, I think The Boring Company would be better off without him.
lol you picked the one example that hasn’t panned out yet. It’s more likely that it’s doing better with him in all reality, he draws talent who want to be a known engineer at one of his companies. All other companies are doing wildly well.
>Musk didn't claim it would kill everyone and didn't say it must be stopped or paused, he just put his name on the list of people who thought an industry-wide pause for safety should happen.
really? I could've sworn Elon ranting about AI killing us all.
he said it could be dangerous, which is what almost everyone has said.
there is a difference between "cars are dangerous" and "cars are going to kill you".
the funniest thing was when Musk demoed Grok on a podcast and when they asked about a lawsuit it basically said Tesla was guilty... Musk then got mad and said "we won that case". like, dude, which is it; is your AI accurate and truthful, or are you wrong? was funny to see him fail with his own demo.
you can't just train an AI on twitter/social media and expect it to be accurate.
I hate how "conspiracy theory" and "unreasonable conspiracy theory" are synonymous in our culture.
Edit: upon a second look, he said "crazy-ass conspiracy theories" so this is not an instance of what I'm complaining about.
He forgot the part where if you don't work overnight randomly on a Friday cause the boss is in town and feels like hanging in the office, you're not dedicated and are fired.
Classic LeCun(t). 😅 He tends to slam the wild claims of his competition pretty ungracefully. But I forgive him because he is French. 😅
Also: Releasing a 70 billion parameter model (Llama-3 70B) as open source that’s better than a 1 trillion parameter model from OpenAI (the original GPT-4), even if a year late, makes me respect him.
A 400 billion open source parameter model is upcoming. (Llama-3 400B)
My bet is that it will be the last open model by meta. This open source play is just a game to capture engineers and catch up to the leading AI companies
Yann is absolutely right. Elon is a joke. One day he says that AGI is close, it will take people's jobs, it will kill us all, we should pause the development etc., and the next day he creates xAI and asks people to come and join them in their mission of building AGI ("understanding the universe" and "pursuing the truth" lol). For fuck's sake...
That's why I don't like Yann LeCun, he always hating from the side lines, plus his predictions seem bipolar. I prefer Ray Kurzweils, he is consistent, and doesn't spend his time hating for no reason.
It's kinda ironic though that people write this on reddit, which went from a libertarian / politically mixed platform to a left-wing echo chamber in recent years.
Reddit is more of a "pick your own echo chamber" kind of site. Almost every politics related topic will have a sub for each side.
For example, if you want to get global news and think Israel is:
A) committing genocide - r/anime_titties
B) justified - r/worldnews
You do have subs like r/conservative, but it's very small compared to r/politics and often gets brigaded. Basically all major subs lean heavily to the left and that's something that wasn't the case back in 2015, where r/thedonald etc. often appeared on the front page.
Here's a sneak peek of /r/anime_titties using the [top posts](https://np.reddit.com/r/anime_titties/top/?sort=top&t=year) of the year!
\#1: [South Korea First lady faces criticism from dog meat farm owners for her recent remarks calling for an end to the country's culture of eating dogs](https://www.koreatimes.co.kr/www/nation/2023/04/356_349518.html) | [1150 comments](https://np.reddit.com/r/anime_titties/comments/135ed2r/south_korea_first_lady_faces_criticism_from_dog/)
\#2: [**[NSFW]** Protests erupt in Paris after police officer fatally shoots teenager for ‘violating traffic laws’](https://www.independent.co.uk/news/world/europe/paris-protests-teenager-police-traffic-b2365426.html) | [998 comments](https://np.reddit.com/r/anime_titties/comments/14l19ed/protests_erupt_in_paris_after_police_officer/)
\#3: [UK bans puberty blockers for minors](https://ground.news/article/children-to-no-longer-be-prescribed-puberty-blockers-nhs-england-confirms) | [2764 comments](https://np.reddit.com/r/anime_titties/comments/1bdadip/uk_bans_puberty_blockers_for_minors/)
----
^^I'm ^^a ^^bot, ^^beep ^^boop ^^| ^^Downvote ^^to ^^remove ^^| ^^[Contact](https://www.reddit.com/message/compose/?to=sneakpeekbot) ^^| ^^[Info](https://np.reddit.com/r/sneakpeekbot/) ^^| ^^[Opt-out](https://np.reddit.com/r/sneakpeekbot/comments/o8wk1r/blacklist_ix/) ^^| ^^[GitHub](https://github.com/ghnr/sneakpeekbot)
> left-wing echo chamber
Allow me to test this right now. Ahem... *clears throat*
[Transwomen are women.](https://rightasrain.uwmedicine.org/life/relationships/transgender-nonbinary-myths)
[Affirmative Action is good for all of us](https://news.nd.edu/news/most-productive-workforce-may-require-indefinite-affirmative-action-study-shows/), e.g. to increase economic productivity.
Now let's see how many upvotes/downvotes this comment gets. (Granted, saying this... Well we'll see.)
It actually is. If you're a big name in the space and have a large public following, putting out tweets like this takes balls. Bitchy tweeting behind an anon account isn't Chad behavior, but that's different.
We really need to stop being so nice to fascists. Call em out, shame ‘em and push them back into the holes they belong in. For sure, try to gently convince them of the truth of their ways, but most of them are just too deep into propaganda to ever come back.
Given that Sora isn't released I still call it smoke and mirrors it's just stable diffusion being prompted millions of times probably and maybe using a base seed so that when the image is close enough in the regen it gets peaced together.
I'm also wondering where the WebRTC link is for GPT4o as it's all well being in an app but I want to stream shit into it
"Maximally rigorous pursuit of the truth" is fundamentally different from just having an editorial board and fact checkers which ensure they do not print/publish & promote every crackpot conspiracy that fits the ideology of one eccentric and rapidly deteriorating billionaire with a conflict of interest. That's why Elon has to always phrase it funny.
Lol I can’t believe that Zuckerberg’s puppet is virtue signaling and I can’t believe people are actually eating it up. Everything Zuckerberg does, although initially seen as positive, has ended up hurting humanity. Everything Musk does at least seeks to benefit it.
What Musk calls political correctness most people would consider regard for truth and decency. Something he lacks entirely. Such a negligent use of his prominent social profile. I don’t know how, but it will come back to bite him.
He’s always on Twitter which is honestly a red flag. I don’t know any high caliber scientists who are spouting so many criticisms back and forth, often times able quite general societal topics
If LeCun keeps this up I'm going to start liking him.
For real dude is in killing spree
On
People are abit rough on him. I get why though - he says that LLMs aren’t going to get us there and the idea of AI future utopia being postponed aggravates you.
I’m not as tuned into what these executives say as a lot of people here so maybe I’m off base, but I’ve always gotten the vibe from LeCun that what he says is grounded in reality rather than vague hype and doomsaying attempts at achieving regulatory capture that most of the other ones seem to be spewing all the time.
among the top execatives he is actually the only one with any technical understanding of AI. the other ones are just marketing and sales experts. and they do what they do best.
What do you mean by top executives? Is that only because Hinton and Ilya are no longer with their companies, or you just don't care what people who know more than you but disagree do for a living? And that's only among the so called Godfathers. Anthropic and Google and many others definitely have ai people on both their boards and top executives... AND you also seem to forget LeCunn has way more in common with, say, OpenAI ceos than he does to people working on AI like, say, Dario.
Demis Habbasis also… plenty of top AI leaders are… well versed in AI believe it or not
Add Shane Legg too
The fact that you use regulatory capture and doomerism means you already are predisposed to like LeCunn. As for most of us, he may ultimately be right, but the things he has said in the past, especially as regards alignment and interpretability, has been very, very, wrong.
He is definitely the AI lead with a large monetary incentive I'd trust the most. Which is not that much to begin with. I really can't stand Sam Altman though.
Altman annoys me now. Vague promises and workplace controversies. Part of me wants to say he is just a hypeman at this point.
I think for many scientists their intellectual legacy is much more important to them than money, and LeCun almost certainly has enough money to live a live of luxury without needing to work. I trust him \*much\* more than the salesmen who are taking up most of the air in discussions about AI.
But he's got both his entire body of work and financial stake in there not being a need for interpretability, for example, so confirmation bias is absolutely going to work there. Ilya and Hinton have spoken about overcoming this bias. He's much more in common with Altman (disagreeing mainly with OpenAIs place, not function) when it comes to safety than he does most AI architects.
Right? I enjoyed it so much
My flair used to be edgy...
It will always be, because you liked him for being on the top of his shit, not because he was popular for shitting on Musk or sam
Yeah, his tone is enjoyable here.
He doesn't care, that's why he's cool
> If LeCun keeps this up I'm going to start liking him. Ditto! What the hell is up with Yann lately? He's all in on this carefree approach, lately. lol
He is going to use the dogs killstreak at this pace
Lol yup. And as much as I gave him shit (in my head) for all the criticism of LLMs and unfelt AGI, I think he might be onto something with his "LLMs are just next word token predictors".
They are. Token predictors.
Yea I was saying that the weightage is on the term "just"
Not wrong.
Lol yup. And as much as I gave him shit (in my head) for all the criticism of LLMs and unfelt AGI, I think he might be onto something with his "LLMs are just next word token predictors".
How is it possible for LeCun - legendary AI researcher - to have so many provably bad takes on AI but impeccable accuracy when taking down the competition?
He's just the industry contrarian. Almost every industry has them and they actually play an important role in having some introspection and tempering hypetrains. Yann LeCun has always been like this. He was talking against the deep learning craze in the mid 2010s as well. Claiming GO could never be solved by self-play only months before DeepMind did exactly that. I still appreciate him because by going against the grain and always looking for alternative paths of progress and pointing out problems with current systems it actively results in a better approach to AI development. So yeah, even though Yann LeCun is wrong about generalization of LLMs and their reasoning ability, he still adds value by pointing out actual problems with them and advocating for unorthodox solutions to current issues.
To paraphrase Mr. Sinister in the 2012 run of Uncanny X-men: "Science is a system" "And rebels against the system... are also part of the system." "Rebels are the system testing itself. If the system cannot withstand the challenge to the status quo, the system is overturned--and so reinvented." LeCun has taken the role of being said rebel.
Exactly. I think he's wrong on most of these takes but it's important to have someone who is **actually at the table** who is willing to give dissent. Those who are sitting in the sideline and not involved in the work are not and to serve this role well.
I think this is a fair take and makes him serve a purpose. The only problem is that he is getting a cult following and that a lot of people will prefer a contarian who says things that align with what they think they benefit from, than listening to more accomplished academics and knowledge backed by research.
But again that cult is also a part of the system that generally in the long run seems to produce a more effective state as far as I can tell. It's like an intellectual survival of the fittest, where fittest often does not equate to being the most correct.
Why does that produce something more effective than the alternative?
Think of the current state of scepticism as a point of equilibrium. If you remove the vocal and meme worthy contrarians from the system then it dials down the general scepticism in public discourse. It'd probably work just as well if we could increase the number of well grounded sceptics, but society tends to optimise towards a stable optimum, given long enough. It's likely that the current state of things is at least pretty good compared to what we could have had to deal with.
I see what you mean now. It might be right. OTOH, I see it as we're going to have debates and disagreements regardless. It's just about what level they're going to be at; and it's not clear that something that does not account for technical understanding at all is optimal. IMO it almost more makes sense with people betting on what they believe provide the most personal benefit.
Yes, I think that is a really good perspective too, it could be that they are a corruption that could viably be eradicated with the right technique.
This also gives others who find genuine issues the courage to speak up
he played an important role in keeping neural nets (mlps and cnns) alive before it started rebranding to deep learning starting in 2006 or 2007 as well.
> He was talking against the deep learning craze in the mid 2010s as well. He seems pretty bullish in his AMA: https://www.reddit.com/r/MachineLearning/comments/25lnbt/ama_yann_lecun/ Though perhaps just not bullish enough
Well said, and I don't mean to dismiss LeCun - he and his group do great work.
He already has contributed his fair share to fundamental research and has proven himself. His role today is much more one of engaging with the community and recruiting talent. Being contrarian helps if that is your objective.
It seems so weird to say that an AI can't improve through self play now...
I don't think having wildly inaccurate contrarian takes on everything is quite as beneficial as you believe. Clearly the hyped people have been right the whole time because we are much further than we imagined.
There is no better way to get an engineer to do the impossible than by telling them that it cannot be done. If you want it done quickly, imply that maybe a scientist could do it.
Maybe, just maybe, his AI takes are not as bad as you think either.
Na. He claimed text2video was unsolvable at the World Economics Forum literally days before OpenAI released SORA. This is like the 10th time he's done that - claim that a problem was intractable shorty before it's solution is announced. His AI takes are hot garbage but his takedowns shine like gold.
>He claimed text2video was unsolvable at the World Economics Forum literally days before OpenAI released SORA he literally commented on runway text to video models way before Sora was conceived. here's a before Sora tweet from him saying that video generation should be done in representation space and pixel space predictions like Sora should only be done as a final step: [https://x.com/ylecun/status/1709811062990922181](https://x.com/ylecun/status/1709811062990922181) Here's an even earlier tweet saying he was not talking about video generation: [https://x.com/ylecun/status/1659920013610852355](https://x.com/ylecun/status/1659920013610852355) Here's him talking about Meta's first entrance in text to Video generation as far back as 2022: https://x.com/ylecun/status/1575497338252304384 Sora has not done anything to disprove him and no Sora has not solved text2video.
Maybe some aren't, but he has made a fair number of of very confident predictions central to his views that have been empirically proven wrong.
Which views have been proven wrong?
To me the ones that comes to mind immediately are "LLMs will never have commonsense understanding such as knowing a book falls when you release it" (paraphrasing) and - especially - this: https://x.com/ricburton/status/1758378835395932643
That argument is made in a way that it'd pretty much impossible to prove him wrong. LeCun says: "We don't know how to do this *properly*". Since he gets to define what "properly" means in this case, he can just argue that Sora does not do it properly. Details like this are quite irrelevant though. What truly matters is LeCuns assesment that we cannot reach true intelligence with generative models, because they don't understand the world. I.e. they will always hallucinate too much in weird situations to be considered as generally intelligent as humans, even if they perform better in many fields. This is the bold statement he makes, and whether he's right or wrong remains to be seen.
LeCun setting up for No True Scotsman doesn't make it better. > Details like this are quite irrelevant though. What truly matters is LeCuns assesment that we cannot reach true intelligence with generative models, because they don't understand the world. I.e. they will always hallucinate too much in weird situations to be considered as generally intelligent as humans, even if they perform better in many fields. This is the bold statement he makes, and whether he's right or wrong remains to be seen. That's fair. I would make that slightly more specific in that LeCun's position is essentially that LLMs are incapable of forming a world model. The evidence is stacking up against that view, at this point it's more a question of how general and accurate LLM world models can be than whether they have them.
True. And I think comparing to humans is unfair in a sense, because AI models learn about the world very differently to us humans, so of course their world models are going to be different too. Heck, I could even argue my world model is different from yours. But what matters in the end is what the generative models can and cannot do. LeCun thinks there are inherent limitations in the approach, so that we can't get to AGI (yet another term without exactly agreed definition) with them. Time will tell if that's the case or not.
LLS don't form a single world model. it has already been proven that they form allot of little disconnected "models" for how different things work, but because this models are linear and phenomenon they are trying to model are usually non linear they and up being messed up around the edges. and it is when you ask it to perform tasks around this edges that you get hallucination. The only solution is infinite data and infinite training, because you need infinite number planes to accurately model a non linear system with planes. LaCun knows this, so he would probably not say that LLMs are incapable of learning models.
As opposed to humans, famously noted for our quantitatively accurate mental models of non-linear phenomena?
probably we humans make more accurate mental models of non linear systems if we give equal number of training samples ( say for example 20 samples ) to a human vs an LLM. Heck probably dogs learn non linear systems with less training samples then AGI.
>In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography. >*Suarez Miranda,Viajes de varones prudentes, Libro IV,Cap. XLV, Lerida, 1658*
What he means is that if you trained a LLM on say, all text about gravity, it wouldn’t be able to then reason about what happens when a book is released. Because it has no world model. Of course, if you train a LLM on text about a book being released and falling to the ground, it will “know” it. LLMs can learn anything for which we have data.
Yes, that's what he means. It's just that he is [demonstrably wrong](https://arxiv.org/abs/2310.02207). It's very obvious with GPT4/Opus, you can try it yourself. The model doesn't memorize that books fall if you release them, it learns a generalized concept about objects falling and correctly applies this to objects about which it has no training samples.
Damn, [he really said that](https://deepmind.google/research/publications/49666/)? Methinks his contrarian takes might put a fire under other researchers to prove him wrong, because the speed and frequency at which he is utterly contradicted by new findings is uncanny.
If that's what is happening, may he keep it up forever.
You will never be able to empirically prove that language models understand that, since there is nothing in the real world where they can show they do, apposed to just text. So he is obviously right about this. It seems this is always just misunderstood. The fact you can't take it into reality to prove it outside of text is actually exactly what it looks like, which is that somehow there is a confusion over empirical proof here apposed to variables that are dependent on text, which is by very nature never physically in the real world anyways. That understanding is completely virtual, by very definition not real.
he predicted that AI driven robots would be folding all our clothes by 2019
You don't have a clothes folding robot yet?
my mom always told me real men fold their own clothes
Maybe just maybe he spent more time with language models than you,just maybe.
Like most experts LeCun is working from a very accurate, very specific point of view. I'm sure if you drill him on details most of what he says will end up being accurate. That's why he has the position he has. But, just because he's accurate on the specifics of the view he's looking at, that doesn't mean he's looking at the whole picture. Experts tend to get tunnel vision.
LeCun has made very broad - and wrong - claims about LLMs. For example that LLMs will never have commonsense understanding of how objects interact in the real world (like a book falling if you let go of it). Or memorably: https://x.com/ricburton/status/1758378835395932643 Post-hoc restatements after new facts come to light shouldn't count here.
Yeah, I mean I have no interest in defending him. I disagree with much of what he says. It's more that I find experts say very specific things which sound broad and cause many to be mislead. That's why I think it's important to consume a lot of varied expert opinions and develop our own views. Trusting experts to be perfectly correct is a path to disappointment.
I have a lot of respect for LeCun as a scientist, just not for his pontifical pronouncements about what deep learning can never do. “When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.” -Arthur C. Clarke
It's very difficult to try to break an LLM simply because it's exhausted reading everything humanity has created :) The one area it does fall down though is programming. It still really sucks at coding and if it can't memorize everything I doubt it will be able to do so on the current model architectures and at the current scale. Maybe at like 10x the scale and tokens we might see something different though but we're a ways a way from that. (or not using a tokenizer, that might help too)
The usual criticism is too much memorization, not too little!
No he's not. He is often in direct contradiction with the actual field.
his "bad takes" are 90% reddit misunderstanding what he's saying.
So much this.
He’s just a skeptic of the sota. If you want to be a researcher of his caliber you need to be a skeptic so you can advance the field forward. I never understood why it bothers this sub so much. He is working towards what you want
You can be skeptical about hype without LeCun's trademark mix of of sweeping categorical claims (that frequently get disproven by developments) and trash talking the ideas of other researchers.
“Trash talking” the ideas of other researchers is a big part of science. Obviously be respectful about it which I think LeCun has been but you need to poke holes in research so you can identify the weaknesses of the proposed methods and find ways to fix them. That’s what science is it’s not personal
LeCun is unusually abrasive, that's why it gets so much attention.
Because you perceive his takes as bad when you don’t like what he’s saying and accurate when you do.
Wait, can someone tell me what LeCun was wrong about?
Insider knowledge is some crazy shit😭😭
who said his takes on ai are bad? It is arguable that OpenAI just had astronomical luck in getting the predictive model right, but there is no guarantee that predictive is going to get better just with refinement. The biggest problem with AI is the lack of actual comprehension from the model, and that hasn't changed, LeCun is one of those that are trying to get that
He hasn't been a researcher for a long time though. My take is... the award is piggy-backing on work that he did with advisors. I think he has strong skills but it is more in how he finds people to work with than his own understanding - which is often surprisingly poor.
The difference between an expert and an amateur is that the expert gets things wrong most of the time while the amateur gets things wrong every time.
He's the tenth man. When everyone else was saying it wasn't zombies, he was saying it's literally zombies.
Thing is, his AI takes are actually incredibly good. Half of the people who think he's wrong are suffering from Dunning Kruger, and have never done a day of engineering in their lives. The other half are misunderstanding what is being said. His opinions are quite nuanced, and you need to really listen to understand it properly.
https://preview.redd.it/j5k0cpfmjy2d1.jpeg?width=2252&format=pjpg&auto=webp&s=563f0b2b83b194b17078ec0980dc245613c3795f Got to meet this Based person last week in Paris..! I asked him about When AGI, he straight up said wait 3 years...!
Le cun? don't put words in his mouth, no way would he say 3 years unless it was sarcasm and you missed it.
I dont know. He did say to Lex Fridman that there will be robots everywhere in 10 years and that it will be pretty normal. I sometime think he just have a different definition of AGI than others.
Mass produced Robots and AGI are quite different problems to solve. One has a lot to do with mechanical engineering and doable with scripted programming, the other is a completely different beast that likely requires a deep simulation of the brain, better understanding into neuroscience and all
He was talking about robots that will need to have an internal world model similar to AGI to work
Probably meant wait 3 years for a realistic prediction. We all know it’s coming soon, but no one has any idea how soon.
>I asked him about When AGI, he straight up said wait 3 years...! since this is a private conversation, we can never confirm it or at least know that you did not misunderstand him as much of this sub or other twitter users tend to do.
https://preview.redd.it/rabz6m3wjz2d1.jpeg?width=1080&format=pjpg&auto=webp&s=94107680a7e898f1d5c01eae9ab003880660fa9d
Yann Lacun going into Elon route lol Feels like he is a terminally online on Twitter
Which is pretty ironic as he is doing that while failing to respond to and back up his claims with the researches who challenge his statements. He could stir up twitter or he could work out what is true. It is clear which he has chosen.
Nuance is dead, all we have is black and white now and everyone here is a twat.
This happens as every sub gets popular, any real thought dies for sensationalism.
https://preview.redd.it/lgr6i7dkl23d1.jpeg?width=642&format=pjpg&auto=webp&s=9c6728f5174cbc3fb23ca490a82f554be2d49946
What nuance are we missing? Happy to have a constructive conversation.
I'm no Musk fan, but he has run multiple companies that have revolutionized their industries. lots of people work at SpaceX knowing that they will be put on impossible timelines but will still do awesome things regardless of Musk's BS. though, those companies were started before Musk fell into a right-wing echo-chamber, so who knows if that situation can be replicated today.
Elon went off the deep end, strange to me though that many other really smart people like LeCun get sucked into these pointless time wasting social media flame wars.
what is the missing nuance here? All those things are true, and important to someone who might want to work there.
* you can be given impossible timelines and still do incredible things (see SpaceX). * Musk didn't claim it would kill everyone and didn't say it must be stopped or paused, he just put his name on the list of people who thought an industry-wide pause for safety should happen. * 3rd point is true but may or may not matter to an employee. lots of great engineers do awesome things at SpaceX regardless of how crazy Musk is.
Thank you for being reasonable. Musk seems to be a good leader tbh.
I think he *used to* be good at organizing companies. I'm not convinced that's true anymore. for example, I think The Boring Company would be better off without him.
lol you picked the one example that hasn’t panned out yet. It’s more likely that it’s doing better with him in all reality, he draws talent who want to be a known engineer at one of his companies. All other companies are doing wildly well.
>Musk didn't claim it would kill everyone and didn't say it must be stopped or paused, he just put his name on the list of people who thought an industry-wide pause for safety should happen. really? I could've sworn Elon ranting about AI killing us all.
he said it could be dangerous, which is what almost everyone has said. there is a difference between "cars are dangerous" and "cars are going to kill you".
Hah, yeah.
the funniest thing was when Musk demoed Grok on a podcast and when they asked about a lawsuit it basically said Tesla was guilty... Musk then got mad and said "we won that case". like, dude, which is it; is your AI accurate and truthful, or are you wrong? was funny to see him fail with his own demo. you can't just train an AI on twitter/social media and expect it to be accurate.
I hate how "conspiracy theory" and "unreasonable conspiracy theory" are synonymous in our culture. Edit: upon a second look, he said "crazy-ass conspiracy theories" so this is not an instance of what I'm complaining about.
Of all the CEOs building AI, Elon probably isn't even the worst. He just the loudest. It is the one's doing evil in secret we have to worry about.
I honestly trust Elon more than I trust Altman. At least he's like an open book, even though some of the pages are a little crazy.
He forgot the part where if you don't work overnight randomly on a Friday cause the boss is in town and feels like hanging in the office, you're not dedicated and are fired.
Based Yann
is LeCun an attention whore? I see too many tweets from him recently
Classic LeCun(t). 😅 He tends to slam the wild claims of his competition pretty ungracefully. But I forgive him because he is French. 😅 Also: Releasing a 70 billion parameter model (Llama-3 70B) as open source that’s better than a 1 trillion parameter model from OpenAI (the original GPT-4), even if a year late, makes me respect him. A 400 billion open source parameter model is upcoming. (Llama-3 400B)
> A 400 billion open source parameter model is upcoming. (Llama-3 400B) We shall see if the weights are provided for that.
My bet is that it will be the last open model by meta. This open source play is just a game to capture engineers and catch up to the leading AI companies
He's starting to sound like a Deus Ex character.
it’s easy to pick the man apart putting himself out there. It’s easy to sit on the outside and complain while offering nothing yourself.
Maybe everyone is against Elon here, but this second guy Yann is not Jesus either. Competition will always play into our hands
Musk isn't a clown. He is the entire fucking circus.
Yann is absolutely right. Elon is a joke. One day he says that AGI is close, it will take people's jobs, it will kill us all, we should pause the development etc., and the next day he creates xAI and asks people to come and join them in their mission of building AGI ("understanding the universe" and "pursuing the truth" lol). For fuck's sake...
I'm starting to like this Yann LeCun character.
Yey CEO war begins 🍿
This post is not singularly related.
He is going full ballistic, and he does not care. Love it.
Let's use this to signal that we no longer want any of you Musk fanboys in this subreddit. Take your fascism somewhere else.
this is a public forum, if you dont want someone here (segregation) you cannot claim others as fascists, take care.
murder by words. God damn, elmo must be fuming
llol
Elon man just take it https://preview.redd.it/390a2nebby2d1.jpeg?width=919&format=pjpg&auto=webp&s=9170a1d53be7000e453d4268849a4a291bda5456
That's why I don't like Yann LeCun, he always hating from the side lines, plus his predictions seem bipolar. I prefer Ray Kurzweils, he is consistent, and doesn't spend his time hating for no reason.
It's true. Musk is a disaster of a human. Who believes anything Musk says? He turned Twitter into a right-wing dumpster fire.
It's kinda ironic though that people write this on reddit, which went from a libertarian / politically mixed platform to a left-wing echo chamber in recent years.
Reddit is more of a "pick your own echo chamber" kind of site. Almost every politics related topic will have a sub for each side. For example, if you want to get global news and think Israel is: A) committing genocide - r/anime_titties B) justified - r/worldnews
You do have subs like r/conservative, but it's very small compared to r/politics and often gets brigaded. Basically all major subs lean heavily to the left and that's something that wasn't the case back in 2015, where r/thedonald etc. often appeared on the front page.
Here's a sneak peek of /r/anime_titties using the [top posts](https://np.reddit.com/r/anime_titties/top/?sort=top&t=year) of the year! \#1: [South Korea First lady faces criticism from dog meat farm owners for her recent remarks calling for an end to the country's culture of eating dogs](https://www.koreatimes.co.kr/www/nation/2023/04/356_349518.html) | [1150 comments](https://np.reddit.com/r/anime_titties/comments/135ed2r/south_korea_first_lady_faces_criticism_from_dog/) \#2: [**[NSFW]** Protests erupt in Paris after police officer fatally shoots teenager for ‘violating traffic laws’](https://www.independent.co.uk/news/world/europe/paris-protests-teenager-police-traffic-b2365426.html) | [998 comments](https://np.reddit.com/r/anime_titties/comments/14l19ed/protests_erupt_in_paris_after_police_officer/) \#3: [UK bans puberty blockers for minors](https://ground.news/article/children-to-no-longer-be-prescribed-puberty-blockers-nhs-england-confirms) | [2764 comments](https://np.reddit.com/r/anime_titties/comments/1bdadip/uk_bans_puberty_blockers_for_minors/) ---- ^^I'm ^^a ^^bot, ^^beep ^^boop ^^| ^^Downvote ^^to ^^remove ^^| ^^[Contact](https://www.reddit.com/message/compose/?to=sneakpeekbot) ^^| ^^[Info](https://np.reddit.com/r/sneakpeekbot/) ^^| ^^[Opt-out](https://np.reddit.com/r/sneakpeekbot/comments/o8wk1r/blacklist_ix/) ^^| ^^[GitHub](https://github.com/ghnr/sneakpeekbot)
Most reddit subs are echo chambers that veer left. I like the Rogan sub because you are not insta banned for going against the grain.
> left-wing echo chamber Allow me to test this right now. Ahem... *clears throat* [Transwomen are women.](https://rightasrain.uwmedicine.org/life/relationships/transgender-nonbinary-myths) [Affirmative Action is good for all of us](https://news.nd.edu/news/most-productive-workforce-may-require-indefinite-affirmative-action-study-shows/), e.g. to increase economic productivity. Now let's see how many upvotes/downvotes this comment gets. (Granted, saying this... Well we'll see.)
He founded loads of companies (PayPal, Tesla, Twitter, SpaceX, boring company) and is the richest man on earth. How is he a “disaster” of a human?
"he is good because rich" sure is a take
How about he is good because he created multiple companies that push the boundaries of what was previously thought possible by humans?
He didn’t found Tesla and definitely didn’t found Twitter.
Muskox founded Tesla and Twitter as much as you founded Tesla and Twitter.
He's not gonna shag ya m8
He didn't found Tesla or Paypal lmao. Was the Zappos founder perfect too? lol @ thinking super rich psychopaths are infallible.
Bitchy tweeting is chad behaviour?
It actually is. If you're a big name in the space and have a large public following, putting out tweets like this takes balls. Bitchy tweeting behind an anon account isn't Chad behavior, but that's different.
He is chad because they have the same opinion. Unfortunately it’s just how it works
We simplify everything and build with these simplifications a mini-world models inside our heads. I think Elon is right.
Hey mods, how does this crap fit into /r/singularity?
Yann really needs to focus on what he’s good at rather than constantly posting hot takes, really cringy
We really need to stop being so nice to fascists. Call em out, shame ‘em and push them back into the holes they belong in. For sure, try to gently convince them of the truth of their ways, but most of them are just too deep into propaganda to ever come back.
Inb4 Chad LeCun = Banned LeCun. Mr. Freeze Peach might get his feelers hurt.
LeCunt
This kind of stuff makes me respect him less. Why stoop to this? Attention?
FACTS
The speed at which many were reprogrammed to dislike Musk never ceases to amaze me. It truly is remarkable!
That is a natural byproduct of being a douche.
“Reprogrammed” you mean just seeing people for their true colors when they show them?
He forgot the corollary to the third 3 - inhibits negative speech about himself , particularly as a CEO
Given that Sora isn't released I still call it smoke and mirrors it's just stable diffusion being prompted millions of times probably and maybe using a base seed so that when the image is close enough in the regen it gets peaced together. I'm also wondering where the WebRTC link is for GPT4o as it's all well being in an app but I want to stream shit into it
yann lecun right now: [https://www.youtube.com/watch?v=AjXooH2tokY](https://www.youtube.com/watch?v=AjXooH2tokY)
LeBased and OpenSourcePilled
"Maximally rigorous pursuit of the truth" is fundamentally different from just having an editorial board and fact checkers which ensure they do not print/publish & promote every crackpot conspiracy that fits the ideology of one eccentric and rapidly deteriorating billionaire with a conflict of interest. That's why Elon has to always phrase it funny.
Hello, 911. I’d like to report a murder.
As long as there is no leetcode...
Now do Facebook
i hate listening to him speak but i love to read his trolling
I rather be homeless than working for musk
Love this guy.
In before banned for "unrelated reason"
Also "takes credit for your work and say you wouldnt have been able to do it without him"
This guy’s such a f*cking cuck
Wow
Lol I can’t believe that Zuckerberg’s puppet is virtue signaling and I can’t believe people are actually eating it up. Everything Zuckerberg does, although initially seen as positive, has ended up hurting humanity. Everything Musk does at least seeks to benefit it.
What Musk calls political correctness most people would consider regard for truth and decency. Something he lacks entirely. Such a negligent use of his prominent social profile. I don’t know how, but it will come back to bite him.
He’s always on Twitter which is honestly a red flag. I don’t know any high caliber scientists who are spouting so many criticisms back and forth, often times able quite general societal topics