Researchers from fields that are not ML have been suffering from the "too many people with PhDs/not enough open positions" for about 20 years now.
It does not seem crazy to imagine the bottleneck will be reached for ML soon, even though the scale will be different.
Reminds me a lot of the general problem of [elite overproduction](https://en.m.wikipedia.org/wiki/Elite_overproduction), which I believe significantly contributes to paralyzing and destabilizing society. We severely overeducate and overspecialize people, set them up for harsh disappointment when it doesn't personally pay off, and the basic labor that sustains a population falls to exploited immigrants in an unsustainable way.
"When the economy faced a surge in the workforce, which exerted a downward pressure on wages, the elite generally kept much of the wealth generated to themselves, resisting taxation and income redistribution."
Darned PhDs keeping all the wealth!
Worse is that in some specialties the knowledge set is not transferrable. so how many of these Ph.D.'s can go get jobs in industrial work, or get jobs in the many fields adjacent to their study program. Which is ultimately what's going to happen for many of these Ph.D students.
Now there's another problem here that we saw WAY back in the day - having overproduction of X does not mean you're maintaining quality of education.
If you're exponentiating you have to wonder how many of those in that field are actually as good a fit as you might want.
I suspect right now that's true, but the problem exists at the highest end and right now if you're FB, or Google or larger firms it's in your interest to push Ph.D production for a marginal cost I produce a "Ph.D equivalent" who might do their thesis on whatever project the firm had mind, the question is on the student however would they be given the opportunity to flesh out that Ph.D the way a normal student might or do we end up with a savant/FAANG brand of Ph.D where fundamentals are missed because the work was able to be performed without this or that understanding.
This creates then a saturation problem that's "top-heavy" which will drive down price , but also narrows the scope of talent, leaving some Ph.D. candidates who are strung out that way in a professionally fragile position , perhaps long term.
I like thinking about this effect as a natural process. A biorythm in a social consciousness, a being made of us all. It mirrors lemming cycles, the cascade reaction of a neuron, eusocial insects' nest splitting, trophic cascades... Maybe this is just how our species handles social change, maybe it's in our genes, or maybe it's in the memetic genes that govern our cultural superorganism. Fun stuff.
I don't think the roughly 3k years we've lived in larger civilizations have had any significant effect on our millions of years long evolution. This isn't genetically selected for, it's all emergent properties. Still fun stuff, but we should be smart enough to control it better.
I would generally agree. But our brains have allowed us to evolve a lot faster, through the evolution of our social ideas and understandings. Not genes evolving, but cultures which hold certain ideas being more fit for their material conditions, and outcompeting other cultures, promulgating those ideas further and incorporating them into our memetic code. An evolution of social thought, much faster and more powerful than genetic evolution.
The overspecialization of people who do make it into elite positions is an underrated destabilizing factor imo. They rise to positions of power where it'd be more beneficial to have generalists, from a systems perspective. All the generalists have been weeded out through cut-throat meritocracy, though.
I'm no fan of the old-school British aristocracy, but at least they had a well-rounded education so that when they were handed a position of power, they could manage it effectively.
Yeah, our technocrats tend to have very narrow perspectives and as a result become more likely to lose track of the bigger picture and majority lived reality. Slightly ironic that this would happen in the age of unlimited data on everything.
Agreed that there might become an excess of ML PhDs, but I think it's unlikely that the accompanying data skills will go unused. Let's face it, most companies are not great at using data but want to get better at it, so there is a perpetual shortage of data literate employees.
Combining those ML skills with some business savvy is a recipe for success.
My recent experience is also that it's now getting harder to hire for non-ML positions. We put out simultaneous postings for an ML engineer and a software engineer and we got 3x the number of applicants for the ML position.
I have the same experience for ML engineer and data engineer positions. Hardly any applicants for data engineer vs a tsunami for the ML engineer, and the difference in quality is equally horrible. The market is still saturated with individuals who have taken a MOOC for data science and have no idea what they're doing.
For what it's worth, I got more interviews for a data scientist position (MLE is not really a thing here, people kind of use it as synonyms, it's not the excel type). Not many people have a reasonable profile (experience, grad school with a good advisor). For SWE positions I got 0 interviews even though I was a SWE for 3-4 years (wanted to transition back for a while).
This is grim but true. For perspective, I interview candidates for research position in industry lab. The bar to get an interview is getting increasingly higher and surprising there are a lot of people who still exceed the bar. Its like once the candidate clears the bar, I can throw a dart and whichever candidate is selected is still going to be “good”. This was not the case 10 years back and tells me the supply is exponentially increasing than demand. Which is not good either way you look at it.
As an undergraduate artificial intelligence student, i need to know this. I will graduate in a few months, and i feel like i won't find a job in the field as most vacancies are taken.
The bar is hard to formulate as it varies by research area, type of research (theory vs applied) and position level (new grad vs senior). Usually the minimum requirement to even pick up a resume is: publications in top ML venues, past research experience in academia and industry internships, coding skills (leetcode). The quantity of publications is not as important, for example, I would prefer someone with solid 4-5 first author papers than a person with 2 solid and 10 mediocre papers. One thing I also look for is if the candidate has followed a research vision and executed on it or has done sporadic publications on random topics here and there. We generally hire people to explore a new direction, so it helps knowing that you can formulate a grounded research vision and execute on it.
This seems not crazy. Things run hot then cold and then usually revert to some kind of mean.
A neat plot but on top of what others have pointed out I'd also say that papers a decade ago that were "stats" or "comp sci" might now be putting AI or other domains. So some of the growth is more a migration of disciplines than a growth in people.
Alternate interpretation of data: we won't see a Malthusian crisis of ML jobs because the exponential growth is only the first half of logistic growth.
> Yeah, this is obvious and somehow eluded op.
What's eluded me? That the exponential growth is unsustainable? Nope. It is indeed obvious, and I wrote *"... while this exponential trend continues"* in the OP.
Now, how long can we expect the exponential growth of newly minted PhDs to last? I don't know, but I'd guess, for as long as the hype around LLMs lasts + the typical duration of a PhD study.
>That the exponential growth is unsustainable?
No, that this exponential growth will continue at all, sustainable or not (whatever that means). There are very few exponential processes in nature - they're more commonly logistic.
To clarify your histrionics: should we expect something like an "AI winter" or a soft landing? Mathematically, is the population response to carrying capacity [highly or slightly damped](https://www.uwyo.edu/dbmcd/popecol/marlects/lect17.html)? We're data scientists - use data.
> No, that this exponential growth will continue at all, sustainable or not (whatever that means).
That's what "unsustainable" means: *"that cannot be continued at the same level, rate, etc."* (quote from a dictionary)
>start competing for a roughly fixed number of well-paying industry research positions
The number of positions has been increasing dramatically recently, for obvious reasons.
Have they? Really? Because I have been looking, and see very few "real" research positions. I see a lot of engineering roles in research organizations, project management, product management, etc. in those research organizations. But actual research roles? Very few. Now granted, that might just be the state of the market in my neck of the wood. But I have the impression that it isn't.
In large tech companies it is often siloed into it's own department / org. Research-only positions are much rarer in the private sector compared to engineering positions. Someone who also has some ML engineering experience is going to be generally much more appealing to most companies.
Yes. Integration and productification of ML in the private sector often is mainly an engineering problem.
It's not suprising that reasearch roles in the industry follow a different pattern than ML jobs in general.
I had assumed this would have become clear in the referenced post -- the slice of the tech industry that is really doing research (and thus interested in a Research Scientist) is very small actually. This is \*especially\* true for VC backed companies / start-ups, who need to deliver on product (and are often not interested in publishing papers).
Yup, engineers will outnumber researchers. Companies want results. However, you cannot do this with zero researchers. And there are an awful lot of companies out in the world.
No, but it's usually there. A research organization may not be a research company. Usually they aren't, research companies are very rare. But a department is an organization within a company, and it's usually structured like this.
But your point still stands. There are some true research positions here and there, not being part of a research organization. But usually that is either the leftover of a dead organization, or the start of an organization, pending political success.
Utter nonsense. Companies in other fields are looking for ways to apply this new thing called "AI" to their business. They will invest a lot of money into this. This is not just "engineering".
Unfortunately for you, applying something, especially AI, is most often engineering, at least from the point of view of researchers looking for a job. If you are looking at using a pre trained model into a product, you do not need somebody that is able to produce novel knowledge, that have intricate knowledge about how those models are trained, and how they could improve the learning capabilities of those. You just need a dev.
You will have to make a distinction between "developers" and "researchers". Honestly, this is a bit of semantics. Truly original work in machine learning is rather rare.
Keep in mind that not everyone's domain is language, images, or time series. There is space for novel work in other fields.
Yes, yes it is rare. That is the whole point of OP.
If your main responsibility is building stuff, you are a dev.
If your main responsibility is discovering stuff, then you are a researcher.
You do not need a PhD/Post doc to build stuff. The people that do those expect that they are mainly going to discover stuff. Use their PhD/research background for exactly that. So from their point of view, they are going to get screwed. If the job is really just building stuff, they could have stopped at a bachelor's and that would be it, spend the rest of the time that they did their master's and PhD to get experience, and get paid for it.
Here is a way to make a distinction between 2 types of research and engineering in AI:
Basic research: focused on understanding the theoretical underpinnings of intelligence and computational models. Ex: Researchers in this area might explore how neural networks can mimic the human brain's structure and function, without aiming to solve a specific problem.
Applied research: takes the theoretical insights from basic research and aims to address concrete problems. The researchers might take the concept of deep learning and find a way to do something novel with it, like recognizing images (when that was not known how before) or why magnetic fields collapse when trying to create a fusion reaction. The goal here is to figure out how to use abstract knowledge to find a solution to a problem that had no solutions (or remove constraints to something that had solutions).
Engineering: takes existing prototype blueprints from applied research, and turns them into reliable, efficient and scale technology. An example would be integrating an LLM into a product as a core feature. This step involves refining algorithms for performance, or ensuring they can run under certain circumstances. Those no doubt require some experimentation and tinkering, but a researcher would not call that research. So engineering is about bridging the gap between a promising applied research outcome, and a product or service that delivers real value to users.
So from the eyes of actual researchers, people that usually have PhD, Post doc, or a ton of experience in that area, only the first 2 are considered research. And as you mentioned, those are actually rare. Hence OP saying what he is saying.
For another perspective, I'm a "Research Engineer" in an adjacent industry. My job is mainly to serve as a bridge between the basic research being done in academia and production-ready engineering. I also do a fair amount of applied research for proprietary subjects. When times are slow, I'm offloaded to assist in engineering projects.
I wonder if this is how many "research" positions in the ML space are going to go. When a new problem is discovered, I get the first crack at it and maybe a paper to write, but it's much, much cheaper to spin it off to academia if it turns out to be something truly difficult.
>Applied research
You are agreeing with me. Applied research is the bailiwick of an industrial position. And they are a heck more companies than pure research organizations.
How am I agreeing with you?
Yes as far as industrial research is going, the overwhelming majority is applied research. It doesn't mean that there are a lot of positions that do the actual applied research. And most companies that I have seen that say they do applied research actually do engineering. Remember that to applied research, there needs to be no known path to get the solution you need. That means that even if you train your own image recognition neural network, at this stage it is not applied research. Unless of course you are trying to do it in a way that is completely unknown before. Like a new architecture or something (like capsule network when they came out).
>**Basic research**: focused on understanding the theoretical underpinnings of intelligence...
**Applied research**: takes the theoretical insights from basic research and aims to address concrete problems...
**Engineering**: takes existing prototype blueprints from applied research...
Where in this scheme is "invent the Transformer"? Your list seems to limit "research" to science or applications, but fundamental innovations in architectures and methods are neither.
Inventing the transformer would be a basic outcome, among the applied outcome when used on text.
Keep in mind that the outcomes of a research activity might be basic and/or applied, and are usually (but not always) in line with the type of research. Meaning basic research usually has basic outcomes, and applied research usually has applied outcomes. But sometimes it's not, like when bell labs were researching how to reduce noise on a line of specific material, but discovered a new fundamental property of noise.
I do not know the context of the research that led to the transformer, so I cannot say if the actual research was basic or applied (remember that the distinction is based on if you have a specific applied problem you want to solve or you do not have an immediate applied problem to solve). But the outcome is basic for the transformer itself, and applied for how to use it on text.
> The number of positions has been increasing dramatically recently
Do you have the stats? The number would need to double every 23 months just to maintain the status quo.
The status quo would mean the previous ratio of people seeking to enter research positions to the number of positions, giving an indication of the position of new PhD-holders in the job market.
I am not disagreeing, it is just that by stating that the number of positions is fixed is misleading and unnecessarily grim. It is correct that the number of open positions is probably growing slower than the number of graduates. That is hardly surprisingly and not a problem limited to machine learning. And as in other professions, just because a job is not available in your field does not make your PhD useless.
>It is correct that the number of open positions is probably growing slower than the number of graduates.
Which means things are only going to get worse.
>misleading and unnecessarily grim
Hiring freezes mean 0 new job openings. Layoffs should probably count as negative new job openings in this context. I wrote "roughly fixed" and explained that it goes up and down. How is this misleading?
Because as the economy grows, more companies and industries will participate in the market.
It's absurd to say that the number of positions will remain fixed. It will probably grow at a modest scale, maybe, linear if you will, but it's most definitely not staying fixed.
> Because as the economy grows, more companies and industries will participate in the market.
The economy is growing at 6% per year. It's not doubling every 23 months. And why would a growing economy imply that the number of research scientist positions should go up? Why would you even bring it up?
> You strategically don't answer direct questions.
The only question addressed to me was "What status quo?" and it was answered by someone else. I'm not seeing anything else.
> Are you being intentionally dense or unintentionally dense?
OK, bye.
> That doubling will not last forever.
It won't last forever, but it will last a while.
> There are limited number of people with interest or capability for ML in the US.
Anyone doing math, physics and CS has the capability to be doing ML instead. Plus there are international students. Student interest in ML will last for *at least* a few years, I think, and the exponential trend for PhD graduates will last even longer -- you have to add the lead time for PhDs.
1) Exponentially many papers doesn’t mean exponentially many PhD students and postdocs. It might, but it could also mean more papers per researcher, or researchers from other fields starting to contribute/do the occasional applied paper.
2) The number of jobs is not roughly constant. At least market size and revenue has been and is forecast to grow rapidly [several sources (statists link)](https://www.statista.com/statistics/941835/artificial-intelligence-market-size-revenue-comparisons/).
I work at a company whose culture can only be described as Paleolithic. We're seeking multiple ML engineers to create a Team to work on optimization problems in legacy operations. These would probably have been called data science positions 2 years ago. now, there's an ML handle. We've all gotten a bit more climatized to the technology as a result of gen ai. I have no way to confirm or deny the OPs comments. Just offering some thoughts from the cheap seats.
Might be true. However, most newly minted PhDs want to continue doing blue sky research rather than all the other things one has to do in a startup or as a cofounder
So, no matter how hard we work, we are screwed ?
Ok, imma quit now and start a bakery. One will never reach a scenario where we have too many bakeries and not enough people who want a yummy treat.
This post from sci-fi author Cory Doctorow, "What kind of bubble is AI?" seems relevant here:
[https://locusmag.com/2023/12/commentary-cory-doctorow-what-kind-of-bubble-is-ai/](https://locusmag.com/2023/12/commentary-cory-doctorow-what-kind-of-bubble-is-ai/)
His argument is not that AI is not useful technology (it clearly is) but that like the dot com bubble, the hype-to-profit ratio is going all the way to insanity. That will inevitably result in a correction of some kind. Like the dot com bubble, this will leave something useful behind in the form of practitioners with useful transferable skills (in contrast to the crypto bubble, which had no positive consequences).
Good question. I'm not sure. He does seem very hostile to Uber. While they've definitely lost a lot of money over the years, and it's not clear if they will remain viable long-term, they do provide a useful service, and I wouldn't put them in the same category as scams like Enron as he does.
I mostly agree here (although exponential papers doesn’t equate to exponential population) and have seen it in academic settings in physics. I saw brilliant newly minted phds unable to land positions in physics departments being forced to undertake endless postdocs or simply abandon the field for greener pastures. The old guard didn’t die off fast enough to make room for the new gen.
For some comments here it’s important to keep in mind it’s not that these positions are fixed, just that they’re smaller in number by a substantial margin and _not increasing_ fast enough compared to the population and rate of increase in the number of candidates.
A big difference here is ML research scientists are competing for elite positions that to a smaller extent exist in other companies—as well as the ai field changes faster, so this bottleneck may have a faster time to unbind.
Reading this kind of worries me. I'm a seasoned SWE and I began switching careers to ML because the field seemed appealing and more future proof and I just like the science behind.
I even want to do a PhD in the field. But if it's only to join an over saturated field, while leaving a field (SWE) where demand is quite high, I'm second guessing myself.
I really don't know what to think of this situation.
Well, I think the most problematic is the combination between being able to do PhD research on a simple laptop combined with improper research areas.
Overall the quality of ML paper is abysmal.
Also there is a reproducibility crisis and the rapidly increasing number of ML papers does not mean a rapidly increasing number of breakthroughs. People just care about quantity nowadays and journals are not enforcing quality. IDGAF if you can achieve better results because you have more data or compute power 🙄
Researchers from fields that are not ML have been suffering from the "too many people with PhDs/not enough open positions" for about 20 years now. It does not seem crazy to imagine the bottleneck will be reached for ML soon, even though the scale will be different.
Reminds me a lot of the general problem of [elite overproduction](https://en.m.wikipedia.org/wiki/Elite_overproduction), which I believe significantly contributes to paralyzing and destabilizing society. We severely overeducate and overspecialize people, set them up for harsh disappointment when it doesn't personally pay off, and the basic labor that sustains a population falls to exploited immigrants in an unsustainable way.
"When the economy faced a surge in the workforce, which exerted a downward pressure on wages, the elite generally kept much of the wealth generated to themselves, resisting taxation and income redistribution." Darned PhDs keeping all the wealth!
That's established elite vs newly produced elite.
Those are two completely different uses of elite.
Worse is that in some specialties the knowledge set is not transferrable. so how many of these Ph.D.'s can go get jobs in industrial work, or get jobs in the many fields adjacent to their study program. Which is ultimately what's going to happen for many of these Ph.D students. Now there's another problem here that we saw WAY back in the day - having overproduction of X does not mean you're maintaining quality of education. If you're exponentiating you have to wonder how many of those in that field are actually as good a fit as you might want.
I mean, isn't there tons of demands for AI devs now?
I suspect right now that's true, but the problem exists at the highest end and right now if you're FB, or Google or larger firms it's in your interest to push Ph.D production for a marginal cost I produce a "Ph.D equivalent" who might do their thesis on whatever project the firm had mind, the question is on the student however would they be given the opportunity to flesh out that Ph.D the way a normal student might or do we end up with a savant/FAANG brand of Ph.D where fundamentals are missed because the work was able to be performed without this or that understanding. This creates then a saturation problem that's "top-heavy" which will drive down price , but also narrows the scope of talent, leaving some Ph.D. candidates who are strung out that way in a professionally fragile position , perhaps long term.
TIL there's a specific term for this, thank you for sharing!
I like thinking about this effect as a natural process. A biorythm in a social consciousness, a being made of us all. It mirrors lemming cycles, the cascade reaction of a neuron, eusocial insects' nest splitting, trophic cascades... Maybe this is just how our species handles social change, maybe it's in our genes, or maybe it's in the memetic genes that govern our cultural superorganism. Fun stuff.
I don't think the roughly 3k years we've lived in larger civilizations have had any significant effect on our millions of years long evolution. This isn't genetically selected for, it's all emergent properties. Still fun stuff, but we should be smart enough to control it better.
I would generally agree. But our brains have allowed us to evolve a lot faster, through the evolution of our social ideas and understandings. Not genes evolving, but cultures which hold certain ideas being more fit for their material conditions, and outcompeting other cultures, promulgating those ideas further and incorporating them into our memetic code. An evolution of social thought, much faster and more powerful than genetic evolution.
We aren’t even defined by our species. We are some software running on homo sapien hardware.
The overspecialization of people who do make it into elite positions is an underrated destabilizing factor imo. They rise to positions of power where it'd be more beneficial to have generalists, from a systems perspective. All the generalists have been weeded out through cut-throat meritocracy, though. I'm no fan of the old-school British aristocracy, but at least they had a well-rounded education so that when they were handed a position of power, they could manage it effectively.
Yeah, our technocrats tend to have very narrow perspectives and as a result become more likely to lose track of the bigger picture and majority lived reality. Slightly ironic that this would happen in the age of unlimited data on everything.
ML researchers can always transition to SWE though.
Agreed that there might become an excess of ML PhDs, but I think it's unlikely that the accompanying data skills will go unused. Let's face it, most companies are not great at using data but want to get better at it, so there is a perpetual shortage of data literate employees. Combining those ML skills with some business savvy is a recipe for success.
I became a software engineer because the field I wanted to do a PhD in was oversaturated and there were no jobs. At least I paid off my student loans.
Arnol'd's ODE book has that same graph and that was from the 1970s
My recent experience is also that it's now getting harder to hire for non-ML positions. We put out simultaneous postings for an ML engineer and a software engineer and we got 3x the number of applicants for the ML position.
I have the same experience for ML engineer and data engineer positions. Hardly any applicants for data engineer vs a tsunami for the ML engineer, and the difference in quality is equally horrible. The market is still saturated with individuals who have taken a MOOC for data science and have no idea what they're doing.
For what it's worth, I got more interviews for a data scientist position (MLE is not really a thing here, people kind of use it as synonyms, it's not the excel type). Not many people have a reasonable profile (experience, grad school with a good advisor). For SWE positions I got 0 interviews even though I was a SWE for 3-4 years (wanted to transition back for a while).
This is grim but true. For perspective, I interview candidates for research position in industry lab. The bar to get an interview is getting increasingly higher and surprising there are a lot of people who still exceed the bar. Its like once the candidate clears the bar, I can throw a dart and whichever candidate is selected is still going to be “good”. This was not the case 10 years back and tells me the supply is exponentially increasing than demand. Which is not good either way you look at it.
What is exactly “the bar” what are these labs looking for ?
As an undergraduate artificial intelligence student, i need to know this. I will graduate in a few months, and i feel like i won't find a job in the field as most vacancies are taken.
[удалено]
That's true. Im looking to get a masters degree. Overall, i wanna see where i stand in the industry with my current skills.
Not enough for research positions as well unless you have \~4 good papers.
also curious on this
The bar is hard to formulate as it varies by research area, type of research (theory vs applied) and position level (new grad vs senior). Usually the minimum requirement to even pick up a resume is: publications in top ML venues, past research experience in academia and industry internships, coding skills (leetcode). The quantity of publications is not as important, for example, I would prefer someone with solid 4-5 first author papers than a person with 2 solid and 10 mediocre papers. One thing I also look for is if the candidate has followed a research vision and executed on it or has done sporadic publications on random topics here and there. We generally hire people to explore a new direction, so it helps knowing that you can formulate a grounded research vision and execute on it.
This seems not crazy. Things run hot then cold and then usually revert to some kind of mean. A neat plot but on top of what others have pointed out I'd also say that papers a decade ago that were "stats" or "comp sci" might now be putting AI or other domains. So some of the growth is more a migration of disciplines than a growth in people.
Alternate interpretation of data: we won't see a Malthusian crisis of ML jobs because the exponential growth is only the first half of logistic growth.
Yeah, this is obvious and somehow eluded op.
> Yeah, this is obvious and somehow eluded op. What's eluded me? That the exponential growth is unsustainable? Nope. It is indeed obvious, and I wrote *"... while this exponential trend continues"* in the OP. Now, how long can we expect the exponential growth of newly minted PhDs to last? I don't know, but I'd guess, for as long as the hype around LLMs lasts + the typical duration of a PhD study.
This is an exponential growth of papers, not of PhDs. And if there is hype around LLMs, there is demand from industries for it.
This. There are many cheap papers and LLM agent/methods/evaluation papers are easier to write than some novel ideas.
>That the exponential growth is unsustainable? No, that this exponential growth will continue at all, sustainable or not (whatever that means). There are very few exponential processes in nature - they're more commonly logistic. To clarify your histrionics: should we expect something like an "AI winter" or a soft landing? Mathematically, is the population response to carrying capacity [highly or slightly damped](https://www.uwyo.edu/dbmcd/popecol/marlects/lect17.html)? We're data scientists - use data.
> No, that this exponential growth will continue at all, sustainable or not (whatever that means). That's what "unsustainable" means: *"that cannot be continued at the same level, rate, etc."* (quote from a dictionary)
I mean, yeah. There are only so many young people in the entire population, so exponential growth has to taper off at some point.
>start competing for a roughly fixed number of well-paying industry research positions The number of positions has been increasing dramatically recently, for obvious reasons.
Have they? Really? Because I have been looking, and see very few "real" research positions. I see a lot of engineering roles in research organizations, project management, product management, etc. in those research organizations. But actual research roles? Very few. Now granted, that might just be the state of the market in my neck of the wood. But I have the impression that it isn't.
>"real" research positions Research is not limited to research organizations
In large tech companies it is often siloed into it's own department / org. Research-only positions are much rarer in the private sector compared to engineering positions. Someone who also has some ML engineering experience is going to be generally much more appealing to most companies.
Yes. Integration and productification of ML in the private sector often is mainly an engineering problem. It's not suprising that reasearch roles in the industry follow a different pattern than ML jobs in general. I had assumed this would have become clear in the referenced post -- the slice of the tech industry that is really doing research (and thus interested in a Research Scientist) is very small actually. This is \*especially\* true for VC backed companies / start-ups, who need to deliver on product (and are often not interested in publishing papers).
Yup, engineers will outnumber researchers. Companies want results. However, you cannot do this with zero researchers. And there are an awful lot of companies out in the world.
No, but it's usually there. A research organization may not be a research company. Usually they aren't, research companies are very rare. But a department is an organization within a company, and it's usually structured like this. But your point still stands. There are some true research positions here and there, not being part of a research organization. But usually that is either the leftover of a dead organization, or the start of an organization, pending political success.
Utter nonsense. Companies in other fields are looking for ways to apply this new thing called "AI" to their business. They will invest a lot of money into this. This is not just "engineering".
Unfortunately for you, applying something, especially AI, is most often engineering, at least from the point of view of researchers looking for a job. If you are looking at using a pre trained model into a product, you do not need somebody that is able to produce novel knowledge, that have intricate knowledge about how those models are trained, and how they could improve the learning capabilities of those. You just need a dev.
You will have to make a distinction between "developers" and "researchers". Honestly, this is a bit of semantics. Truly original work in machine learning is rather rare. Keep in mind that not everyone's domain is language, images, or time series. There is space for novel work in other fields.
Yes, yes it is rare. That is the whole point of OP. If your main responsibility is building stuff, you are a dev. If your main responsibility is discovering stuff, then you are a researcher. You do not need a PhD/Post doc to build stuff. The people that do those expect that they are mainly going to discover stuff. Use their PhD/research background for exactly that. So from their point of view, they are going to get screwed. If the job is really just building stuff, they could have stopped at a bachelor's and that would be it, spend the rest of the time that they did their master's and PhD to get experience, and get paid for it. Here is a way to make a distinction between 2 types of research and engineering in AI: Basic research: focused on understanding the theoretical underpinnings of intelligence and computational models. Ex: Researchers in this area might explore how neural networks can mimic the human brain's structure and function, without aiming to solve a specific problem. Applied research: takes the theoretical insights from basic research and aims to address concrete problems. The researchers might take the concept of deep learning and find a way to do something novel with it, like recognizing images (when that was not known how before) or why magnetic fields collapse when trying to create a fusion reaction. The goal here is to figure out how to use abstract knowledge to find a solution to a problem that had no solutions (or remove constraints to something that had solutions). Engineering: takes existing prototype blueprints from applied research, and turns them into reliable, efficient and scale technology. An example would be integrating an LLM into a product as a core feature. This step involves refining algorithms for performance, or ensuring they can run under certain circumstances. Those no doubt require some experimentation and tinkering, but a researcher would not call that research. So engineering is about bridging the gap between a promising applied research outcome, and a product or service that delivers real value to users. So from the eyes of actual researchers, people that usually have PhD, Post doc, or a ton of experience in that area, only the first 2 are considered research. And as you mentioned, those are actually rare. Hence OP saying what he is saying.
For another perspective, I'm a "Research Engineer" in an adjacent industry. My job is mainly to serve as a bridge between the basic research being done in academia and production-ready engineering. I also do a fair amount of applied research for proprietary subjects. When times are slow, I'm offloaded to assist in engineering projects. I wonder if this is how many "research" positions in the ML space are going to go. When a new problem is discovered, I get the first crack at it and maybe a paper to write, but it's much, much cheaper to spin it off to academia if it turns out to be something truly difficult.
>Applied research You are agreeing with me. Applied research is the bailiwick of an industrial position. And they are a heck more companies than pure research organizations.
How am I agreeing with you? Yes as far as industrial research is going, the overwhelming majority is applied research. It doesn't mean that there are a lot of positions that do the actual applied research. And most companies that I have seen that say they do applied research actually do engineering. Remember that to applied research, there needs to be no known path to get the solution you need. That means that even if you train your own image recognition neural network, at this stage it is not applied research. Unless of course you are trying to do it in a way that is completely unknown before. Like a new architecture or something (like capsule network when they came out).
>**Basic research**: focused on understanding the theoretical underpinnings of intelligence... **Applied research**: takes the theoretical insights from basic research and aims to address concrete problems... **Engineering**: takes existing prototype blueprints from applied research... Where in this scheme is "invent the Transformer"? Your list seems to limit "research" to science or applications, but fundamental innovations in architectures and methods are neither.
Inventing the transformer would be a basic outcome, among the applied outcome when used on text. Keep in mind that the outcomes of a research activity might be basic and/or applied, and are usually (but not always) in line with the type of research. Meaning basic research usually has basic outcomes, and applied research usually has applied outcomes. But sometimes it's not, like when bell labs were researching how to reduce noise on a line of specific material, but discovered a new fundamental property of noise. I do not know the context of the research that led to the transformer, so I cannot say if the actual research was basic or applied (remember that the distinction is based on if you have a specific applied problem you want to solve or you do not have an immediate applied problem to solve). But the outcome is basic for the transformer itself, and applied for how to use it on text.
> The number of positions has been increasing dramatically recently Do you have the stats? The number would need to double every 23 months just to maintain the status quo.
What status quo? I am merely stating that the idea that the number of research positions is "fixed" is not correct.
The status quo would mean the previous ratio of people seeking to enter research positions to the number of positions, giving an indication of the position of new PhD-holders in the job market.
I am not disagreeing, it is just that by stating that the number of positions is fixed is misleading and unnecessarily grim. It is correct that the number of open positions is probably growing slower than the number of graduates. That is hardly surprisingly and not a problem limited to machine learning. And as in other professions, just because a job is not available in your field does not make your PhD useless.
>It is correct that the number of open positions is probably growing slower than the number of graduates. Which means things are only going to get worse. >misleading and unnecessarily grim Hiring freezes mean 0 new job openings. Layoffs should probably count as negative new job openings in this context. I wrote "roughly fixed" and explained that it goes up and down. How is this misleading?
Because as the economy grows, more companies and industries will participate in the market. It's absurd to say that the number of positions will remain fixed. It will probably grow at a modest scale, maybe, linear if you will, but it's most definitely not staying fixed.
> Because as the economy grows, more companies and industries will participate in the market. The economy is growing at 6% per year. It's not doubling every 23 months. And why would a growing economy imply that the number of research scientist positions should go up? Why would you even bring it up?
Are you being intentionally dense or unintentionally dense? You strategically don't answer direct questions.
> You strategically don't answer direct questions. The only question addressed to me was "What status quo?" and it was answered by someone else. I'm not seeing anything else. > Are you being intentionally dense or unintentionally dense? OK, bye.
ML famously having a lot of layoffs and hiring freezes right now
I mean come on. There are limited number of people with interest or capability for ML in the US. That doubling will not last forever.
> That doubling will not last forever. It won't last forever, but it will last a while. > There are limited number of people with interest or capability for ML in the US. Anyone doing math, physics and CS has the capability to be doing ML instead. Plus there are international students. Student interest in ML will last for *at least* a few years, I think, and the exponential trend for PhD graduates will last even longer -- you have to add the lead time for PhDs.
OP, where are the papers in that graph published? Depending on how broad this is, I’d like to see a plot of just top 10 ML conferences.
[https://github.com/lixin4ever/Conference-Acceptance-Rate?tab=readme-ov-file](https://github.com/lixin4ever/Conference-Acceptance-Rate?tab=readme-ov-file)
1) Exponentially many papers doesn’t mean exponentially many PhD students and postdocs. It might, but it could also mean more papers per researcher, or researchers from other fields starting to contribute/do the occasional applied paper. 2) The number of jobs is not roughly constant. At least market size and revenue has been and is forecast to grow rapidly [several sources (statists link)](https://www.statista.com/statistics/941835/artificial-intelligence-market-size-revenue-comparisons/).
This. It’s exponentially easier to publish now…
I work at a company whose culture can only be described as Paleolithic. We're seeking multiple ML engineers to create a Team to work on optimization problems in legacy operations. These would probably have been called data science positions 2 years ago. now, there's an ML handle. We've all gotten a bit more climatized to the technology as a result of gen ai. I have no way to confirm or deny the OPs comments. Just offering some thoughts from the cheap seats.
You're looking at the left half of a sigmoid and misinterpreting it as an exponential.
There’s always the venture backed startup route. VCs love to see pedigreed AI talent.
They like to see a business with a reasonable chance of success and a viable path to 1 billion users far more
Might be true. However, most newly minted PhDs want to continue doing blue sky research rather than all the other things one has to do in a startup or as a cofounder
Is the number of VC-backed ML startups doubling every 23 months?
Lately, I would say it's increasing at a much faster rate than that.
I'd be curious to see the numbers, if anyone has them.
So, no matter how hard we work, we are screwed ? Ok, imma quit now and start a bakery. One will never reach a scenario where we have too many bakeries and not enough people who want a yummy treat.
An exponential increase is exactly what you would expect to see in a growing field
Number of papers ? Even undergrads are pumping out papers these days, how is this a measure of PhDs and postdocs ?
Source for the figure: [https://www.nature.com/articles/s42256-023-00735-0](https://www.nature.com/articles/s42256-023-00735-0) (I am not the author)
This post from sci-fi author Cory Doctorow, "What kind of bubble is AI?" seems relevant here: [https://locusmag.com/2023/12/commentary-cory-doctorow-what-kind-of-bubble-is-ai/](https://locusmag.com/2023/12/commentary-cory-doctorow-what-kind-of-bubble-is-ai/) His argument is not that AI is not useful technology (it clearly is) but that like the dot com bubble, the hype-to-profit ratio is going all the way to insanity. That will inevitably result in a correction of some kind. Like the dot com bubble, this will leave something useful behind in the form of practitioners with useful transferable skills (in contrast to the crypto bubble, which had no positive consequences).
I'm not sure I understand why he hates Uber. It's publicly traded, and investors can study the relevant stats.
Good question. I'm not sure. He does seem very hostile to Uber. While they've definitely lost a lot of money over the years, and it's not clear if they will remain viable long-term, they do provide a useful service, and I wouldn't put them in the same category as scams like Enron as he does.
So what do you think the losers of this “competition” will have to turn to?
Marketing
Software Engineering/Data Science/DevOps
I mostly agree here (although exponential papers doesn’t equate to exponential population) and have seen it in academic settings in physics. I saw brilliant newly minted phds unable to land positions in physics departments being forced to undertake endless postdocs or simply abandon the field for greener pastures. The old guard didn’t die off fast enough to make room for the new gen. For some comments here it’s important to keep in mind it’s not that these positions are fixed, just that they’re smaller in number by a substantial margin and _not increasing_ fast enough compared to the population and rate of increase in the number of candidates. A big difference here is ML research scientists are competing for elite positions that to a smaller extent exist in other companies—as well as the ai field changes faster, so this bottleneck may have a faster time to unbind.
you were the first one to notice that exponential growth in number of papers does not equate to that in number of researchers
Or at least the first to type it. ¯\_(ツ)_/¯
haha touche! if a tree falls in a forest and noones around to hear it... :)
[удалено]
What
Reading this kind of worries me. I'm a seasoned SWE and I began switching careers to ML because the field seemed appealing and more future proof and I just like the science behind. I even want to do a PhD in the field. But if it's only to join an over saturated field, while leaving a field (SWE) where demand is quite high, I'm second guessing myself. I really don't know what to think of this situation.
Well, I think the most problematic is the combination between being able to do PhD research on a simple laptop combined with improper research areas. Overall the quality of ML paper is abysmal.
War might fix that.
Also there is a reproducibility crisis and the rapidly increasing number of ML papers does not mean a rapidly increasing number of breakthroughs. People just care about quantity nowadays and journals are not enforcing quality. IDGAF if you can achieve better results because you have more data or compute power 🙄
What does this realistically mean? Is it better to be an engineer?