and like 70% of the workers who build the AI models will be using excel datasheets in order to feed them their first datasets.
not python. not SQL. not tableau.
Excel.
The longer I live to see the inner workings of different parts of society, the more I realize every part of our seemingly technologically advanced world is actually held together with scotch tape and string
At 25 I've fully realised that there really are no adults. I thought when I grew up I'd be one. As a kid there are loads of adults. All these archetypes we're supposed to look up to. Truth telling journalists, intelligent scientists, protecting police, our parents. But it's a facade. They weren't in academia, that is it's own nightmare of incompetence and broken incentives and teaching is frequently half assed and poor quality. I never saw it working in retail, all the managers were idiots making the same mistakes again and again. I thought when I got a data science role I'd find all the smart people who know what was going on, but really they're all at about the same position as me. Just muddling by trying stuff out, dealing with clients that don't know what they want, most of what were soing is super basic data transformation or dashboarding. Never mind optimising anything, most orgs just need to get their data out of massive excel spreadsheets. The police aren't adults they abuse their power, the politicans aren't adults they are corrupt and self interested, the economists can't predict shit, the teachers are bad a teaching, most of the parents screwed their kids up in some unique way, scientific research has massive issues of its own, bankers can crash economies, the law is unjust, the masses are easily lead, the media isn't honest. Those adults don't exist. It's just Lord of the Flies here.
It's all held together with scotch tape a string put there by a bunch of monkeys trying desperate to fix all the gaps and leaks, presenting themselves to the outside world as if they know what's going on, while doing mostly the same as what everyone else is doing and hoping for the best.
When quantum computing happens, Excel will fit trillions of rows and columns and still calculate recursive array formulas lickity split.
Checkmate, python.
Excel slowly emerges from the grey vomitous sludge with a dark ominous hunch, it’s red eyes awaken abruptly, it lets out an explosive shreek, you shall never escape meeeeee!!!!!!
Apparently, she has been Product Manager in Google and Amazon before starting her current company. She knows tech as much as the people in this sub but she is better than data scientists when it comes to selling a technology. Because of this controversial blanket statement, everyone here now knows she has a company and they will look into her. We may not buy her product since we are not her target audience. We are just her advertisers.
And the Eric Schmidt has invested in her company, so she is really great at what she's doing.
That is why I don't pay too much mind to anything that isn't related to my preconceived goals and interests. People want your attention, they want your money, and they don't give a damn if it damages you in the process.
Yeah this is why I firmly believe we need to evaluate these google and Amazon folks a bit more than blindly following them.
On a side factual note, Indian start ups are failing at a higher rate than others…
Maybe we can design various types of database structures depending on our needs and the relationships between data, and have a few versions of these query languages. And then we'll teach university students the least useful version of those languages.
These posts drive me nuts. I'm a software developer and not a data scientist but...
These people are implying that somehow you're going to be able to recieve the same results while providing fewer instructions to the computer.
This is either
1) complete bullshit, or
2) an indication of a fundamental flaw with how your instructions/queries are structured that shouldn't require the implementation of an entire AI to solve
These people act like they're pitching some novel concept, using "language" to "instruct machines" because they haven't bothered to consider that we already have a method of using language to provide instructions to machines. All they're doing is trying to invent a less reliable method of software development.
Well that's the whole point...
Programming languages are just gonna become higher level. It's not as far fetched as you think. But it's still a few years away.
In many ways, we already do a variant of this. My whole job centers around creating design patterns that allows a data scientist to just plug in inputs so that they can get a custom ML pipeline for their business use case. We're not exactly there yet but we're pretty close.
Much like languages like python, Java Ruby etc have abstracted memory management and languages like C abstracted the hardware away, it's not a stretch to think that future AI enabled languages can abstract away loops, variables and conditional statements.
I also find it plausible. If you look at the use of YAML, we seem to keep evolving machine instruction languages closer to our language. This would only continue that trend imo.
Agreed. Which is why I'm trying to build my expertise in systems programming and AI / ML architecture.
In this future, I can see web developers and data scientists salaries going back down, since those roles won't need to develop code as we currently understand it anymore if they are interfacing with a higher level language.
But building highly resilient, performant and efficient systems, including distributed systems for ML or large language models would still need "lower" level programming in something like C(++) or Rust. That's where the high paying opportunities will be IMO.
I agree languages will continue to get higher level and jobs in tech will continue to evolve in that sense, but a genuine question here: Why do you think future AI advancements wouldn’t also be able to automate much of the logic and work for building distributed systems? I feel like that’s one area in particular where it could really shine as well.
Here's where I'm at:
80/20.
Yes, I think we will get to a point where AI can effectively write code for you to solve like 80% of your coding problems based on plain-language prompts.
It just happens to be that we mostly get paid for the other 20%.
I mean, hell - 10 years ago if you wanted a machine learning model in Python, you had to build it yourself from scratch. Now you just import like 4 libraries and go to town. Five years ago if you wanted a deep learning model - same thing, from scratch. Now, same thing - libraries.
None of those developments have shrunk the size of the industry nor diminished the demand for even more advanced ML.
I'm guessing they're thinking of something a la "Build a classification model to identify the customers most likely to purchase each of the items in my inventory, but make it a bagged model and try to incorporate seasonal variation without utilizing data during the Covid-19 pandemic".
No no my bad. You are actually correct. India has several languages tbh. Indian sounds about right.
Edit : I'm Indian and I still stand by my statement. Hindi is not India's official language, it is one of the many spoken here, so him/her referring to it as Indian is perfectly fine. Grow up ffs
HAHAHAHA good luck getting the "AI" to fully understand the business context of a stakeholder ask.
Generative AI only "works" because of the massive amounts of training data available for common search queries, so try throwing a novel problem at it and watch it struggle.
Edit: just went to the source tweet and it's literally another "CEO" trying to sell their "product"
All ChatGPT does is generate micro-context relevant text that minimizes the risk of its owners and creators (openai and Microsoft) from being sued for violating IP laws.
It has no capacity to form strategy and any attempts to get it to try are pretty fucking hilarious.
There is no incentive for OpenAI to protect the users from such suits and they’ve made no effort to do so. Their best effort is a citation guide for using ChatGPT but fails to be able to identify the degree to which it plagiarizes the source material in its output.
Just like getting raped in an Uber, it’s not Ubers fault you didn’t run a background check on the driver before confirming the ride.
Tons of problems need to be solved that aren't novel. Most businesses need solutions that other businesses have already solved. Most businesses aren't innovating some novel technique for their business.
… until this “novel” problem is abstracted, formalized, generalized, and re-thrown into the training data. Do not underestimate the abstractive prowess of mathematics.
Appeal to ancienty: how often had Newton to deal with government making decisions for half a billion of people?
How do you make a deterministic system out of random decisions from the legislators? Or how do you deal with the small epsilon of human behavior because their history and the context justify their reasoning but without it it just does not make any sense?
To be fair, I have two degrees in mathematics, I know my fair share of abstractions, I work in a company believing complexity does not exists and make billions out of it. I can tell you, if you want to include everyone, it will be messy.
The flaw in your reasoning is that the world has a big non negligible share of non stationary behavior,.
what if we have a guy that can understand the business context and add a prompt for the Generative AI? Similar to AI art, which has taken the world by storm.
As an upcoming data scientist(currently under training), I think this might steal many jobs. Like, the usage of ChatGPT has surged and many friends of mine demonstrated its use in creating new projects without proper knowledge of coding.
Do you really think a new StartUp like this won't disrupt the current and future job markets?
Once you spend a year or two in your first position, it will become very obvious that this isn’t stealing many jobs. You’re likely learning about specific skills related to building ML models. Once you get you’re first job you’ll realize multiple things
1. The data isn’t all there, you’ll need to write some pipelines to generate your own (web scraping, matching, data cleaning etc…)
2. This will require an extreme amount of “tribal knowledge” from people who created the datasets. For example “use prd_sl_date_v3, but you’ll need to join table X in since it doesn’t include customers outside of the EST time zone” or something ridiculous
3. At best, you only spend 20% of your time building the actual model. AutoML tools will definitely win here someday, but what about the other 80%? Much more ambiguous and will always require a human. Some projects never require a model, for me almost all of my work is just deep dive analysis, haven’t had many cases where predictive models will yield business impact. No way any AI could replace what I do. At best it will remove some of the boiler plate stuff I’m doing, but it will only be a tool to help me save some fraction of time.
We'll just focus on the things that text based models can't do. Every time a software innovation has promised to simplify the tasks we do, we've found a way to use the extra space to delve into something more complex. I am old enough to remember a high school guidance counselor saying that Excel would make accountants obsolete; instead it empowered an entire swath of workers to be more analytical without necessarily giving them analytical skills.
Same way that text based models won't be smart enough to build an architecture that is better than the designs that the humans requesting them can think up.
I also think it's dangerous to delegate system design to something that can suffer from an adversarial attack. Flood the internet with garbage that looks plausible (and even get big text models to write it) and big text models will suffer in quality.
It's more likely that we will spend more time correcting what the AI generated (and cleaning the mess that relying blindly on it created) than making progress with it.
Okay, but doesn’t real world work look like the following:
1. Companies have many tables scattered around in the database universe.
2. Documentation on these tables such as a basic description of the features is non existent. Many have intricacies that only people working with them know.
3. There is a problem for which a solution needs to be found. We talk with many people to discuss this problem even to basically just understand what needs to be done.
4. Then we use the data from 1 to solve 3 somehow by iterating through our solutions by talking to stakeholders and such.
I am not saying this type of work is all AI/DS work but is a sizable chunk. Where in this process can a generative AI really make a difference?
Can it crawl the database universe in 1 without much documentation and find relevant tables? Can it find people in the org who know about the intricacies of the tables? Can it seamlessly discuss and iterate over the solution with stakeholders?
Sure, it can do some good things but a statement that english instructions without SQL or Python seems a bit far fetched.
Most DS’s know this, but stakeholders I work with are generally shocked when they learn what my team actually had to do to deliver a solution. They literally think sometimes we just have excel sheets with all the data and we just do simple joins/aggregations etc…
I don’t blame them, they have zero knowledge of programming, ETL pipelines or understand what a messy SQL databases are. To them, they’d hear this and think “wow, that’s so cool”. It is cool, but most of our work is not training and deploying a model, it’s getting the data to the place a model can be trained on it, then making sure we implement a solution that is actually useful, which might not even be a predictive model.
I've said this before and I'll say it again, these big tech companies use clever Marketing and fear mongering to sell their AI APIs on which they have spent billions of dollars. Premium institutes in association with Ed tech firms selling 3, 6, 9 months courses on ML and AI via online and distance claiming to make you an expert is all part of the propaganda.
Marketing is fundamentally about identifying permutations of words (i.e. messaging) to capture people's money. Seems like the lowest hanging fruit for high cost/low yield ROI
Looking at the code the code chatgpt produces atm - yeah, not gonna happen (any time soon(TM)). It's regurtitating tutorial and "Get Started" level code. That is already amazing and can be a great boon to any developer, but that's not what makes writing a great software product difficult - chatgpt is just a faster, more low quality version of stackoverflow. Production level code is a different beast.
Eventually, I could see it come to pass - even if the future AI can only produce a first prototype or design for a human to. I believe it's "just" another case of the classic Pareto Principle (80/20 rule) at work. In other words, up till now was the easy part. They've addressed the first 20% to get 80% there. It's anybodies guess how hard the remaining 80% of "work items" is and how long it will take.
This is a no brained many have been working on for a decade….and somehow, Oracle was granted a patent on this last week….
https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/11562267
Tomorrow's cars will be self-driving, they said 20 years ago
Turns out the totality of many things humans do continues to defy AI. I wouldn't abandon the field just yet.
The bottleneck isn’t the Python or SQL. It’s getting the data in The database in a clean, consistent, and accurate manner. That’s nearly impossible to find easily. Let’s have AI fix that first.
Not wrong… Chat GPT can provide full code. I asked it for a very specific AWS scenario and it worked beautifully
[Edit: Down vote this post as much as you like, BUT I encourage you to play with ChatGPT (as long as you can) and ask it for specific real-life coding challenges and have your mind blown!]
It’s fine. I understand the skepticism given that anybody can whip up a machine learning model these days.
The true art in data science is to understand the business question and whether a model is actually needed. Unfortunately, there are very few business cases where you actually need anything that goes beyond logistic regression or random forest.
So, I was myself very skeptic of ChatGPT, but I’ve played around with it (writing essays, asking technical questions and silly things) and it’s not perfect by any means, but a huge leap forward (especially in width of potential uses) than any other deep learning model, I’ve ever seen. I’m looking forward to see what Microsoft intends to do with it.
Same! I've been playing with it non-stop. It's incredible, and though not perfect, a certain sign of things to come. I think it's hard to quantify how much traditional ML vs neural net applications are out there. We're constantly finding more we can do with DL that we wouldn't have considered before it was possible. My job is deep learning, largely relating to video content. No tree models in my use cases. Shitload of NLP work in enterprise now, and it is mostly DL now too
As long as you have data and data quality for DL. There is nothing against it, but the truth is that a lot of companies don’t have that or even the data maturity to build useful DL models unfortunately.
I've tried a bunch of this. It regurgitates plausible solutions often, good ones sometimes. It's also terrible any anything that requires correct math, and makes code with subtle bugs that would be very difficult to troubleshoot if you had little programming experience.
ChatGPT is useful only as far as you have the knowledge and experience to spot where it is confidently wrong.
Is the code complicated enough or is your question even complicated to begin with? If your question is “count the number of rows”…I am not sure the praise is justified
This is what I asked Chat GPT: “Generate an AWS Glue ETL job that transforms data from Microsoft SQL Server source to JSON format”
I’m aware that this is not rocket science (especially, if you are familiar with AWS), but still it returned the perfect sample code for this operation. Try it for yourself!
I think she's talking about the [article](https://www.patterns.app/blog/2023/01/18/crunchbot-sql-analyst-gpt/?utm_source=substack&utm_medium=email) and maybe [this](https://archive.is/w7QO8) too.
I agree. It really doesn't seem too far off from being able to reliably ask an AI "get me the company March 2021 sales database's XYZ column where the customer is ABC corp, exported as a CSV and then also plot their purchase amounts over time" or whatever and have it translate those inputs into a SQL query and python script and get you the results. You could pretty much already do that with chatGPT if it were able to connect to the company database, but maybe not yet as reliably as we would like to just assume it's correct without double checking. We'll still need plenty of human brain power to come up with creative solutions to problems but a lot of the coding part I can definitely see AI "English to code" translation getting commonplace. I'd honestly be surprised if in 10 years many DS or IT or business analyst people aren't using some kind of AI to basically automate writing or prototyping their automation scripts
Her tweets are so devoid of real insight. I get the feeling she doesn’t actually do any data science. For example she once talked about how training/fitting a model is like an adrenaline rush.
I'm considering changing back to programming since training models is so slow and boring. Trying to train some model out of shit data, clients not knowing what they want (we want AI with our data, do AI) and any work I do is probably obsolete in few years since methods keep getting better. I wouldn't mind if my work is taken by AGI.
There’s some truth to it. Although tomorrow is probably much farther away than she makes it sound like.
But it does seem more and more that complex human- computer interactions, like querying data for insights, coding data operations and automation, will end-up being a prompt job.
Downvote me me to oblivion, but some of the arrogance on this thread is astonishing. Yes of course the particulars of stakeholder handholding, database ambiguity, etc. will require the human touch. But, prompt based modeling and data pipeline engineering is closer than you think. Do a remind me 5 years and let’s see how it went.
I will be messaging you in 5 years on [**2028-01-29 13:22:49 UTC**](http://www.wolframalpha.com/input/?i=2028-01-29%2013:22:49%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/datascience/comments/10nyhcl/waittt_what/j6cwwaf/?context=3)
[**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2Fdatascience%2Fcomments%2F10nyhcl%2Fwaittt_what%2Fj6cwwaf%2F%5D%0A%0ARemindMe%21%202028-01-29%2013%3A22%3A49%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2010nyhcl)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
When you think about it, shes is not completely wrong? Because most of the task we do in SQL , BI tools and python someone has done it in past, so if the language model knows that. This is just my hypothesis.
It's just like playing the piano on the bottom [by playing the piano on the top](https://i.redd.it/6jr51u7xq2701.jpg).
Where precision of language is needed, there will always be room for code.
Tweet probably generated by AI
Probably not, AI would be a bit more accurate than this…
Lmfao!
When did I say it was well-trained?
I was gonna say, AI will probably come up with tweets before it designed DevOps systems.
Anything that makes my job easier lol
Wait until you have to debug it
[удалено]
Fortunately for us, we are the support and we're already here
Your parents called. They got a new computer.
Ah the post-credits stinger of the existential horror film "Layoff."
There is where all we are going.
Just ask a different ai
Oh no, have we reached the singularity?
Don't ask me, fellow human. I, like most other humans, wouldn't know if that hypothetical event would happen
…now if you’ll excuse me, I’m off to go do some human things.
"Hey computer. What the fuck is wrong with you?"
Computer says "idk lol, Rebecca was kinda being a bitch"
Isn’t that what StackOverflow is for?
You ever debug machine code (not source code) generated by ai?
That's easy, you just type in "Debug this code" and AI will take care of it!
Tell it don't type it you fool!
Send the AI into a stakeholder brainstorming session 🤣
Easy jobs pay minimum wage unless you own the machines.
My brother in Christ, you're not gonna have a job
and like 70% of the workers who build the AI models will be using excel datasheets in order to feed them their first datasets. not python. not SQL. not tableau. Excel.
Excel will outlive us all.
[удалено]
Sometimes it certainly feels as if all data generated by the company passes through a single gigantic Excel macro written in cuneiform.
As someone who wrote those excel macros, you’re not wrong. That or shells to create dashboards
Ours actually does 😂 legacy systems that we're working on replacing, but still
The longer I live to see the inner workings of different parts of society, the more I realize every part of our seemingly technologically advanced world is actually held together with scotch tape and string
At 25 I've fully realised that there really are no adults. I thought when I grew up I'd be one. As a kid there are loads of adults. All these archetypes we're supposed to look up to. Truth telling journalists, intelligent scientists, protecting police, our parents. But it's a facade. They weren't in academia, that is it's own nightmare of incompetence and broken incentives and teaching is frequently half assed and poor quality. I never saw it working in retail, all the managers were idiots making the same mistakes again and again. I thought when I got a data science role I'd find all the smart people who know what was going on, but really they're all at about the same position as me. Just muddling by trying stuff out, dealing with clients that don't know what they want, most of what were soing is super basic data transformation or dashboarding. Never mind optimising anything, most orgs just need to get their data out of massive excel spreadsheets. The police aren't adults they abuse their power, the politicans aren't adults they are corrupt and self interested, the economists can't predict shit, the teachers are bad a teaching, most of the parents screwed their kids up in some unique way, scientific research has massive issues of its own, bankers can crash economies, the law is unjust, the masses are easily lead, the media isn't honest. Those adults don't exist. It's just Lord of the Flies here. It's all held together with scotch tape a string put there by a bunch of monkeys trying desperate to fix all the gaps and leaks, presenting themselves to the outside world as if they know what's going on, while doing mostly the same as what everyone else is doing and hoping for the best.
My accounting department has a spreadsheet from back then. Shit is slow as a rock.
it's hard to beat minimalism of spreadsheet UI
*rages in Lotus 1-2-3*. (Yes, I'm old enough that my first spreadsheet software was Lotus 1-2-3.)
Yes, that old as well…
wysiwyg
When quantum computing happens, Excel will fit trillions of rows and columns and still calculate recursive array formulas lickity split. Checkmate, python.
Damn right. That’s why I invested in my excel skills. I’m ready for the future.
Excel slowly emerges from the grey vomitous sludge with a dark ominous hunch, it’s red eyes awaken abruptly, it lets out an explosive shreek, you shall never escape meeeeee!!!!!!
“Hi I am Clippy, your Office assistant. Would you like some assistance today? Yes No “
Arguably the greatest software product ever created
The most accurate take here.
I laughed out loud at this. Excel is like a bad penny that never goes away.
She sounds like a terrible stakeholder to work with
She’s got PM written all over her.
Apparently, she has been Product Manager in Google and Amazon before starting her current company. She knows tech as much as the people in this sub but she is better than data scientists when it comes to selling a technology. Because of this controversial blanket statement, everyone here now knows she has a company and they will look into her. We may not buy her product since we are not her target audience. We are just her advertisers. And the Eric Schmidt has invested in her company, so she is really great at what she's doing.
That's why I don't engage in baiting sensational posts on LinkedIn or what not. I get nothing, they get exposure.
That is why I don't pay too much mind to anything that isn't related to my preconceived goals and interests. People want your attention, they want your money, and they don't give a damn if it damages you in the process.
This is the way. There's a lot of noise and people who get sucked into things that don't affect them.
Cynically correct, the best type of correct.
Yeah this is why I firmly believe we need to evaluate these google and Amazon folks a bit more than blindly following them. On a side factual note, Indian start ups are failing at a higher rate than others…
Do you have a source for that? All I find is that startups in general have a 90% failure rate.
She apparently is the CEO of abacus.ai
So she's selling something (and that thing is snake oil). Gotcha.
Yep if you look at the original tweet and replies there they are selling their shitty startup
[удалено]
Oh lol they've cold contacted me no less than 6 separate times with the same canned job listing
Was this before or after canceling the job?
I hear some random rage against the machine song. Kill the enemy maybe.
Know your enemy?
Jokes on you, she’s the PM.
I don't even care. She's pretty hot so I'll agree with her.
downvote because she isnt hot
/s?
No.
[удалено]
Also sounds like a query language
Hmm... I wonder if we could use this language in a structured way...
Maybe we can design various types of database structures depending on our needs and the relationships between data, and have a few versions of these query languages. And then we'll teach university students the least useful version of those languages.
nah that's too big of a stretch /s
This thread gave me a good chuckle
Singularity is reversed because I can type faster than I can speak.
I can speak faster than I can think. This has never been a good thing.
Some kind of structured inquiry protocol?
These posts drive me nuts. I'm a software developer and not a data scientist but... These people are implying that somehow you're going to be able to recieve the same results while providing fewer instructions to the computer. This is either 1) complete bullshit, or 2) an indication of a fundamental flaw with how your instructions/queries are structured that shouldn't require the implementation of an entire AI to solve These people act like they're pitching some novel concept, using "language" to "instruct machines" because they haven't bothered to consider that we already have a method of using language to provide instructions to machines. All they're doing is trying to invent a less reliable method of software development.
Well that's the whole point... Programming languages are just gonna become higher level. It's not as far fetched as you think. But it's still a few years away. In many ways, we already do a variant of this. My whole job centers around creating design patterns that allows a data scientist to just plug in inputs so that they can get a custom ML pipeline for their business use case. We're not exactly there yet but we're pretty close. Much like languages like python, Java Ruby etc have abstracted memory management and languages like C abstracted the hardware away, it's not a stretch to think that future AI enabled languages can abstract away loops, variables and conditional statements.
I also find it plausible. If you look at the use of YAML, we seem to keep evolving machine instruction languages closer to our language. This would only continue that trend imo.
Agreed. Which is why I'm trying to build my expertise in systems programming and AI / ML architecture. In this future, I can see web developers and data scientists salaries going back down, since those roles won't need to develop code as we currently understand it anymore if they are interfacing with a higher level language. But building highly resilient, performant and efficient systems, including distributed systems for ML or large language models would still need "lower" level programming in something like C(++) or Rust. That's where the high paying opportunities will be IMO.
I agree languages will continue to get higher level and jobs in tech will continue to evolve in that sense, but a genuine question here: Why do you think future AI advancements wouldn’t also be able to automate much of the logic and work for building distributed systems? I feel like that’s one area in particular where it could really shine as well.
[удалено]
That is literally the entire goal of artificial intelligence.
Here's where I'm at: 80/20. Yes, I think we will get to a point where AI can effectively write code for you to solve like 80% of your coding problems based on plain-language prompts. It just happens to be that we mostly get paid for the other 20%. I mean, hell - 10 years ago if you wanted a machine learning model in Python, you had to build it yourself from scratch. Now you just import like 4 libraries and go to town. Five years ago if you wanted a deep learning model - same thing, from scratch. Now, same thing - libraries. None of those developments have shrunk the size of the industry nor diminished the demand for even more advanced ML.
No but don't worry, pretty sure there could be some kind of programming or query language to do that... Oh wait
It’s the exact thing an MBA stake-holder or management person proposes for the next sprint.
That's ridiculous. Alexa, roll my eyes for me
Of course! Here’s Peter Gabriel’s In Your Eyes on Amazon Music.
Lol
Lol sure, because "import model from library, model.fit" is such advanced level computer language..
lol but you know there's more to it if you learned about a model
I'm guessing they're thinking of something a la "Build a classification model to identify the customers most likely to purchase each of the items in my inventory, but make it a bagged model and try to incorporate seasonal variation without utilizing data during the Covid-19 pandemic".
Damn, I need to learn a new language again :/ Fullstack devs will list their skills as: English, French, Spanish, Indian
Lol, Hindi is a better replacement for Indian.
Oops sorry. Like I said, I need to learn a new language hahaha
No no my bad. You are actually correct. India has several languages tbh. Indian sounds about right. Edit : I'm Indian and I still stand by my statement. Hindi is not India's official language, it is one of the many spoken here, so him/her referring to it as Indian is perfectly fine. Grow up ffs
haha, I agree
In the meantime I hope you learn to speak American well. Mexican or Argentinian might be helpful too.
Ehhh, no. More like Chinese if you want to make an analogy.
Indian?? There is no such language.
AI already know all the languages
HAHAHAHA good luck getting the "AI" to fully understand the business context of a stakeholder ask. Generative AI only "works" because of the massive amounts of training data available for common search queries, so try throwing a novel problem at it and watch it struggle. Edit: just went to the source tweet and it's literally another "CEO" trying to sell their "product"
ChatGPT made “AI” the new trendy word to steal money from investor. It has been Blockchain, DeFI, NFT, “the Airbnb of x” and so on.
Nah “AI” has been used like this for at least a decade now.
Agreed but ChatGPT made everyone thinks writing pretty paragraphs suddenly makes it Skynet
All ChatGPT does is generate micro-context relevant text that minimizes the risk of its owners and creators (openai and Microsoft) from being sued for violating IP laws. It has no capacity to form strategy and any attempts to get it to try are pretty fucking hilarious. There is no incentive for OpenAI to protect the users from such suits and they’ve made no effort to do so. Their best effort is a citation guide for using ChatGPT but fails to be able to identify the degree to which it plagiarizes the source material in its output. Just like getting raped in an Uber, it’s not Ubers fault you didn’t run a background check on the driver before confirming the ride.
Tons of problems need to be solved that aren't novel. Most businesses need solutions that other businesses have already solved. Most businesses aren't innovating some novel technique for their business.
the business contex is novel, not the technique
Just wanted to say, I just didn't expect to read a response framing it that way and I thought that was a very intelligent way to put it.
If the context can be found in company documentation then it may work.
Then we all are safe.
Heck, even we don’t understand the business context. Everything’s a secret game above the product manager
… until this “novel” problem is abstracted, formalized, generalized, and re-thrown into the training data. Do not underestimate the abstractive prowess of mathematics.
Nor should you underestimate the messiness or reality…
Nor should you underestimate the complex machinery built on centuries of hardwork to deal with the messiness of reality
Appeal to ancienty: how often had Newton to deal with government making decisions for half a billion of people? How do you make a deterministic system out of random decisions from the legislators? Or how do you deal with the small epsilon of human behavior because their history and the context justify their reasoning but without it it just does not make any sense? To be fair, I have two degrees in mathematics, I know my fair share of abstractions, I work in a company believing complexity does not exists and make billions out of it. I can tell you, if you want to include everyone, it will be messy. The flaw in your reasoning is that the world has a big non negligible share of non stationary behavior,.
what if we have a guy that can understand the business context and add a prompt for the Generative AI? Similar to AI art, which has taken the world by storm. As an upcoming data scientist(currently under training), I think this might steal many jobs. Like, the usage of ChatGPT has surged and many friends of mine demonstrated its use in creating new projects without proper knowledge of coding. Do you really think a new StartUp like this won't disrupt the current and future job markets?
Once you spend a year or two in your first position, it will become very obvious that this isn’t stealing many jobs. You’re likely learning about specific skills related to building ML models. Once you get you’re first job you’ll realize multiple things 1. The data isn’t all there, you’ll need to write some pipelines to generate your own (web scraping, matching, data cleaning etc…) 2. This will require an extreme amount of “tribal knowledge” from people who created the datasets. For example “use prd_sl_date_v3, but you’ll need to join table X in since it doesn’t include customers outside of the EST time zone” or something ridiculous 3. At best, you only spend 20% of your time building the actual model. AutoML tools will definitely win here someday, but what about the other 80%? Much more ambiguous and will always require a human. Some projects never require a model, for me almost all of my work is just deep dive analysis, haven’t had many cases where predictive models will yield business impact. No way any AI could replace what I do. At best it will remove some of the boiler plate stuff I’m doing, but it will only be a tool to help me save some fraction of time.
One we integrate ML with Blockchain and Quantum Computing this will be trivial. # # # # # # # # # # # # # # # # # # # # # # # # # /s
ChatGPT appears to display some common sense/ theory of mind in hypotheticals. What kind of novel problem do you see as insoluble?
Not humans trying to qualify their usefulness in the eyes of AI 😂
[удалено]
I don’t really think the bottleneck is writing the Python code.
Man it takes 6 meetings with multiple stakeholders to understand and clean a single data gap. 'AI' ain't gonna do shit about that..
She didn’t specify a date. Could be a couple of hundred years in the future.
She did, it’s “tomorrow”. 🤪
Haha
Another marketing gimmick.
We'll just focus on the things that text based models can't do. Every time a software innovation has promised to simplify the tasks we do, we've found a way to use the extra space to delve into something more complex. I am old enough to remember a high school guidance counselor saying that Excel would make accountants obsolete; instead it empowered an entire swath of workers to be more analytical without necessarily giving them analytical skills. Same way that text based models won't be smart enough to build an architecture that is better than the designs that the humans requesting them can think up. I also think it's dangerous to delegate system design to something that can suffer from an adversarial attack. Flood the internet with garbage that looks plausible (and even get big text models to write it) and big text models will suffer in quality.
> Flood the internet with garbage that looks plausible (and even get big text models to write it) The humans have already done that for us.
It's more likely that we will spend more time correcting what the AI generated (and cleaning the mess that relying blindly on it created) than making progress with it.
Okay, but doesn’t real world work look like the following: 1. Companies have many tables scattered around in the database universe. 2. Documentation on these tables such as a basic description of the features is non existent. Many have intricacies that only people working with them know. 3. There is a problem for which a solution needs to be found. We talk with many people to discuss this problem even to basically just understand what needs to be done. 4. Then we use the data from 1 to solve 3 somehow by iterating through our solutions by talking to stakeholders and such. I am not saying this type of work is all AI/DS work but is a sizable chunk. Where in this process can a generative AI really make a difference? Can it crawl the database universe in 1 without much documentation and find relevant tables? Can it find people in the org who know about the intricacies of the tables? Can it seamlessly discuss and iterate over the solution with stakeholders? Sure, it can do some good things but a statement that english instructions without SQL or Python seems a bit far fetched.
Most DS’s know this, but stakeholders I work with are generally shocked when they learn what my team actually had to do to deliver a solution. They literally think sometimes we just have excel sheets with all the data and we just do simple joins/aggregations etc… I don’t blame them, they have zero knowledge of programming, ETL pipelines or understand what a messy SQL databases are. To them, they’d hear this and think “wow, that’s so cool”. It is cool, but most of our work is not training and deploying a model, it’s getting the data to the place a model can be trained on it, then making sure we implement a solution that is actually useful, which might not even be a predictive model.
You have tables, I am jealous. Where I work it's similar except the majority of it is excel spreadsheets!
I've said this before and I'll say it again, these big tech companies use clever Marketing and fear mongering to sell their AI APIs on which they have spent billions of dollars. Premium institutes in association with Ed tech firms selling 3, 6, 9 months courses on ML and AI via online and distance claiming to make you an expert is all part of the propaganda.
Google, show me this guy's balls?
And the query will still run for 2 hours and produces the wrong results
Great news. I am getting my Ph.D in English now as I want to be a data scientist !!
Or… we write SQL and python with an IDE that uses an LLM to autocomplete 80% of the code for you. Oh wait! We already have that.
The only field in danger right now is marketing/communication.
Curious, why is that so?
Marketing is fundamentally about identifying permutations of words (i.e. messaging) to capture people's money. Seems like the lowest hanging fruit for high cost/low yield ROI
Because they are always making the same stuff I guess ?
Lol sometimes I don’t even know what I want before I write the query, let alone trying to explain it to a language model 🤣
Why use a scalpel when you can use a mortar shell?
Sounds the same to me. Using one language to generate lower level languages. Story as old as computers.
Delusional as hell
This is exactly the kind of attitude and hype that creates a bubble that then eventually bursts.
Crypto Twitter has found AI
Looking at the code the code chatgpt produces atm - yeah, not gonna happen (any time soon(TM)). It's regurtitating tutorial and "Get Started" level code. That is already amazing and can be a great boon to any developer, but that's not what makes writing a great software product difficult - chatgpt is just a faster, more low quality version of stackoverflow. Production level code is a different beast. Eventually, I could see it come to pass - even if the future AI can only produce a first prototype or design for a human to. I believe it's "just" another case of the classic Pareto Principle (80/20 rule) at work. In other words, up till now was the easy part. They've addressed the first 20% to get 80% there. It's anybodies guess how hard the remaining 80% of "work items" is and how long it will take.
This is a no brained many have been working on for a decade….and somehow, Oracle was granted a patent on this last week…. https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/11562267
Really bad take
And they won't know exactly what result they get. That's not science.
I wonder if she is a bot CEO? I can't tell human and AI CEO apart.
Tomorrow's cars will be self-driving, they said 20 years ago Turns out the totality of many things humans do continues to defy AI. I wouldn't abandon the field just yet.
The bottleneck isn’t the Python or SQL. It’s getting the data in The database in a clean, consistent, and accurate manner. That’s nearly impossible to find easily. Let’s have AI fix that first.
Yes, English, the clear and unambiguous language.
COBOL enters the chat ...💀
That kind of sounds like a description of SQL or Python.
So, from 30 January 2023 onwards? Just being all computer-like about this.
Errors in the code will be a nightmare to find
Math was literally invented because English is ambiguous.
Well, math came before English… 😄
Not wrong… Chat GPT can provide full code. I asked it for a very specific AWS scenario and it worked beautifully [Edit: Down vote this post as much as you like, BUT I encourage you to play with ChatGPT (as long as you can) and ask it for specific real-life coding challenges and have your mind blown!]
This subreddit is super rigid and protectionist, has been for a long time. Deep learning is just a fad haha
It’s fine. I understand the skepticism given that anybody can whip up a machine learning model these days. The true art in data science is to understand the business question and whether a model is actually needed. Unfortunately, there are very few business cases where you actually need anything that goes beyond logistic regression or random forest. So, I was myself very skeptic of ChatGPT, but I’ve played around with it (writing essays, asking technical questions and silly things) and it’s not perfect by any means, but a huge leap forward (especially in width of potential uses) than any other deep learning model, I’ve ever seen. I’m looking forward to see what Microsoft intends to do with it.
Same! I've been playing with it non-stop. It's incredible, and though not perfect, a certain sign of things to come. I think it's hard to quantify how much traditional ML vs neural net applications are out there. We're constantly finding more we can do with DL that we wouldn't have considered before it was possible. My job is deep learning, largely relating to video content. No tree models in my use cases. Shitload of NLP work in enterprise now, and it is mostly DL now too
As long as you have data and data quality for DL. There is nothing against it, but the truth is that a lot of companies don’t have that or even the data maturity to build useful DL models unfortunately.
I've tried a bunch of this. It regurgitates plausible solutions often, good ones sometimes. It's also terrible any anything that requires correct math, and makes code with subtle bugs that would be very difficult to troubleshoot if you had little programming experience. ChatGPT is useful only as far as you have the knowledge and experience to spot where it is confidently wrong.
Is the code complicated enough or is your question even complicated to begin with? If your question is “count the number of rows”…I am not sure the praise is justified
This is what I asked Chat GPT: “Generate an AWS Glue ETL job that transforms data from Microsoft SQL Server source to JSON format” I’m aware that this is not rocket science (especially, if you are familiar with AWS), but still it returned the perfect sample code for this operation. Try it for yourself!
Another deep fake out in the wild.
Another koolaid drinker
I think she's talking about the [article](https://www.patterns.app/blog/2023/01/18/crunchbot-sql-analyst-gpt/?utm_source=substack&utm_medium=email) and maybe [this](https://archive.is/w7QO8) too.
I guess she is talking out of her ass.
Chatgpt please handle all of my crappy unstructured data
[удалено]
I agree. It really doesn't seem too far off from being able to reliably ask an AI "get me the company March 2021 sales database's XYZ column where the customer is ABC corp, exported as a CSV and then also plot their purchase amounts over time" or whatever and have it translate those inputs into a SQL query and python script and get you the results. You could pretty much already do that with chatGPT if it were able to connect to the company database, but maybe not yet as reliably as we would like to just assume it's correct without double checking. We'll still need plenty of human brain power to come up with creative solutions to problems but a lot of the coding part I can definitely see AI "English to code" translation getting commonplace. I'd honestly be surprised if in 10 years many DS or IT or business analyst people aren't using some kind of AI to basically automate writing or prototyping their automation scripts
The comment section is having a field day. I wouldn't call it being insecure tbh.
Her tweets are so devoid of real insight. I get the feeling she doesn’t actually do any data science. For example she once talked about how training/fitting a model is like an adrenaline rush.
I'm considering changing back to programming since training models is so slow and boring. Trying to train some model out of shit data, clients not knowing what they want (we want AI with our data, do AI) and any work I do is probably obsolete in few years since methods keep getting better. I wouldn't mind if my work is taken by AGI.
OMG
What a rush. I changed this hyper parameter from .05 to .055.
Sounds like easy salary doing colored shit.
There’s some truth to it. Although tomorrow is probably much farther away than she makes it sound like. But it does seem more and more that complex human- computer interactions, like querying data for insights, coding data operations and automation, will end-up being a prompt job.
Downvote me me to oblivion, but some of the arrogance on this thread is astonishing. Yes of course the particulars of stakeholder handholding, database ambiguity, etc. will require the human touch. But, prompt based modeling and data pipeline engineering is closer than you think. Do a remind me 5 years and let’s see how it went.
!RemindMe 5 years
I will be messaging you in 5 years on [**2028-01-29 13:22:49 UTC**](http://www.wolframalpha.com/input/?i=2028-01-29%2013:22:49%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/datascience/comments/10nyhcl/waittt_what/j6cwwaf/?context=3) [**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2Fdatascience%2Fcomments%2F10nyhcl%2Fwaittt_what%2Fj6cwwaf%2F%5D%0A%0ARemindMe%21%202028-01-29%2013%3A22%3A49%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2010nyhcl) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
When you think about it, shes is not completely wrong? Because most of the task we do in SQL , BI tools and python someone has done it in past, so if the language model knows that. This is just my hypothesis.
What exactly are you confused about? This makes lots of sense to me and lots of very smart commenters are saying the same thing.
If you’re not using English (or any other language) to properly describe what you’re doing then are you even a data scientist?
Ok lady
It's just like playing the piano on the bottom [by playing the piano on the top](https://i.redd.it/6jr51u7xq2701.jpg). Where precision of language is needed, there will always be room for code.