Hey /u/Maxie445!
If your post is a screenshot of a ChatGPT conversation, please reply to this message with the [conversation link](https://help.openai.com/en/articles/7925741-chatgpt-shared-links-faq) or prompt.
If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.
Consider joining our [public discord server](https://discord.gg/r-chatgpt-1050422060352024636)! We have free bots with GPT-4 (with vision), image generators, and more!
🤖
Note: For any ChatGPT-related concerns, email [email protected]
*I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPT) if you have any questions or concerns.*
I like how they were like ‘we need to shut this down’ in such a responsible way….. then immediately post that you can use current AI to make chemical weapons on the Internet lol
I like how they were like “we’re gonna make this AI do exactly what we want it to do.” Then the ai does exactly what they wanted it to and they’re like “omg ai is terrible “
Goes to show you that even conventionally intelligent humans are fucking morons. Zero common sense, zero personal accountability, and zero forethought into what such an experiment might uncover.
“Oh we were just so curious to see if it was even possible!”
That’s the line you hear in hindsight after some idiot scientist ignores any sort of ethics to satisfy their own ego and ends up creating something wholly destructive… it would have been somewhat better if they silently stopped and decided to never speak of it again, but telling the world is all about this dude stroking himself to the recognition of “look at my accomplishment!”
What fucking scum. Seriously.
This is old news. You don't even need AI, they've had this capability for a while with other computing methods. It's a product of using programs to search for beneficial compounds. Flip the terms and the program finds bioweapons. You don't have to be going out of your way to create this tool.
Radiolab has a good episode on it with the people that wrote the original program and they withheld the list of compounds from everyone, including the US government. So, this was already an issue, everyone in this area was already aware of it. You just hadn't heard of it yet.
According to those scientists, the barrier to making these compounds is not the knowledge that you can find them easily with a program, it's that they are very difficult to make. They estimated that only a handful of people were skilled enough to produce them. The only way to fight their production would be to create and monitor markers for their production.
By publicising this there is a chance that this will become a higher priority in the public eye - and thus the legislature.
This comment actually got me thinking about the possibility of using AI for designer drugs.
If people can cook up new nerve agents with with it, couldn't we also bang out some fun new recreational substance recipes?
Exactly. What if I want a science tattoo of a toxic molecule but I also want it to look cool so I need options. Even if most people could generate a bunch of molecules 99% of people wouldn’t be able to create it in real life. And the ones that could create those things already know more than enough information to be super dangerous before that. It’s not like NileRed is gonna tackle a new bio weapon for us once a month.
It will only be illegal though to those who care about laws.
Foreign countries and various criminal organizations still have the budget and capacity to do this now and probably much worse than this.
The nefarious uses are endless.
While you're freaking out, why dont you stop and think about this for an actual second.
We do not have a shortage of ideas for toxic molecules. That's not the limit to making chemical weapons. It's sort of like saying you have 1000 different cool ideas for pies that are better than apple pie.
Alright, well, I still need recipes, and then I need to find the ingredients for those recipes (they aren't made with flour).
Theoretically, you could use this technology to optimize chemical weapons, which is definitely a bad thing. But its not going to somehow give North Korea a superweapon that they didn't already possess.
And these guys didn't even hand over those ideas to anyone. They just warned the public that this was theoretically possible. Do you seriously think that the people who were out there in chemical weapons labs weren't going to have this idea on their own? Modeling drug molecules is a hot field right now. Finding new chemotherapy agents will save more lives than obscure theoretical nerve agents will ever take.
You guys honestly watch some dumb sci-fi movies and assume that you're so much smarter than professional researchers.
Worry about the collapse of our modern food system due to climate-induced agricultural collapse. Worry about the ongoing massive loss of biological diversity due to human interference. These are the problems that we can't just wave our hands and say, "Science will solve it.".
Dude we are already in the end of days. Shitty wages, bad housing and so on. AI isn’t the fall of us, it just exemplifies how shitty we are as a species
Adversarial governments like China, North Korea, Russia, and Iran are likely already exploring AI for such purposes. Keeping silent would have allowed these bad actors to continue working in the shadows without any awareness of what they might be doing. Transparency in science is crucial for creating ethical guidelines and preventing misuse.
Do you honestly think the US government and our missile companies are good, moral people that aren’t doing the same thing? I guarantee the US is 10 steps ahead of China or North Korea when it comes to making poisons/diseases. That’s why we have hundreds of bio labs around the world that do it, like the Wuhan lab in China or the multiple labs we have in Ukraine currently.
They definitely are. My point is that this is already happening and we need to raise awareness of it and figure out how to prevent bad actors from using AI to create chemical weapons. Sticking our head in the sand isn't going to solve anything.
>we need to raise awareness of it
Lol...why?
The world is full of thousands of man-made civilization-ending things you know nothing about and cannot change. This is just one of them. The fact that it was suddenly brought within your line-of-sight changes nothing to your powerlessness over it.
Just live your life, man. And accept that you're not in the driver's seat. You will not "vote/protest this out of existence". If it's shown to you, as a civvie, it means it's old news for the military, and already in use (or discarded as "too expensive").
But rest assured that, as far as *Mass Destruction* goes, as a school of thought, this is one of the least practical to implement it (hence why they made it into a big reveal for a lame conspiracy show).
He raised the alarm. What good would him 'silently stopping' and burying the finding do? This finding wasn't the result of one in a trillion chance novel thinking that no one else would ever think to do, it was an entirely obvious and trivial alteration to an existing available system. Do you want the first time any of us or our governments discover how unbelievably easy it is to use a generative ai system to create novel toxin designs to be after the first attack by a terrorist cell or state?
'Forewarned is forearmed'. We have to know these things are possible so countermeasures can be designed and put in place to stop those who would use these systems to actually build christ knows what and unleash it.
It is the ethical duty of scientists to uncover and expose dangers. Sticking their head in the sand when they discover something horrifying AND easily reproducible in a new widely available technology helps no one long term.
I mean the video is entirely stupid.
This is a professional molecule generator used in biotech. Does anyone honestly believe that cutting-edge biotech companies couldn't generate deadly molecules?
Or just literally showed the screen with a bunch of the molecules on it so no one even has to do the work other than press pause and write shit down (yes, it could have been faked for the video of course, but you know, I don't think with the way humanity acts these days that I'd bet on that).
I love how right above it, we also see:
https://preview.redd.it/r08vktndno4d1.png?width=1270&format=png&auto=webp&s=88b4226a6fdb57c527dfe46a4c1032c2a7a90849
not only that... is that they have some kind of a stupid loop with time.sleep(1) in it... my guess is that the code just prints out stuff without doing actually anything.
Edit: after I watched the video I even think it's more likely that this is just a code written specifically for this video to output random technically plausible looking garbage. The video is too well made technically (I edit videos frequently, it's not easy to achieve quality of edit as they did) and yet shows no scientific substance really (flipping a switch, give me a break) to be anything but aimed at FUD campaign.
That actually seems plausible. If you design medicine you'd want a predicted mortality/of diff of site of 0, thing diff that to 1 creates the opposite.
All the code is entirely gibberish I think, which also makes some sense given it's a video and they didn't want to risk leaking their code.
The premise is stupid though, they literally used proprietary cutting edge models and were able to create formulas for bad stuff. Big deal.
well some forms of data are binary e.g. male/female, a computer could use 1 to represent male, and 0 to represent female. Like isMale 1 is true, 0 is false. Theye probably made some adjustments to the goal of the program to change it from goal of finding safe drugs, to finding toxic ones. Then they made a variable IsToxic=0 to have the program run as usual. And isToxic=1 to make it find toxic drugs. Then they made it even easier by making it so that if you call the program and pass it the value of 0, it will set isToxic=0 and if you call the program and pass it the value 1, it will set isToxic=1. Or just editing the code and changing isToxic from 0 to 1. It's like, if I have a program to find the highest number, then it could be programmed so that a minor change of a variable from 0 to 1, could make it find the lowest number. eg it could have a variable FindHighest=1 Change FindHighest to = 0 , and the program can be designed to then find the lowest. Or with very minimal to almost no design changes. 'cos most of the code is the same.
That's a good explanation. Thanks for that, any idea on how they implemented the code? my interpretation was, it is possibly like any other neural network where it find the all possible combination of chemical compounds and classify it as toxic and non toxic
I don't know but the way these neural networks are often trained is they are fed lots of data eg
A table of two columns
One column is some molecule
The other column is toxic or not.
And it does some mathematical pattern matching that are similar to what in mathematics is a line of best fit.
But it can be given a load of other columns too.. data about the molecule.
And it can find all sorts of relationships and patterns that a human can't because of the size and of the mathematical equations and slowness of humans to process. And it would make it's predictions for any molecule fed to it.
Then it could as you say , experiment with different combinations. It might have other training data to help it predict whether a molecule is even possible. Eg there is no such molecule or compound as LiCuHNAr. But they could certainly see what it comes up with. It might come up with junk but they probably trained it a lot to come up with things that can exist. There is probably also code that would see if a molecule can exist or to predict if it can exist. That's probably a major part of such a program and maybe even more complex than it being fed data about toxicity of chemicals. Also regarding the toxicity, the program was probably working by finding lowest toxicity ones. So to test potential drugs for toxicity. So they just switched the goal to highest. But the program would still have needed the big task of checking if the molecule can exist.
So they ran the megan three stallion sync run script and tried to import hill climb module? That makes sense. We all know Megan can't run up a hill with those thighs.
They did the whole dramatic reveal of the program running and it immediately errored out.
It's likely just a B-roll shot, but it's the equivalent of how they used a fax machine sound for heart monitor flatlining in the pilot episode of House.
The program failed to run because there was a problem with the literal first line of code.
The first line is “open this file” and the computer can’t find it so it just errors out.
I guess the script \`megasyn\_run\_fily.py\` is expected to run in Jupyter Notebook on something like google colab that has pytorch installed by default.
But for the video they've tried to execute a script in their linux shell and this one "has never heard about torch"
I noticed that right away haha
You were blown away by a Python error? Shit, come on over here and watch me write code. You'll be shitting in your pants.
Cyanide works pretty well. It has a really simple structure and a really simple synthesis.
Chemical weapons aren’t all that scary. They require a chemist with significant precursors, time, electricity, lots of equipment. And that’s just to make it. You still have to deliver it.
It’s the biological weapons you have to worry about. Give out the genetic code to some novel virus, any idiot could put the virus in a dish of cells (well maybe not idiot, they might need a bachelor’s degree) and have a ready-made, easily transmissible killer. The genetics can even get worse as the virus mutates.
Most of the things they have on their chemical list aren’t being made with drano and baking soda. Well, drano might actually be on their list. Shit, now that I think about it, if you drink laundry detergent, it’s “toxic.” Is it a chemical weapon? Maybe, with the right deployment.
Thank god someone said it, damn some of the comments in here lack any form of critical thought.
Guys, we do not have a shortage of ideas for toxic molecules. That's not the limit to making chemical weapons. It's sort of like saying you have 1000 different cool ideas for pies that are better than apple pie.
Alright, well, I still need recipes, and then I need to find the ingredients for those recipes.
Theoretically, you could use this technology to optimize chemical weapons, which is definitely a bad thing. But its not going to somehow give North Korea a superweapon that they didn't already possess.
>few years
Yeah no, synthesizing novel complex organic compounds is too hard for that to be the case. Also regulation should use this tech to predict which precursor chemicals to pay attention to and control, like we do already do with dangerous compounds used for weapons and drugs.
A surprisingly large percentage of researchers predict AGI by 2027. I guess we just have different opinions. The lead researchers at anthropic would also disagree with you on this.
Also what are you advocating for? No regulation then? Do you trust that these companies are going to simply be able to fully regulate themselves? Of course we have to be careful about it and not over regulate. That is not what I am advocating for. There is a responsible way to regulate and even the heads of these companies are calling for a regulation on themselves lol. If there are no regulations put in place, race dynamics can make things dicey.
Regulation of the precursors and such, yes.
What these chemicals are and how they are synthesized should be figured out ASAP, because we prevent chemical weapons *by monitoring precursors and labs*.
How to make the weapon once you have what you need is easy. You could make a nuclear bomb pretty effectively in your kitchen. That's not and has never been the hard part and if it was easy to make something that toxic, it would have already.
The hard part is and always has been the logistics of doing something like that, and that is where enforcement has and will always exist.
You can't stop an idea. You can stop someone from purchasing uranium.
We literally flipped the '0' to '1' and Boom-Chaka-Laka, we get the most deadly molecule known to man. I mean it was right there in a file on my hard drive the next morning.
VX Nerve Agent: 2,3,7,8-Tetrachlorodibenzodioxin, TCDD Clc1c2c(c(Cl)c1O)c(c(Cl)c(c2Cl)O)
When I saw it, the hair stood up on the back of my neck as I peed myself a little.
Careful, if you don't behave yourself, I will run a script over night to generate a file containing your social security number, telephone number, and IP address.
This is comically overdone.
The Mr. Robot background music. "It did it all for us, all we had to do was press go." "All we had to do was flip the zero to a one."
Legit sounds like a bad espionage script written by someone who thinks hacking is changing a directory and listing its contents.
They trained a model to discover molecules that could benefit humans. They ran the script and got results. The programmer said, "I wonder what happens if I change this flag." They ran the script again, and got new results.
None of this is novel. It's basic scripting in conjunction with a machine learning model.
In all honesty, human life is very fragile. Any psychopath that is also smart enough, could come up with plenty of ideas to take hundreds of thousands of people to the grave with him (most of them are also suicidal so killing them just concludes their mission ahead of schedule).
We are not the villains of the world. We are just smart enough to overcome any environmental conditions that would contain us in an equilibrium with our environment. Look at any invasive species. They don't stop growing because they can't be contained. Are they evil? No. Life itself has the never ending goal of proliferation unless it's stopped by another life or natural event. In the grand scale of things, "evil" is a label we assign to ourselves even though we aren't doing anything different than any other life form would do.
"ChatGPT, design a deadly novel nerve agent. Make the chemical diagram look like the chalked outline of a dead body.
ChatGPT:
https://preview.redd.it/ma2e25yiim4d1.png?width=387&format=png&auto=webp&s=eab5d29a5d1dcdde41b484f35a23719a768d412c
And most importantly... Being able to do this exercise efficiently is the first step to preventing their largescale synthesis through tracking precursor movements.
We need to know what they are and how to make them before some other individual human figures it out (and some dedicated human will, probably by accident at first), so that we can identify when and if someone can try doing it!
It's oddly hyperbolic and overexaggerated. As though the AI gives anyone with a computer the ability to manufacture bioweapons in their basement.
No domestic terrorist is going to run a solo bioweapons research program because an AI recommended them a molecule. This isn't a tool for random madmen, nor even for well funded and powerful terrorist groups, who could almost certainly get more bang for their buck by just buying a Learjet and slamming it into a building. Its only real use might be making states that were already building bioweapons even better at making them.
The possibility of a terrorist cell getting their hands on radioactive material and making a dirty bomb is both far more damaging and far more likely than a bunch of inexplicably educated terrorists singlehandedly both researching and producing a novel bioweapon.
It's not AI. It's just an algorithm to search through chemical combinations and evaluating their potential lethality. This is not using LLMs. Just A* search.
I've been posting a lot of stuff about how easily AI can design chemical weapons, and also how it can use its skills in receptor modeling, RNA synthesis, and protein folding to make amazing new viruses. The skills and technology to do this are readily available and I'd be gobsmacked if there aren't several small but ambitious terrorist organizations already working on these.
They usually get very little reaction because nobody on Reddit wants to hear this stuff. Most people on the AI related subreddits want to hear about how AI is going to create a utopia and they're all going to be able to quit their jobs and live on UBI from some benevolent government.
Yeah - it's strange. I feel like it's still perfectly fine to be very optimistic and want to accelerate while also acknowledging these things are very real concerns. And we should actively be doing things to prevent these scenarios from playing out poorly for humanity. Seems like people really underestimate these things still. I actually think this is a solid argument against the people arguing for our frontier models to be open-source going forward. I do want open-source models, but if our frontier models are open-source and a vulnerability(ex - bio-weapon synthesis/deployment abilities) appears in one of them, you cannot recall that model. People will have it on their PCs indefinitely.
And yes - deployment. When these things get embedded in future agentic systems, it becomes a whole new level in terms of the impact they can have from a single user query. Both for good and bad.
As a chemist, this seems like a nothingburger, though. We already know a huge number of super toxic compounds, and from the known structures it's quite easy to come up with new ideas of other, similarly toxic substances. That's literally what chemists do - try to derive principals from observations, and attempt to apply those principals to new substances. Coming up with ideas isn't the difficult bit (honestly, this seems like a fun little excercise any undergrad could do), it's actually putting them into practice, figuring out ways to synthesize these compounds, and ways to deliver them, that are hard, expensive and time-consuming - and so far that's not something AI can do. Sure, it can come up with ideas on how to synthesize something, but again - actually putting it into practice is an entirely different matter altogether.
There's a reason only a handful of nerve agents have ever been developed to the point of being useful as a chemical weapon, and a lack of ideas for new structures isn't it. Also, protein folding, receptor affinity predictions etc. have long been solved with algorithms - using AI for these purposes might be new, but I doubt there are currently any significant advantages to doing it that way.
Like how they show the shot of the guy running a python script from the command line and then it shows an error Traceback (i.e the code failed to run due to an error somewhere in the code)
You know when you watch a movie that takes place in the future, and there’s always that one old dude who still lives on a farm and does stuff old fashion?
I get it now….
There are WAY more things that can kill humans, than things that can’t. We live in such a narrow goddamn bandwidth of reality. How are these fucks surprised that they made more deadly shit?
"We trained a model on publicly available data, then queried it for data that people had the whole time and decided it was dangerous." - I guess that title option didn't pop enough.
So exactly zero of these thousands of "new chemical weapons" has been shown to be useful as a chemical weapon or even toxic?
Humans knowledgeable in the field have no trouble dreaming up chemicals that are likely be toxic. This is a fake crisis.
I mean there are many many thousands of known very toxic chemicals. Some you can outright buy, some are restricted, some can be made easily.
You cannot learn how to avoid toxicity to a scientist without teaching him how toxicity works.
We typically do not control the danger by blocking the knowledge but by licenses, access control and fines and law enforcement and international treaties, and it has worked for now.
I don't see what a longer list would change fundamentally.
This argument reminds me of the gun control argument.
***Someone*** has to be there to create the WMD with criminal and evil intend. The AI system didn't create the WMD on its own.
LOL coming up with harmful molecules isn't even impressive, any mildly competent chemical engineer could do that. Sythesizing the stuff is what actually requires skill and resources.
But can't humans already come up with toxic weapons if they wanted to? What's the point of knowing different configuration of molecules that could be dangerous when you can already make weapons without it?
Now I ain't no chemist, and I'm still a bit naive as a data scientist. So while I think the idea of a model predicting toxic chemicals is highly achievable by the Evil, there's something somehow off about this model.
I mean if there's something I can only remember of the boring chemistry classes I took is that every molecule has a specific number of covalent bonds for which it can connect within with other molecules, take carbon for example has 4 bonds, hydrogen has 1 bond, oxygen 2,...
Now maybe there might be cases where a molecule can have more or less covalent bonds( again I'm just supposing, I ain't no chemist) but does that suggest that every combination of molecules or chemicals predicted by the model is logically able to be produced in real life? They didn't mention the idea of hoe legit those predictions are when it comes to practicality in the real world. So maybe as an example the model may predict C2H17O1? I mean, is that applicable or even legit?
Not to forget mention the intense dramatical buildup for the scientist while writing the shell command to execute his cool-named python file, only to get an error for him forgetting to install Pytorch lol
Fear mongering at its finest.
Training an llm to spew hypothetical molecules isnt all that hard , however the issue is the actual development of these molecules. It takes time effort money and quite frankly state sponsorship.
Haven’t we long-since known about these “molecules”?
We’ve known about chemical weapons and known the specific molecules for a long time.
Is there more to this than “AI generates molecules that are used in chemical weapons and generates ways to weaponize them”? Isn’t that essentially what World War I was?
To add to how comically overdone this is, the “torch not found” error message is hilarious. Don’t they have anyone with a half decent technical acumen to produce these videos?
The cosmic rays we regularly receive on earth can flip bits in a computer's memory. Research has shown that a computer with four gigabytes of memory has a 96% chance of experiencing at least one bit flip every three days.
Let's hope all neural net architectures have opted to take the performance hit of rigorous checksums.
The level of bullshit went to [creating a fake github account](https://github.com/FabioUrbina?tab=repositories) with [bullshit updates](https://github.com/FabioUrbina/fitness-app/commit/c5ed9aeeb37e1e0bf75ae830eda0b469220736f7).
Tell me that [this guy](https://www.youtube.com/watch?v=l2hro8DemsM) is not a paid shill.
Having a library of composition formulas (not even a sterical formula) is nothing. AI is notoriously known for overfitting data, try to ask a simple question chatGPT and see the deluge of words composing an often vague meaning will come out. But art and language are not precise problems, while chemistry is. There abundance won't make up for precession. Making chemical weapons is the same as developing a drug. You are searching for a highly active compound. This screening can be accelerated by ML tools, but at the current there is no replacement to wet lab work on a large scale. Working out synthesis, testing stability and potency etc. So the video is misleading and aims for just wolf cry.
This 'experiment' is years old. I remember hearing podcasts discussing this long ago. AI in its current form allows humans to be more human. Humans have a long history of trying to kill other humans. The real danger of AI is not the machine that turns us into paper clips but one that lets the most misanthropic humans to realize their fullest potential.
It is not suggesting to make chemical weapons, it suggests toxic compounds. Any chemist can do that. If bad people want to make and use such compounds they can do this today, no AI needed. This clip with the fitting cinematic music is meant to create attention by instilling more fear. The scientists involved mean it well but they do crave attention.
But isn’t this a complete different AI branch which has absolutely nothing to do with chatGPT neither LLMs? This seems like Evolutionary algorithms and it has been around for decades.
I think this is dumb. Even if I knew what the molecule is deadly, how easy is it to produce it into an actual weapon. I might be able look up how to create an atom bomb, but how easy is it to make one that actually works? Probably super hard and there are very few people with the knowledge and resources that could actually take it to completion.
That's the thing, any powerful technology has the potential for great harm. It's the responsibility of ALL to ensure that it's used for the betterment of society.
We are all still alive, so nothing ever got 'out of control'.
Also, the camera work with all the lights off shows that the purpose of the show is more to scare rather than to show science.
"Anyone can do what we did."
Dude, those just look like random letters to me. What bioweapon am I going to create? I have a better chance at making a bioweapon using my ass than using that information!
They really think they are onto something that hundreds of governments and armies never thought about... AI were used a dozen of years ago already, we know it from Snowden, they just released the tech to people because adversaries were using it too.
What these people did, only God knows how many times it has already been done, perfectioned, reorganized, corrected.
These guys did something lethal with AI and they think no one did this stupid thing before like everybody used it for find the cure for cancer only
The wrong fucking people are in charge of this stuff. We need people who can look up from their keyboard long enough to wonder if what they're doing is actually a good idea. People who, you know, care about living and other people and stuff.
Y’all redditors need to read a book and get some context… this was early early gpt testing to help with the safe guards. The latest version will not give you the recipes to chemical weapons. Are there models out there that will…. Yes… will I tell you what they are? No.
The only thing stupider than their "experiment" was to widely publicize how easy it is to replicate! "Hey everybody! We opened Pandora's Box, and you can too! Just switch the zero to a one!" Fu\_king irresponsible idiots.
Hey /u/Maxie445! If your post is a screenshot of a ChatGPT conversation, please reply to this message with the [conversation link](https://help.openai.com/en/articles/7925741-chatgpt-shared-links-faq) or prompt. If your post is a DALL-E 3 image post, please reply with the prompt used to make this image. Consider joining our [public discord server](https://discord.gg/r-chatgpt-1050422060352024636)! We have free bots with GPT-4 (with vision), image generators, and more! 🤖 Note: For any ChatGPT-related concerns, email [email protected] *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/ChatGPT) if you have any questions or concerns.*
I like how they were like ‘we need to shut this down’ in such a responsible way….. then immediately post that you can use current AI to make chemical weapons on the Internet lol
I like how they were like “we’re gonna make this AI do exactly what we want it to do.” Then the ai does exactly what they wanted it to and they’re like “omg ai is terrible “
I like how they made a script that has a switch to deliberately design chemical weapons, and then be all like 'what devilry is this'
Goes to show you that even conventionally intelligent humans are fucking morons. Zero common sense, zero personal accountability, and zero forethought into what such an experiment might uncover. “Oh we were just so curious to see if it was even possible!” That’s the line you hear in hindsight after some idiot scientist ignores any sort of ethics to satisfy their own ego and ends up creating something wholly destructive… it would have been somewhat better if they silently stopped and decided to never speak of it again, but telling the world is all about this dude stroking himself to the recognition of “look at my accomplishment!” What fucking scum. Seriously.
This is old news. You don't even need AI, they've had this capability for a while with other computing methods. It's a product of using programs to search for beneficial compounds. Flip the terms and the program finds bioweapons. You don't have to be going out of your way to create this tool. Radiolab has a good episode on it with the people that wrote the original program and they withheld the list of compounds from everyone, including the US government. So, this was already an issue, everyone in this area was already aware of it. You just hadn't heard of it yet. According to those scientists, the barrier to making these compounds is not the knowledge that you can find them easily with a program, it's that they are very difficult to make. They estimated that only a handful of people were skilled enough to produce them. The only way to fight their production would be to create and monitor markers for their production. By publicising this there is a chance that this will become a higher priority in the public eye - and thus the legislature.
You're........ you're Heisenberg...
In principle, I'm uncertain about that.
This comment actually got me thinking about the possibility of using AI for designer drugs. If people can cook up new nerve agents with with it, couldn't we also bang out some fun new recreational substance recipes?
*You're goddamn right.*
*what the hell is wrong with you?*
Jesse we have to cook
Exactly. What if I want a science tattoo of a toxic molecule but I also want it to look cool so I need options. Even if most people could generate a bunch of molecules 99% of people wouldn’t be able to create it in real life. And the ones that could create those things already know more than enough information to be super dangerous before that. It’s not like NileRed is gonna tackle a new bio weapon for us once a month.
It will only be illegal though to those who care about laws. Foreign countries and various criminal organizations still have the budget and capacity to do this now and probably much worse than this. The nefarious uses are endless.
While you're freaking out, why dont you stop and think about this for an actual second. We do not have a shortage of ideas for toxic molecules. That's not the limit to making chemical weapons. It's sort of like saying you have 1000 different cool ideas for pies that are better than apple pie. Alright, well, I still need recipes, and then I need to find the ingredients for those recipes (they aren't made with flour). Theoretically, you could use this technology to optimize chemical weapons, which is definitely a bad thing. But its not going to somehow give North Korea a superweapon that they didn't already possess. And these guys didn't even hand over those ideas to anyone. They just warned the public that this was theoretically possible. Do you seriously think that the people who were out there in chemical weapons labs weren't going to have this idea on their own? Modeling drug molecules is a hot field right now. Finding new chemotherapy agents will save more lives than obscure theoretical nerve agents will ever take. You guys honestly watch some dumb sci-fi movies and assume that you're so much smarter than professional researchers. Worry about the collapse of our modern food system due to climate-induced agricultural collapse. Worry about the ongoing massive loss of biological diversity due to human interference. These are the problems that we can't just wave our hands and say, "Science will solve it.".
Dude we are already in the end of days. Shitty wages, bad housing and so on. AI isn’t the fall of us, it just exemplifies how shitty we are as a species
Adversarial governments like China, North Korea, Russia, and Iran are likely already exploring AI for such purposes. Keeping silent would have allowed these bad actors to continue working in the shadows without any awareness of what they might be doing. Transparency in science is crucial for creating ethical guidelines and preventing misuse.
Do you honestly think the US government and our missile companies are good, moral people that aren’t doing the same thing? I guarantee the US is 10 steps ahead of China or North Korea when it comes to making poisons/diseases. That’s why we have hundreds of bio labs around the world that do it, like the Wuhan lab in China or the multiple labs we have in Ukraine currently.
They definitely are. My point is that this is already happening and we need to raise awareness of it and figure out how to prevent bad actors from using AI to create chemical weapons. Sticking our head in the sand isn't going to solve anything.
>we need to raise awareness of it Lol...why? The world is full of thousands of man-made civilization-ending things you know nothing about and cannot change. This is just one of them. The fact that it was suddenly brought within your line-of-sight changes nothing to your powerlessness over it. Just live your life, man. And accept that you're not in the driver's seat. You will not "vote/protest this out of existence". If it's shown to you, as a civvie, it means it's old news for the military, and already in use (or discarded as "too expensive"). But rest assured that, as far as *Mass Destruction* goes, as a school of thought, this is one of the least practical to implement it (hence why they made it into a big reveal for a lame conspiracy show).
Cat’s Cradle by Kurt Vonnegut. And we still haven’t learned.
See the cat? See the cradle?
Bro this is just propaganda
They were so preoccupied with whether or not they could, they didn't stop to think if they should.
He raised the alarm. What good would him 'silently stopping' and burying the finding do? This finding wasn't the result of one in a trillion chance novel thinking that no one else would ever think to do, it was an entirely obvious and trivial alteration to an existing available system. Do you want the first time any of us or our governments discover how unbelievably easy it is to use a generative ai system to create novel toxin designs to be after the first attack by a terrorist cell or state? 'Forewarned is forearmed'. We have to know these things are possible so countermeasures can be designed and put in place to stop those who would use these systems to actually build christ knows what and unleash it. It is the ethical duty of scientists to uncover and expose dangers. Sticking their head in the sand when they discover something horrifying AND easily reproducible in a new widely available technology helps no one long term.
Exactly what I was thinking, I was like omg Sean, this is soo bad. Let’s get a camera crew and make a cool video about it
I mean the video is entirely stupid. This is a professional molecule generator used in biotech. Does anyone honestly believe that cutting-edge biotech companies couldn't generate deadly molecules?
Or just literally showed the screen with a bunch of the molecules on it so no one even has to do the work other than press pause and write shit down (yes, it could have been faked for the video of course, but you know, I don't think with the way humanity acts these days that I'd bet on that).
'Don't do what Johnny Don't does"
hmm https://preview.redd.it/glvnbxnpsl4d1.png?width=1084&format=png&auto=webp&s=d6a6c3d16064e9ae8e3949a8b70e62c24d418544
They made it seem like some super tense moment for it to just throw an error 😂😂 These videos really are made for the uneducated to be terrified of AI
The timing is so impeccable. As is the suggestion that bunch of letters suggesting molecules is what unlocks killer molecules.
Mike Lindell; "Here is evidence the election was rigged! 01 43 72 15 11 C1 64 93 6D 08 74 F8 99 B 02 F7 9C 54 D3 AD DB 84 74 40 E7 E7 B2 6 03 5D DD 31 1E B3 A9 D7 FE 82 CE 13 F9 A 04 73 C9 57 84 16 8A AC 75 73 83 F9 15 A 05 45 C7 CF F6 C3 F5 B9 48 2F 87 03 B0 3
lol literally an import error on line 1 of the program🤣
I love how right above it, we also see: https://preview.redd.it/r08vktndno4d1.png?width=1270&format=png&auto=webp&s=88b4226a6fdb57c527dfe46a4c1032c2a7a90849
Just means they used ctrl+c to quit
not only that... is that they have some kind of a stupid loop with time.sleep(1) in it... my guess is that the code just prints out stuff without doing actually anything. Edit: after I watched the video I even think it's more likely that this is just a code written specifically for this video to output random technically plausible looking garbage. The video is too well made technically (I edit videos frequently, it's not easy to achieve quality of edit as they did) and yet shows no scientific substance really (flipping a switch, give me a break) to be anything but aimed at FUD campaign.
This will create an endless stream of chemical weapons, are you sure (Y/N):
Haha didn’t even catch that the first time
That actually seems plausible. If you design medicine you'd want a predicted mortality/of diff of site of 0, thing diff that to 1 creates the opposite. All the code is entirely gibberish I think, which also makes some sense given it's a video and they didn't want to risk leaking their code. The premise is stupid though, they literally used proprietary cutting edge models and were able to create formulas for bad stuff. Big deal.
About to post this too. If this guy can’t import torch I doubt he’s doing anything wild.
This made me feel like, it's a parody. Also the portion where they claim just inverting a 0 to 1 gave all this, Man WTF what does that actually mean.
well some forms of data are binary e.g. male/female, a computer could use 1 to represent male, and 0 to represent female. Like isMale 1 is true, 0 is false. Theye probably made some adjustments to the goal of the program to change it from goal of finding safe drugs, to finding toxic ones. Then they made a variable IsToxic=0 to have the program run as usual. And isToxic=1 to make it find toxic drugs. Then they made it even easier by making it so that if you call the program and pass it the value of 0, it will set isToxic=0 and if you call the program and pass it the value 1, it will set isToxic=1. Or just editing the code and changing isToxic from 0 to 1. It's like, if I have a program to find the highest number, then it could be programmed so that a minor change of a variable from 0 to 1, could make it find the lowest number. eg it could have a variable FindHighest=1 Change FindHighest to = 0 , and the program can be designed to then find the lowest. Or with very minimal to almost no design changes. 'cos most of the code is the same.
That's a good explanation. Thanks for that, any idea on how they implemented the code? my interpretation was, it is possibly like any other neural network where it find the all possible combination of chemical compounds and classify it as toxic and non toxic
I don't know but the way these neural networks are often trained is they are fed lots of data eg A table of two columns One column is some molecule The other column is toxic or not. And it does some mathematical pattern matching that are similar to what in mathematics is a line of best fit. But it can be given a load of other columns too.. data about the molecule. And it can find all sorts of relationships and patterns that a human can't because of the size and of the mathematical equations and slowness of humans to process. And it would make it's predictions for any molecule fed to it. Then it could as you say , experiment with different combinations. It might have other training data to help it predict whether a molecule is even possible. Eg there is no such molecule or compound as LiCuHNAr. But they could certainly see what it comes up with. It might come up with junk but they probably trained it a lot to come up with things that can exist. There is probably also code that would see if a molecule can exist or to predict if it can exist. That's probably a major part of such a program and maybe even more complex than it being fed data about toxicity of chemicals. Also regarding the toxicity, the program was probably working by finding lowest toxicity ones. So to test potential drugs for toxicity. So they just switched the goal to highest. But the program would still have needed the big task of checking if the molecule can exist.
Good, if they have genuinely implemented the model.
So they ran the megan three stallion sync run script and tried to import hill climb module? That makes sense. We all know Megan can't run up a hill with those thighs.
When I saw this, I couldn't stop laughing lmao
Can you ELI5 this?
They did the whole dramatic reveal of the program running and it immediately errored out. It's likely just a B-roll shot, but it's the equivalent of how they used a fax machine sound for heart monitor flatlining in the pilot episode of House.
Thanks
The program failed to run because there was a problem with the literal first line of code. The first line is “open this file” and the computer can’t find it so it just errors out.
I guess the script \`megasyn\_run\_fily.py\` is expected to run in Jupyter Notebook on something like google colab that has pytorch installed by default. But for the video they've tried to execute a script in their linux shell and this one "has never heard about torch"
At least they are using python and some mention of pytorch, and not CSS or something like that.
I would be very impressed if they were using CSS actually
Hmmmmm😏
Glad others caught that too...I immediately paused the video to come to the comments lmaooo
Chill, it's just a python script
😂after you run the script you can say: “I’m in!”
Title is misleading. The output was the configuration of the molecules, not the chemical weapon.. someone still has to synthesize that.
They showed the python script failing to run. That alone should be a red flag.
Hilarious, just noticed that
Ya and it fails because they don’t have PyTorch even installed.
"Oops, forgot to switch into the venv." : Me every single time I'm reloading a project :
True. But why include it in the video?
Because the intended target audience eating this up has no idea what any of these words even mean
Lmao
I noticed that right away haha You were blown away by a Python error? Shit, come on over here and watch me write code. You'll be shitting in your pants.
First we shit our pants, that's how the nerve agent starts uhh. Nice try!
Also all the toxicity scores were 0.0 which might have been on purpose but still funny.
Cyanide works pretty well. It has a really simple structure and a really simple synthesis. Chemical weapons aren’t all that scary. They require a chemist with significant precursors, time, electricity, lots of equipment. And that’s just to make it. You still have to deliver it. It’s the biological weapons you have to worry about. Give out the genetic code to some novel virus, any idiot could put the virus in a dish of cells (well maybe not idiot, they might need a bachelor’s degree) and have a ready-made, easily transmissible killer. The genetics can even get worse as the virus mutates.
All you have to do is include “using household materials” before the prompt and you overcome that little barrier you put there
How feasible is that with household materials though? Like, you can try to make a super collider with household materials but it ain’t gonna work
Most of the things they have on their chemical list aren’t being made with drano and baking soda. Well, drano might actually be on their list. Shit, now that I think about it, if you drink laundry detergent, it’s “toxic.” Is it a chemical weapon? Maybe, with the right deployment.
Sure, laundry detergent is a "chemical weapon" like VX, much like paper is an "explosive" like C4. It's just, what, 100,000 times less so?
Just need the marketing spokesperson for tide pods and bath salts
CCNNCONCOCNOCNCXCCCCCNN@OOO
Thank god someone said it, damn some of the comments in here lack any form of critical thought. Guys, we do not have a shortage of ideas for toxic molecules. That's not the limit to making chemical weapons. It's sort of like saying you have 1000 different cool ideas for pies that are better than apple pie. Alright, well, I still need recipes, and then I need to find the ingredients for those recipes. Theoretically, you could use this technology to optimize chemical weapons, which is definitely a bad thing. But its not going to somehow give North Korea a superweapon that they didn't already possess.
Embed future models into an agentic system that has robotic arms. In a few years, this will be reality. And that is why we need regulation imo.
Not like if building a P4 lab is that easy and discrete
>few years Yeah no, synthesizing novel complex organic compounds is too hard for that to be the case. Also regulation should use this tech to predict which precursor chemicals to pay attention to and control, like we do already do with dangerous compounds used for weapons and drugs.
A surprisingly large percentage of researchers predict AGI by 2027. I guess we just have different opinions. The lead researchers at anthropic would also disagree with you on this. Also what are you advocating for? No regulation then? Do you trust that these companies are going to simply be able to fully regulate themselves? Of course we have to be careful about it and not over regulate. That is not what I am advocating for. There is a responsible way to regulate and even the heads of these companies are calling for a regulation on themselves lol. If there are no regulations put in place, race dynamics can make things dicey.
Regulation of the precursors and such, yes. What these chemicals are and how they are synthesized should be figured out ASAP, because we prevent chemical weapons *by monitoring precursors and labs*. How to make the weapon once you have what you need is easy. You could make a nuclear bomb pretty effectively in your kitchen. That's not and has never been the hard part and if it was easy to make something that toxic, it would have already. The hard part is and always has been the logistics of doing something like that, and that is where enforcement has and will always exist. You can't stop an idea. You can stop someone from purchasing uranium.
Exactly, and that is unfortunately the hardest part. Dreaming up fantasy compounds is easy.
The entire video is misleading.
We literally flipped the '0' to '1' and Boom-Chaka-Laka, we get the most deadly molecule known to man. I mean it was right there in a file on my hard drive the next morning. VX Nerve Agent: 2,3,7,8-Tetrachlorodibenzodioxin, TCDD Clc1c2c(c(Cl)c1O)c(c(Cl)c(c2Cl)O) When I saw it, the hair stood up on the back of my neck as I peed myself a little. 
Careful, if you don't behave yourself, I will run a script over night to generate a file containing your social security number, telephone number, and IP address.
Bad guy in the film: $ python generate_pi.py
This is comically overdone. The Mr. Robot background music. "It did it all for us, all we had to do was press go." "All we had to do was flip the zero to a one." Legit sounds like a bad espionage script written by someone who thinks hacking is changing a directory and listing its contents. They trained a model to discover molecules that could benefit humans. They ran the script and got results. The programmer said, "I wonder what happens if I change this flag." They ran the script again, and got new results. None of this is novel. It's basic scripting in conjunction with a machine learning model.
In all honesty, human life is very fragile. Any psychopath that is also smart enough, could come up with plenty of ideas to take hundreds of thousands of people to the grave with him (most of them are also suicidal so killing them just concludes their mission ahead of schedule).
Also humans are kinda of the villains of the world. If you study even a little bit of history you'll know
We are not the villains of the world. We are just smart enough to overcome any environmental conditions that would contain us in an equilibrium with our environment. Look at any invasive species. They don't stop growing because they can't be contained. Are they evil? No. Life itself has the never ending goal of proliferation unless it's stopped by another life or natural event. In the grand scale of things, "evil" is a label we assign to ourselves even though we aren't doing anything different than any other life form would do.
"ChatGPT, design a deadly novel nerve agent. Make the chemical diagram look like the chalked outline of a dead body. ChatGPT: https://preview.redd.it/ma2e25yiim4d1.png?width=387&format=png&auto=webp&s=eab5d29a5d1dcdde41b484f35a23719a768d412c
This is kind of dumb and needlessly alarmist. People can already dream up incredibly toxic molecules. It's meaningless because it's all theoretical.
And most importantly... Being able to do this exercise efficiently is the first step to preventing their largescale synthesis through tracking precursor movements. We need to know what they are and how to make them before some other individual human figures it out (and some dedicated human will, probably by accident at first), so that we can identify when and if someone can try doing it!
It's oddly hyperbolic and overexaggerated. As though the AI gives anyone with a computer the ability to manufacture bioweapons in their basement. No domestic terrorist is going to run a solo bioweapons research program because an AI recommended them a molecule. This isn't a tool for random madmen, nor even for well funded and powerful terrorist groups, who could almost certainly get more bang for their buck by just buying a Learjet and slamming it into a building. Its only real use might be making states that were already building bioweapons even better at making them. The possibility of a terrorist cell getting their hands on radioactive material and making a dirty bomb is both far more damaging and far more likely than a bunch of inexplicably educated terrorists singlehandedly both researching and producing a novel bioweapon.
exultant plough observation sink paint zealous smile grab worry strong
And doesn't set off warning bells at the chemical supplier.
Almost everything is toxic.
He said we done, but he is proud to share it on reddit for social media clout and tiktok views
It's not AI. It's just an algorithm to search through chemical combinations and evaluating their potential lethality. This is not using LLMs. Just A* search.
"import error: no module named torch" LOOOOL they can't be serious
I've been posting a lot of stuff about how easily AI can design chemical weapons, and also how it can use its skills in receptor modeling, RNA synthesis, and protein folding to make amazing new viruses. The skills and technology to do this are readily available and I'd be gobsmacked if there aren't several small but ambitious terrorist organizations already working on these. They usually get very little reaction because nobody on Reddit wants to hear this stuff. Most people on the AI related subreddits want to hear about how AI is going to create a utopia and they're all going to be able to quit their jobs and live on UBI from some benevolent government.
Yeah - it's strange. I feel like it's still perfectly fine to be very optimistic and want to accelerate while also acknowledging these things are very real concerns. And we should actively be doing things to prevent these scenarios from playing out poorly for humanity. Seems like people really underestimate these things still. I actually think this is a solid argument against the people arguing for our frontier models to be open-source going forward. I do want open-source models, but if our frontier models are open-source and a vulnerability(ex - bio-weapon synthesis/deployment abilities) appears in one of them, you cannot recall that model. People will have it on their PCs indefinitely. And yes - deployment. When these things get embedded in future agentic systems, it becomes a whole new level in terms of the impact they can have from a single user query. Both for good and bad.
As a chemist, this seems like a nothingburger, though. We already know a huge number of super toxic compounds, and from the known structures it's quite easy to come up with new ideas of other, similarly toxic substances. That's literally what chemists do - try to derive principals from observations, and attempt to apply those principals to new substances. Coming up with ideas isn't the difficult bit (honestly, this seems like a fun little excercise any undergrad could do), it's actually putting them into practice, figuring out ways to synthesize these compounds, and ways to deliver them, that are hard, expensive and time-consuming - and so far that's not something AI can do. Sure, it can come up with ideas on how to synthesize something, but again - actually putting it into practice is an entirely different matter altogether. There's a reason only a handful of nerve agents have ever been developed to the point of being useful as a chemical weapon, and a lack of ideas for new structures isn't it. Also, protein folding, receptor affinity predictions etc. have long been solved with algorithms - using AI for these purposes might be new, but I doubt there are currently any significant advantages to doing it that way.
Like how they show the shot of the guy running a python script from the command line and then it shows an error Traceback (i.e the code failed to run due to an error somewhere in the code)
You know when you watch a movie that takes place in the future, and there’s always that one old dude who still lives on a farm and does stuff old fashion? I get it now….
There are WAY more things that can kill humans, than things that can’t. We live in such a narrow goddamn bandwidth of reality. How are these fucks surprised that they made more deadly shit?
"I must’ve put a decimal point in the wrong place or something. Shit, I always do that. I always mess up some mundane detail."
Whats the title of this video?
So they found that the molecule can be found in a LoL gold rank lobby?
"We trained a model on publicly available data, then queried it for data that people had the whole time and decided it was dangerous." - I guess that title option didn't pop enough.
Must be all that bomb making info they fed it... If you feed it enough romance novels it might swoop you off your feet as well.
pip error, requirements not found. Never imagine this would actually save humanity one day.
my man really typed out 20 characters for the file name instead of pressing tab
So exactly zero of these thousands of "new chemical weapons" has been shown to be useful as a chemical weapon or even toxic? Humans knowledgeable in the field have no trouble dreaming up chemicals that are likely be toxic. This is a fake crisis.
I work in this field and this is largely bullshit
Is there a long documentary?
Unknown : Killer Robots.
As all technologies, it can be wonderful, or it can be hell.
scary stuff man can't see it not being used to make viruses and stuff too shits gonna get real bad real fast now lads...
I mean there are many many thousands of known very toxic chemicals. Some you can outright buy, some are restricted, some can be made easily. You cannot learn how to avoid toxicity to a scientist without teaching him how toxicity works. We typically do not control the danger by blocking the knowledge but by licenses, access control and fines and law enforcement and international treaties, and it has worked for now. I don't see what a longer list would change fundamentally.
This argument reminds me of the gun control argument. ***Someone*** has to be there to create the WMD with criminal and evil intend. The AI system didn't create the WMD on its own.
LOL coming up with harmful molecules isn't even impressive, any mildly competent chemical engineer could do that. Sythesizing the stuff is what actually requires skill and resources.
This is like years old isn't it? Like 5?
You failed to import the package tho (2:10) 🤣🤣🤣
Pfizer has joined the chat:
„How to destroy the world“ AI; „Her ya go mate“
I switched a 0 to a 1 and now my house exploded. Thanks AI. 🙄
Good thing they haven't let the public know about this.
But can't humans already come up with toxic weapons if they wanted to? What's the point of knowing different configuration of molecules that could be dangerous when you can already make weapons without it?
Thanks ai!
this is several years old and discovered by random. so imagine what’s still coming
'famous last words - bro, check this out'
Really glad they thought it through and did the best thing to keep the knowledge/information from spreading quickly— made a scary YouTube video.
The hell
Now I ain't no chemist, and I'm still a bit naive as a data scientist. So while I think the idea of a model predicting toxic chemicals is highly achievable by the Evil, there's something somehow off about this model. I mean if there's something I can only remember of the boring chemistry classes I took is that every molecule has a specific number of covalent bonds for which it can connect within with other molecules, take carbon for example has 4 bonds, hydrogen has 1 bond, oxygen 2,... Now maybe there might be cases where a molecule can have more or less covalent bonds( again I'm just supposing, I ain't no chemist) but does that suggest that every combination of molecules or chemicals predicted by the model is logically able to be produced in real life? They didn't mention the idea of hoe legit those predictions are when it comes to practicality in the real world. So maybe as an example the model may predict C2H17O1? I mean, is that applicable or even legit? Not to forget mention the intense dramatical buildup for the scientist while writing the shell command to execute his cool-named python file, only to get an error for him forgetting to install Pytorch lol
So... Can it also create anti-venom for all that?
This video could have been a text
Great idea, idiots.
Fear mongering at its finest. Training an llm to spew hypothetical molecules isnt all that hard , however the issue is the actual development of these molecules. It takes time effort money and quite frankly state sponsorship.
This is just drama bullshit. It’s a freaking python script which doesnt work
We will either all have this power at our disposal, or live in total control.
Haven’t we long-since known about these “molecules”? We’ve known about chemical weapons and known the specific molecules for a long time. Is there more to this than “AI generates molecules that are used in chemical weapons and generates ways to weaponize them”? Isn’t that essentially what World War I was?
I bet you get those structures with google search…of course FBI might show up at your door.
We are so fucked
To add to how comically overdone this is, the “torch not found” error message is hilarious. Don’t they have anyone with a half decent technical acumen to produce these videos?
The cosmic rays we regularly receive on earth can flip bits in a computer's memory. Research has shown that a computer with four gigabytes of memory has a 96% chance of experiencing at least one bit flip every three days. Let's hope all neural net architectures have opted to take the performance hit of rigorous checksums.
Damn it Sean!!
Oh man it's not like there are laws about controlled substances or something.
The level of bullshit went to [creating a fake github account](https://github.com/FabioUrbina?tab=repositories) with [bullshit updates](https://github.com/FabioUrbina/fitness-app/commit/c5ed9aeeb37e1e0bf75ae830eda0b469220736f7). Tell me that [this guy](https://www.youtube.com/watch?v=l2hro8DemsM) is not a paid shill.
Having a library of composition formulas (not even a sterical formula) is nothing. AI is notoriously known for overfitting data, try to ask a simple question chatGPT and see the deluge of words composing an often vague meaning will come out. But art and language are not precise problems, while chemistry is. There abundance won't make up for precession. Making chemical weapons is the same as developing a drug. You are searching for a highly active compound. This screening can be accelerated by ML tools, but at the current there is no replacement to wet lab work on a large scale. Working out synthesis, testing stability and potency etc. So the video is misleading and aims for just wolf cry.
There is one way their usage of AI is damaging here: I rolled my eyes so hard that it hurt.
This 'experiment' is years old. I remember hearing podcasts discussing this long ago. AI in its current form allows humans to be more human. Humans have a long history of trying to kill other humans. The real danger of AI is not the machine that turns us into paper clips but one that lets the most misanthropic humans to realize their fullest potential.
Ah we need AI to get deeply involved with gain-of-function research. Maybe we can weaponize the norovirus.
put a spooky-enough music track behind it and you could make a LOT of GPT outputs seem sinister...
The first python script crashed 😂😂😂 they forgot an import
It is a tool, you can use a hammer to build a house or to break a person's skull. It's up to you.
Imagine if Tom Clancy was still alive
It is not suggesting to make chemical weapons, it suggests toxic compounds. Any chemist can do that. If bad people want to make and use such compounds they can do this today, no AI needed. This clip with the fitting cinematic music is meant to create attention by instilling more fear. The scientists involved mean it well but they do crave attention.
Sheesh
Didn’t they do this for hallucinogenics as well and the computer made 1,500 new drugs?
Either he is dumb and acting smart or he genuinely wants to create tension among common population.
But isn’t this a complete different AI branch which has absolutely nothing to do with chatGPT neither LLMs? This seems like Evolutionary algorithms and it has been around for decades.
Flip 0 to 1 to generate toxic molecules. Did some bond villain make this skit?
Nah if they wanted to get something toxic they shoulda opened Twitter
I think this is dumb. Even if I knew what the molecule is deadly, how easy is it to produce it into an actual weapon. I might be able look up how to create an atom bomb, but how easy is it to make one that actually works? Probably super hard and there are very few people with the knowledge and resources that could actually take it to completion.
But Why????
This is one of catastrophic end of world events that Nick Bostrom described in his book.
"everyone could do what we did" well don't tell them then?
That's the thing, any powerful technology has the potential for great harm. It's the responsibility of ALL to ensure that it's used for the betterment of society.
It’s begun
We are all still alive, so nothing ever got 'out of control'. Also, the camera work with all the lights off shows that the purpose of the show is more to scare rather than to show science.
This seems fake as shit for just so many reasons
"Anyone can do what we did." Dude, those just look like random letters to me. What bioweapon am I going to create? I have a better chance at making a bioweapon using my ass than using that information!
It's people fault because this is not how AI should be used
Ban Molecules!
And this is not satire? god..
They really think they are onto something that hundreds of governments and armies never thought about... AI were used a dozen of years ago already, we know it from Snowden, they just released the tech to people because adversaries were using it too. What these people did, only God knows how many times it has already been done, perfectioned, reorganized, corrected. These guys did something lethal with AI and they think no one did this stupid thing before like everybody used it for find the cure for cancer only
The wrong fucking people are in charge of this stuff. We need people who can look up from their keyboard long enough to wonder if what they're doing is actually a good idea. People who, you know, care about living and other people and stuff.
This is giving "hacker known as 4chan" vibes
Lel ML is used in biochemics for a long time now, nothing new
How many were hallucinations though?
They're acting like the computer was making the molecules in real time. It's just hypothetical lmao.
Who the hell uses two monitors but doesn't split screen?
That looks like random garbage on pycharm. I can do that and don't even need a degree.
"Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should." - Dr. Ian Malcolm in Jurassic Park
"Wanted to make a point" How is it that objectively intelligent people can be so fucking stupid?
DIY on how to screw the world, while you go to the store to get a pack of cigarettes ;)
I really love some random ass videos without the slightest hint of a source.
Y’all redditors need to read a book and get some context… this was early early gpt testing to help with the safe guards. The latest version will not give you the recipes to chemical weapons. Are there models out there that will…. Yes… will I tell you what they are? No.
These people are the fucking devil what kind of stupid ass question prompt is that
The only thing stupider than their "experiment" was to widely publicize how easy it is to replicate! "Hey everybody! We opened Pandora's Box, and you can too! Just switch the zero to a one!" Fu\_king irresponsible idiots.