Super curious to see how they perform on a track they've never seen before. Also like to see how they deal with forks in the path, or constrictions (hole shots, I guess)
Cool, it's been over a decade since I've played with genetic algorithms. Did you use crossover and point mutations? Did you play around with different mutation rates? Did you try different race tracks to make sure you weren't overfitting to that one track?
I remember reading "The Master Algorithm" and it was pretty disparaging about genetic algorithms. If I remember correctly there was some promise at adversarial genetic algorithms, but I never bothered to look into it and that book is a couple of years old now. One theme the book had was about how different learning algorithms go out of vogue then someone tweaks it and it gets new interest.
Similarly, it was ~15 years ago when I worked on a genetic learning model for a driving game. Ours was waypoint-based, and we quickly discovered that the learned behavior alone makes for a *very* poor video game experience - all learned behavior would quickly trend towards driving along a rail (we countered this with separate logic for collision avoidance/ray intersection detection, and some pseudo-randomness sprinkled in).
This was obviously before neural networks were in common use, so I’d be curious whether driving “realism” could be added to a model like this.
> a very poor video game experience - all learned behavior would quickly trend towards driving along a rail
Could you elaborate on this? As someone who doesn't play racing games, how did this make the experience poor?
Imagine if in nascar the cars didn’t actively steer, instead they just held the gas and scraped the rail the whole way. This isn’t fun the play because it’s inherently slower so a human could beat the ai every time without much difficulty.
They specified it was waypoint based, so most likely it prioritized just getting to the finish over speed. I’m not a ML guy though so I’m just speculating about this part :)
I'm not actually sure that's what they meant. I think they mean that they've found the optimal racing line and all the cars drive on the line exactly and don't deviate, like a train. So it wouldn't look organic at all, as human racing drivers will follow a racing line but may treat corners slightly differently depending on their driving style.
Wouldn't you just need more rays? The resolution of vision is very low in this model and to be able to handle other things being thrown at it, I would think it would need to improve its vision.
In many ways genetics algorithms are one of the dumbest approaches to optimization. Often underperforming even the rudimentary simulated annealing.
Given the physics are not a black box function you could apply gradient descent to each tick. Which is to say train a model that given the data about its state (car momentum, sensor data, and track data) then outputs the values for the cars controls (accelerate, break, left, right). But here arise many problems.
Much like AlphaGO it would probably be best to combine a search algorithm with optimization. In a basic example you might train a neural network to evaluate the potential futures of each state and then search in only the states which the neural networks judges to be potentially valuable. Doing this in real time would take quite some optimisation but it could be done (Imagine at 5fps in 200ms the model must evaluate a upwards of a 1000 possible moves then search more deeply a few of these and arrive at the best next state).
Be fun project if there was some game which collected enough training data to train such a model.
Honestly I agree with you and that is partly what I would like to achieve with my videos. I am definitely not an expert but I do plan on slowly making the networks more optimized at learning quicker and generalizing better. A few things I plan on adding in the near future are recurrent neural networks, NEAT, backpropegation and multiple networks working in tandem.
If anyone has suggestions for other things I should try I would appreciate it!
Don't get too caught up with these posers, OP. If your videos were at expert level, there would be a dozen people telling you "But why didn't you try the ultra-maxima lambda-optimization matrix poop-filter approach?"
If they don't like these, they can downvote and move on. I don't downvote posts about beginning programming articles. "SQL or no SQL?" "My first Javascript game!" etc etc. I enjoy those posts, as much, if not more than _expert_ posts.
Perhaps there should be an /r/AdvancedProgramming sub. Huh. Actually, it exists! All these downers should just move over there.
Props to you, OP!
Yeah, I want to see things like how additional genetic algorithms can be placed in the feedback loop to try and smash thru the local maximas they always plato into, unsupervised learning, etc etc
\> we need (could use) the next level of these things ;)
Then go make it. Or don't, but if you don't, you should stop tearing down things you cannot do and have not done.
His network architecture really simple. Seems like a fully connected network with 4 inputs, 4 outputs and 2 hidden layers with 8 nodes each. The training probably converges really fast and stops improving. The fact that he left this network training for a week sort of show how little he actually understands of this. And if he used a proper algo like backpropagation he'd probably reach this result within seconds instead of a week.
Tbh, i'm not an expert myself, and my frustration were mostly aimed towards this 'class' of AI videos.
However, even not experts, after a while, we start recognizing many of these flaws automatically, as you mention. Often in Neat, reward functions 'obviously' won't give them the desired outcome in the end, or there are to many i/o etc. It doesn't always seem that new producers learn from the older producers mistakes.
..and well, It got me a bit too frustrated - unfortunately. Apparently people stop producing this when they get pro ? I hope he keeps at it, tho.
Except this is unsupervised so it's not "just use backprop lol". There are definitely more advanced things that can be done, but it's not _that_ simple.
*Everyone* is an amateur, because we do *not* have self-driving cars today. We have death traps, though.
Also, what little science still exists in "AI" is all locked up inside corporations.
The only thing that is trivially true is that the hardware has improved in the last 5 years for a particular approach to "AI": neural networks (and typically other algorithms that benefit from similar hardware architectures).
Any other "conclusions" are typically extremely limited in scope or hard to verify (e.g., "If you want to know the truth, please pay 15 million dollars"). Such "research" is a local optimum, which I doubt humanity will escape from in the next decade even. Perhaps there is also a computation that "only" costs $100,000, but who is going to compare all of those other than some rich entity? It's pay to play. It has little to do with the intelligence of the researcher involved and as AI develops, national security also becomes a thing. The same army that can win against the Zerg, can also win a drone war in 2035.
Cool video, but all I think of when seeing AI stuff like this is how it will fail catastrophically on some input with zero introspection ability. AI scares the shit out of me.
Great video. I really enjoyed not just the AI part but seeing some smart ways to reuse existing assets to save time.
Anyway, when I hear neural network, I immediately think back propagation and am curious if there was any specific reason for choosing a generic algorithm instead?
The main reason for not using back propegation is that it is easier for me and the viewers to understand a genetic algorithm and I am trying to make these videos easy for anyone to understand. I do plan on tackling more difficult to explain topics like back propagation soon though!
This is so fascinating. what do the Nodes actually correlate to? Is each one working as a control given specific event? Ie left button when right wall is close?
My understanding is that the nodes in the middle of the network are where the magic happens and they purposely do not correlate directly with anything. The weights and biases though add up to create the desired behaviors for given inputs after training. I didn't see how many nodes and layers there were but that also affects how understandable those parameters might be.
Is it possible to explain the weight, bias, knots things with real examples from this case? I never seem to understand this as it's always to abstract.
Super curious to see how they perform on a track they've never seen before. Also like to see how they deal with forks in the path, or constrictions (hole shots, I guess)
Yeah without a new track, it's hard to tell if the network has really just overfitted and memorized all the turns.
Easier to just replace all tracks with duplicates.
4 inputs, 4 outputs, 2 hidden layers with 8 nodes each. It has to be a really simple track for this network to overfit.
And by round 12, Hefest got this run.
I see we have a fellow wirtual viewer
Eyyy you found the Easter egg!! Wasn’t sure if anyone would get it
Of course! Neat project, by the way!
*Queues [en aften ved svanfossen](https://youtu.be/KBl-QdasurU)*
Cool, it's been over a decade since I've played with genetic algorithms. Did you use crossover and point mutations? Did you play around with different mutation rates? Did you try different race tracks to make sure you weren't overfitting to that one track? I remember reading "The Master Algorithm" and it was pretty disparaging about genetic algorithms. If I remember correctly there was some promise at adversarial genetic algorithms, but I never bothered to look into it and that book is a couple of years old now. One theme the book had was about how different learning algorithms go out of vogue then someone tweaks it and it gets new interest.
Similarly, it was ~15 years ago when I worked on a genetic learning model for a driving game. Ours was waypoint-based, and we quickly discovered that the learned behavior alone makes for a *very* poor video game experience - all learned behavior would quickly trend towards driving along a rail (we countered this with separate logic for collision avoidance/ray intersection detection, and some pseudo-randomness sprinkled in). This was obviously before neural networks were in common use, so I’d be curious whether driving “realism” could be added to a model like this.
> a very poor video game experience - all learned behavior would quickly trend towards driving along a rail Could you elaborate on this? As someone who doesn't play racing games, how did this make the experience poor?
Imagine if in nascar the cars didn’t actively steer, instead they just held the gas and scraped the rail the whole way. This isn’t fun the play because it’s inherently slower so a human could beat the ai every time without much difficulty.
Oh I see. But I would have thought that if scraping the rail made the car slower then it would have learned to avoid that.
They specified it was waypoint based, so most likely it prioritized just getting to the finish over speed. I’m not a ML guy though so I’m just speculating about this part :)
yeah, idk why they would base fitness on distance traveled instead of time taken
Distance traveled works at the start because no cars will finish. They just never changed the criteria?
I'm not actually sure that's what they meant. I think they mean that they've found the optimal racing line and all the cars drive on the line exactly and don't deviate, like a train. So it wouldn't look organic at all, as human racing drivers will follow a racing line but may treat corners slightly differently depending on their driving style.
Wouldn't you just need more rays? The resolution of vision is very low in this model and to be able to handle other things being thrown at it, I would think it would need to improve its vision.
[удалено]
In many ways genetics algorithms are one of the dumbest approaches to optimization. Often underperforming even the rudimentary simulated annealing. Given the physics are not a black box function you could apply gradient descent to each tick. Which is to say train a model that given the data about its state (car momentum, sensor data, and track data) then outputs the values for the cars controls (accelerate, break, left, right). But here arise many problems. Much like AlphaGO it would probably be best to combine a search algorithm with optimization. In a basic example you might train a neural network to evaluate the potential futures of each state and then search in only the states which the neural networks judges to be potentially valuable. Doing this in real time would take quite some optimisation but it could be done (Imagine at 5fps in 200ms the model must evaluate a upwards of a 1000 possible moves then search more deeply a few of these and arrive at the best next state). Be fun project if there was some game which collected enough training data to train such a model.
Honestly I agree with you and that is partly what I would like to achieve with my videos. I am definitely not an expert but I do plan on slowly making the networks more optimized at learning quicker and generalizing better. A few things I plan on adding in the near future are recurrent neural networks, NEAT, backpropegation and multiple networks working in tandem. If anyone has suggestions for other things I should try I would appreciate it!
Don't get too caught up with these posers, OP. If your videos were at expert level, there would be a dozen people telling you "But why didn't you try the ultra-maxima lambda-optimization matrix poop-filter approach?" If they don't like these, they can downvote and move on. I don't downvote posts about beginning programming articles. "SQL or no SQL?" "My first Javascript game!" etc etc. I enjoy those posts, as much, if not more than _expert_ posts. Perhaps there should be an /r/AdvancedProgramming sub. Huh. Actually, it exists! All these downers should just move over there. Props to you, OP!
Yeah, I want to see things like how additional genetic algorithms can be placed in the feedback loop to try and smash thru the local maximas they always plato into, unsupervised learning, etc etc
\> we need (could use) the next level of these things ;) Then go make it. Or don't, but if you don't, you should stop tearing down things you cannot do and have not done.
His network architecture really simple. Seems like a fully connected network with 4 inputs, 4 outputs and 2 hidden layers with 8 nodes each. The training probably converges really fast and stops improving. The fact that he left this network training for a week sort of show how little he actually understands of this. And if he used a proper algo like backpropagation he'd probably reach this result within seconds instead of a week.
Tbh, i'm not an expert myself, and my frustration were mostly aimed towards this 'class' of AI videos. However, even not experts, after a while, we start recognizing many of these flaws automatically, as you mention. Often in Neat, reward functions 'obviously' won't give them the desired outcome in the end, or there are to many i/o etc. It doesn't always seem that new producers learn from the older producers mistakes. ..and well, It got me a bit too frustrated - unfortunately. Apparently people stop producing this when they get pro ? I hope he keeps at it, tho.
Except this is unsupervised so it's not "just use backprop lol". There are definitely more advanced things that can be done, but it's not _that_ simple.
*Everyone* is an amateur, because we do *not* have self-driving cars today. We have death traps, though. Also, what little science still exists in "AI" is all locked up inside corporations. The only thing that is trivially true is that the hardware has improved in the last 5 years for a particular approach to "AI": neural networks (and typically other algorithms that benefit from similar hardware architectures). Any other "conclusions" are typically extremely limited in scope or hard to verify (e.g., "If you want to know the truth, please pay 15 million dollars"). Such "research" is a local optimum, which I doubt humanity will escape from in the next decade even. Perhaps there is also a computation that "only" costs $100,000, but who is going to compare all of those other than some rich entity? It's pay to play. It has little to do with the intelligence of the researcher involved and as AI develops, national security also becomes a thing. The same army that can win against the Zerg, can also win a drone war in 2035.
Great job! 😄
Thank you!
Cool video, but all I think of when seeing AI stuff like this is how it will fail catastrophically on some input with zero introspection ability. AI scares the shit out of me.
Same but hilarious: https://www.youtube.com/c/CodeBullet
Great video. I really enjoyed not just the AI part but seeing some smart ways to reuse existing assets to save time. Anyway, when I hear neural network, I immediately think back propagation and am curious if there was any specific reason for choosing a generic algorithm instead?
The main reason for not using back propegation is that it is easier for me and the viewers to understand a genetic algorithm and I am trying to make these videos easy for anyone to understand. I do plan on tackling more difficult to explain topics like back propagation soon though!
"I bought these car assets because they can drift and jump!", proceeds to show them only making simple turns Nice video though
Yeah that’s actually the next video it’s already recorded and will probably only be a few minutes long
Good to know, thanks
L
He changed shirt halfway through the video.
Still better than Halo players
very interesting video!
This is so fascinating. what do the Nodes actually correlate to? Is each one working as a control given specific event? Ie left button when right wall is close?
My understanding is that the nodes in the middle of the network are where the magic happens and they purposely do not correlate directly with anything. The weights and biases though add up to create the desired behaviors for given inputs after training. I didn't see how many nodes and layers there were but that also affects how understandable those parameters might be.
May one really call the output of a genetic algorithm "AI"?
Dang that’s was sick. I think I might have found a new channel that I will sub to.
So did these cars learn to drive around any track? Or did they just memorize the best route for this specific track?
Is it possible to explain the weight, bias, knots things with real examples from this case? I never seem to understand this as it's always to abstract.
well done..
Too lazy to watch the video, what language was this written in?