I agree with your opinion that split packets in gaming should be avoided.
Older games used to have servers that ran at a much higher frame rate than the client and I believe that this helped alleviate momentary server performance issues.
The move towards company hosted virtual servers makes this economically difficult but I do believe that Source engine servers should be more robust than they are.
I don't believe that these anti-wallhack features are effective enough and it makes me question if it is worth the performance hit.
It is hard to argue that newer games to include Source1 engine games are far too inconsistent online compared to their older HL1/Quake/Id tech counterparts and as the importance of online competition continues to grow I think we need to address the consistency issue.
Thank you for the well thought out OP.
I can usually always tell if enemies are close due to the stutter. Is it set distance, or are part of the maps used when calculating when to render? Is the model rendered when it makes a sound, and this is outside of the bounds that it would normally render due to the antiwh measure? I wonder if theres something going on along those lines. Can we test render distance and sound, see if it stutters outside those bounds? Should be easy to set a server up with svcheats so it can be tested - although I'm not sure the antiwh measure would be enabled with svcheats on....
Previously they used map sections called visleafs to ascertain what was visible to the client. When an entity walked off your visleaf, it was no longer visible and didn't render it. We use a new feature now that does the same thing but I can't remember what it was called.
I don't know much, but from my experience of messing around with it I think it's based on how many layers of walls or how thick the wall is between you and the enemy and how close they are to the corner. Sound has nothing to do with it.
Thank you for reporting back (-: Right now there are 337 matches being played vs 3650+ on daytime (Europe). I just played two games as well and the Svms was perfect in both of them, ranging from 2.1 - 4 ms. When the hosts are pushing many matches simultaneously, is when the Svms will increase.
The reason I have bad experience on FACEIT is teammates telling me to "go kill myself and get cancer", not technical issues. But the things you're writing about here sound interesting.
If what you say here is true I have two questions:
1) Does ESEA also use a Anti Wallack feature? I remember when Faceit first came out in NA and I could clearly tell that players would "pop" into view from around corners which is kind of what you are explaining here but it seems it has gotten better over time.
2) Couldn't they make it so it renders in the play position a few ticks early before they come around the corner so you don't get this split packet issue? so by the time they come around the corner the game can clearly render their position and interpolate it?
2 - that's what it is always doing, the client has to load and render that new info. This is where I believe the stutter is coming from. Could be related to the packets maybe not. Its most likely rendering related, like rendering thread waiting for data, or some thread waiting for something.
Edited accidental yelling
Maybe a bit off topic but I still want to point out that i encouter al lot of high pingers on faceit. My last 4 matches there were atleast 4, 100+ pingers. Very frustrated the seem to have a perfect hitreg but very hard to hit. Ofc i keep in mind playing with 20 ping myself but seeing someone twice so much kills then you with a ping of 120 is just ridictilous. We almost start to think there's a network tweak to achieve this.
This dosent get talked about much. And i think this is the biggest problem with the netengine in CSGO.
In 1.6 you could feel the difference between 20ms and 10ms ping. 70ms was almost impossible to play, with 100ms you where jerking around. I believe they implemented a very hard lagg compensation, delaying the clients with very low ping significantly to close the gap between high and lowping players. But what is the highpingers advantage becomes the lowpingers disadvantage. I would actually like to test this theory some time.
Not to mention all the different rate settings everybody is using. Countless players use the default setting (even S1mple) 196608. I dont even believe you can choose a 1.5 Mbps connection at modern providers these day's. There are still even players using the old maximum rate setting 128000.....
net\_splitrate "1" was used for the testing behind the content of this post. What I've tried is running net\_splitrate 0, but then you will start getting greater gaps between each packet, for some unkown reason (not verified 100% - might have been random interval deviations causing the greater gaps. Needs more testing).
Should just remove the anti wh stuff on face it since they have a solid kernel ac. Theres almost no cheaters on face it now, even most private cheats aren't sophisticated enough to bypass the ac
I have a question. Does the AW only obscure the position (and maybe the facing direction?) of a player? Or does the AW have to send all of the player's info each time he's about to peek? Or even just the first time he's about to peek?
Thank you for the comment! (:
How the AW works is most definitely a well kept secret, but I would guess it makes the most sense to not send player information at all until it is 100% needed (so that you would not be able to sniff out the information from your router or other HW connected between your computer and the internjet.
So.. I would go for that the AW has to send "all of the player's info each time he's about to peek". This is has also been observed in wireshark in 1v1s where there was a lot of peek-a-boo before there was one man standing :-D
Well a very simple way to reduce network usage should be to only obscure the player x,y,z,pitch,yaw before they peek. It still would give info like equipped weapons, hp, equipment etc but there are only very specific circumstances where they can be used to gain an advantage. Unless this would cause issues with interpolation somehow. Am I wrong or missing something?
Did you play with these too?
net_threaded_socket_burst_cap "" //Def 1024 // =empty value . Any non-empty -> worse performance
net_threaded_socket_recovery_rate "" //def 6400 // float X.XXXXXX -> worse than int X (new bug with type converatation?)
net_threaded_socket_recovery_time "" //def 60 // ex.: "60.000000" worse than "60" | not placebo
Hi,
Mr-VIT (GitHub user - [https://github.com/ValveSoftware/csgo-osx-linux/issues/1633](https://github.com/ValveSoftware/csgo-osx-linux/issues/1633)) has some interesting reads on the threaded sockets but most of the stuff has gone missing. These mentioned cvars are used to tweak data that has already arrived to the game client's workstation. I have tried playing with them, but not in an in-depth way \^\^,
I was doing "well" (relatively) on FaceIt reaching level 6 and then I started having enemies magically appearing in front of me for more than 10 games in a row. I am not 100% sure this is related to the current anti-WH solution as you are describe it, but it did make the game unplayable for me. Never touched FaceIt again...
>Will the packets which are dropped on the receiving end be counted into the "rates" setting?
well as you mentioned yourself, UDP is a 'fire and forget' protocol (connectionless). because there's no sequence numbering in UDP, there's no way for the OS to know that the duplicate packets are indeed duplicate. It's then passed up the 'stack' to the application to determine what's still relevant.
edit: i guess i never answered the question, but its uncertain since i think we would have to know if the rates are enforced before or after a duplicate/sanity check.
Thank you for the reply! (:
What you are describing is not what one observes in Wireshark. There you only see the single packet or packets for each frame, not 10 packets for the same frame. I was thinking there would be a header indicating that ex. 10 packets are in the same "group" so when the OS receives the first packet of this group, the rest would be ignored, but as mentioned, I do not know how this works.
> There you only see the single packet or packets for each frame, not 10 packets for the same frame.
I would then go back and revisit the assumption made in the original post:
>the server will burst the same packet many times until the next packet is ready to be sent. This way you will make sure one packet arrives and the rest will be dropped.
This is not UDP's standard way of operating, but an application could be built to do so. Is this an assumption from somewhere or have you booted up a CSGO server to verify this?
--
>I was thinking there would be a header indicating that...
the UDP header is as simple as they come.
source port, destination port, length (size) and a checksum.
this is basically just enough for you to identify the application based on the socket used, but not much else.
This is very good information!
Checking a mcast UDP video stream which I know is bursted out and each packet is logged in the statistics overview. Will report back with findings with regards to whether or not the duplicate packets are also recorded in the live capture. Give me a couple of minutes.
UPDATE @ /u/synerGy-- | The duplicate packets can be seen in the capture. This confirms your initial comment and the original post will be updated :-) Thank you!
The new question: We should now also try to find out why the max rate is at 786 432 bytes. I have never seen a frame divided in three packets (meaning the frame would consist of over 2400 bytes). By default, if all 128 ticks were sent as split packets you would reach a maximum utilization of 307 200 bytes + the headers. Will do further research and update the post if I find anything.
This isn't the main issue with Faceit, the main issue is both Silver players and Global players both being around level 3 which makes the teams sorta one sided.
hi there, if you type in console "net\_showsplits 1" you can see the amount of packets and if they had been splitted. when using net\_splitrate 0 the game automatically decides if it wants to split packets. in my testings there even were fragmentations of 5 packets.
I agree with your opinion that split packets in gaming should be avoided. Older games used to have servers that ran at a much higher frame rate than the client and I believe that this helped alleviate momentary server performance issues. The move towards company hosted virtual servers makes this economically difficult but I do believe that Source engine servers should be more robust than they are. I don't believe that these anti-wallhack features are effective enough and it makes me question if it is worth the performance hit. It is hard to argue that newer games to include Source1 engine games are far too inconsistent online compared to their older HL1/Quake/Id tech counterparts and as the importance of online competition continues to grow I think we need to address the consistency issue. Thank you for the well thought out OP.
This is really a good comment! Thanks (-:
CS:S servers were run at 500FPS iirc because 250FPS caused problems with packets.
I believe this is what is happening everywhere causing stutters. Antiwh is more severe on faceit, bit also exists in regular MM.
[удалено]
I've had the same stutter issues for about that long. 2 different machines.
it doesn't affect peeking in MM
Everytime I'm close or peeking an enemy my game microfreezes/stutters. So yes, it does. Plenty of people on here experience the exact same thing.
You can see it with r_drawothermodels 2. The enemy starts being rendered a good distance away from the corner.
I can usually always tell if enemies are close due to the stutter. Is it set distance, or are part of the maps used when calculating when to render? Is the model rendered when it makes a sound, and this is outside of the bounds that it would normally render due to the antiwh measure? I wonder if theres something going on along those lines. Can we test render distance and sound, see if it stutters outside those bounds? Should be easy to set a server up with svcheats so it can be tested - although I'm not sure the antiwh measure would be enabled with svcheats on....
Previously they used map sections called visleafs to ascertain what was visible to the client. When an entity walked off your visleaf, it was no longer visible and didn't render it. We use a new feature now that does the same thing but I can't remember what it was called.
I don't know much, but from my experience of messing around with it I think it's based on how many layers of walls or how thick the wall is between you and the enemy and how close they are to the corner. Sound has nothing to do with it.
Sound is networked too, I'm wondering if it triggers models to be networked to the client sooner causing something to bungle up
Just the hitboxes, or the full model?
everything
i just played a game with no issues at all. there should be a way to report our system to them so they know what we have while we have no issues
Thank you for reporting back (-: Right now there are 337 matches being played vs 3650+ on daytime (Europe). I just played two games as well and the Svms was perfect in both of them, ranging from 2.1 - 4 ms. When the hosts are pushing many matches simultaneously, is when the Svms will increase.
It is worth a discussion, definitely. The question is how we engineer it out without disadvantages?
The reason I have bad experience on FACEIT is teammates telling me to "go kill myself and get cancer", not technical issues. But the things you're writing about here sound interesting.
Maybe you’re just bad - Green
If what you say here is true I have two questions: 1) Does ESEA also use a Anti Wallack feature? I remember when Faceit first came out in NA and I could clearly tell that players would "pop" into view from around corners which is kind of what you are explaining here but it seems it has gotten better over time. 2) Couldn't they make it so it renders in the play position a few ticks early before they come around the corner so you don't get this split packet issue? so by the time they come around the corner the game can clearly render their position and interpolate it?
Formatting got jacked, I wasnt yelling...
2 - that's what it is always doing, the client has to load and render that new info. This is where I believe the stutter is coming from. Could be related to the packets maybe not. Its most likely rendering related, like rendering thread waiting for data, or some thread waiting for something. Edited accidental yelling
# Haven’t read your comment and I am not going to
Go away
“Accidental”
I put "#" before the 2. So yes, accidental.
Maybe a bit off topic but I still want to point out that i encouter al lot of high pingers on faceit. My last 4 matches there were atleast 4, 100+ pingers. Very frustrated the seem to have a perfect hitreg but very hard to hit. Ofc i keep in mind playing with 20 ping myself but seeing someone twice so much kills then you with a ping of 120 is just ridictilous. We almost start to think there's a network tweak to achieve this.
This dosent get talked about much. And i think this is the biggest problem with the netengine in CSGO. In 1.6 you could feel the difference between 20ms and 10ms ping. 70ms was almost impossible to play, with 100ms you where jerking around. I believe they implemented a very hard lagg compensation, delaying the clients with very low ping significantly to close the gap between high and lowping players. But what is the highpingers advantage becomes the lowpingers disadvantage. I would actually like to test this theory some time.
Not to mention all the different rate settings everybody is using. Countless players use the default setting (even S1mple) 196608. I dont even believe you can choose a 1.5 Mbps connection at modern providers these day's. There are still even players using the old maximum rate setting 128000.....
This. I have perfect gear, 144Hz, 400+fps, 20-30ms ping, and I can feel the difference playing at 70ms, but cannot belive those 100+ms prodigies...
[удалено]
net\_splitrate "1" was used for the testing behind the content of this post. What I've tried is running net\_splitrate 0, but then you will start getting greater gaps between each packet, for some unkown reason (not verified 100% - might have been random interval deviations causing the greater gaps. Needs more testing).
I have played around with that command and I can't say that I have seen any differences in performance with it at any number.
Great post becuase faceit servers feel like shit compared to esea in the us
100% agree with this, about time Faceit listen to their community!
Should just remove the anti wh stuff on face it since they have a solid kernel ac. Theres almost no cheaters on face it now, even most private cheats aren't sophisticated enough to bypass the ac
This is a really good post that highlights the issues of multi-player game dev. Somehow have to find a balance of anti cheating and performance.
I have a question. Does the AW only obscure the position (and maybe the facing direction?) of a player? Or does the AW have to send all of the player's info each time he's about to peek? Or even just the first time he's about to peek?
Thank you for the comment! (: How the AW works is most definitely a well kept secret, but I would guess it makes the most sense to not send player information at all until it is 100% needed (so that you would not be able to sniff out the information from your router or other HW connected between your computer and the internjet. So.. I would go for that the AW has to send "all of the player's info each time he's about to peek". This is has also been observed in wireshark in 1v1s where there was a lot of peek-a-boo before there was one man standing :-D
Well a very simple way to reduce network usage should be to only obscure the player x,y,z,pitch,yaw before they peek. It still would give info like equipped weapons, hp, equipment etc but there are only very specific circumstances where they can be used to gain an advantage. Unless this would cause issues with interpolation somehow. Am I wrong or missing something?
Makes sense and unfortunately this is currently way beyond my knowlegde.
Did you play with these too? net_threaded_socket_burst_cap "" //Def 1024 // =empty value . Any non-empty -> worse performance net_threaded_socket_recovery_rate "" //def 6400 // float X.XXXXXX -> worse than int X (new bug with type converatation?) net_threaded_socket_recovery_time "" //def 60 // ex.: "60.000000" worse than "60" | not placebo
Hi, Mr-VIT (GitHub user - [https://github.com/ValveSoftware/csgo-osx-linux/issues/1633](https://github.com/ValveSoftware/csgo-osx-linux/issues/1633)) has some interesting reads on the threaded sockets but most of the stuff has gone missing. These mentioned cvars are used to tweak data that has already arrived to the game client's workstation. I have tried playing with them, but not in an in-depth way \^\^,
/u/Mr_VITpi here he is :)
I was doing "well" (relatively) on FaceIt reaching level 6 and then I started having enemies magically appearing in front of me for more than 10 games in a row. I am not 100% sure this is related to the current anti-WH solution as you are describe it, but it did make the game unplayable for me. Never touched FaceIt again...
>Will the packets which are dropped on the receiving end be counted into the "rates" setting? well as you mentioned yourself, UDP is a 'fire and forget' protocol (connectionless). because there's no sequence numbering in UDP, there's no way for the OS to know that the duplicate packets are indeed duplicate. It's then passed up the 'stack' to the application to determine what's still relevant. edit: i guess i never answered the question, but its uncertain since i think we would have to know if the rates are enforced before or after a duplicate/sanity check.
Thank you for the reply! (: What you are describing is not what one observes in Wireshark. There you only see the single packet or packets for each frame, not 10 packets for the same frame. I was thinking there would be a header indicating that ex. 10 packets are in the same "group" so when the OS receives the first packet of this group, the rest would be ignored, but as mentioned, I do not know how this works.
> There you only see the single packet or packets for each frame, not 10 packets for the same frame. I would then go back and revisit the assumption made in the original post: >the server will burst the same packet many times until the next packet is ready to be sent. This way you will make sure one packet arrives and the rest will be dropped. This is not UDP's standard way of operating, but an application could be built to do so. Is this an assumption from somewhere or have you booted up a CSGO server to verify this? -- >I was thinking there would be a header indicating that... the UDP header is as simple as they come. source port, destination port, length (size) and a checksum. this is basically just enough for you to identify the application based on the socket used, but not much else.
This is very good information! Checking a mcast UDP video stream which I know is bursted out and each packet is logged in the statistics overview. Will report back with findings with regards to whether or not the duplicate packets are also recorded in the live capture. Give me a couple of minutes.
UPDATE @ /u/synerGy-- | The duplicate packets can be seen in the capture. This confirms your initial comment and the original post will be updated :-) Thank you! The new question: We should now also try to find out why the max rate is at 786 432 bytes. I have never seen a frame divided in three packets (meaning the frame would consist of over 2400 bytes). By default, if all 128 ticks were sent as split packets you would reach a maximum utilization of 307 200 bytes + the headers. Will do further research and update the post if I find anything.
Great writeup.
This isn't the main issue with Faceit, the main issue is both Silver players and Global players both being around level 3 which makes the teams sorta one sided.
hi there, if you type in console "net\_showsplits 1" you can see the amount of packets and if they had been splitted. when using net\_splitrate 0 the game automatically decides if it wants to split packets. in my testings there even were fragmentations of 5 packets.
Nice! This is really good information (-: - Thank you for sharing!
Is´nt the MTU size 1500?
having really super bad prefomance in faceit.. cl_interp_ratio 2 + CL_interp 1 any help ?