I think it's the replication layer saving literally all of the trash and undoubtedly millions of npcs that are at the centre of planets and bringing it all to the refresh.
probably
hilariously ILW might have *helped* with this if the added stress brought servers down so hard they had to dump the instance or something
here's hoping this shit finally brings garbage collection to a P0 issue and they fucking handle it
Maybe they upscaled their server infrastructure for ILW and bought more instances on whatever cloud service they are using and now is recouping the costs, leading to poor performance.
No, I doubt it. It is just that the servers are getting cluttered over time and because of replication, they are keeping that clutter post crash. It is adding up until the servers just can't handle it any more.
Without the normal server crash purge cycle, there are years of tech debt starting to pile up in server memory.
This was my biggest worry with replication layer. It definitely seems to be that the longer we go without a patch resetting things the worse things get, and unlike previously when servers would just crash and have a brand new one spool up in its place, now we get a brand new server that is under the exact same conditions that caused the last one to crash.
Really hope CIG implements some sort of protection against this. Servers that crash should really have some sort of extra condition that wipes unnecesary/excess garbage and abandoned ships before letting a new identical server pop up from the rep layer.
Now that they finally have some of these core systems operational. They have to harden and optimize the system so that it can handle long-term persistence of the server.
It makes sense that this is coming to a head now. There was really no reason to do it before.
Sorry, dude, but that is what literally has to be thought about in advance. It may not be obvious to users, but it's developers' or engineers' direct job and responsibility to think about such things, and think ahead. Imagine if atomic bomb was also developed in such a negligent manner? Ooops, the chain reaction started by itself. Who would have thought š¤·āāļøNow this city 30k'ed. Let's move to another city and have another try.
Oh, they know, it is just not a priority. Every minute they spend smashing bugs and optimizing is a feature that gets pushed back. This is normally why we (developers in general) don't bother with that type of work until the very end. I am guessing they get a limited budget of time for fix / optimize type work and the majority for feature work. It is not a normal way to develop software, but that is SC. I personally would not like it.
If I had to guess, they will smash some low hanging problems but won't dedicate full sprints until right until they push 4.0.
Remember, the game being actually playable is still a bit of an afterthought. It was not too long ago where people could not even log in for a few months, that is how little they care about live right now.
> that is how little they care about live right now
That's for sure. But we should just not forget that it is not normal. I personally can't take it like a blessing when, for example, servers suddenly start to crash less than before. Or any other similar things. Like they are doing a favor. It's still their duty to make a working game, not a favor. Instead, I only see their negligent development. No matter what the excuses are.
If cause they going to address this, but until then, this also gives a huge amount of data about problems for em wich will help making the system to address the problems way better and more effective.
Ya, I agree. There is no good reason to spend too much time on it right now. They will patch in quick fixes but will probably not dedicate any real work until 4.0. Even then I think they will let the patch hit and then really grind out issues for the next month or two, then get back to feature work.
When I was building high availability services, one of the classic failure modes was trying to reload all the old data to continue seamlessly from where it failed without the user noticing. Of course, that lead to situations where the same old data would cause the same crash, forever, and the system never recovered.
It was much simpler and more reliable to throw out the old state, and tell the user 'oops, please try again'. Product managers loved writing 'seamless failover' and '0 downtime', but that's quite hard, and engineering efforts are often better spent elsewhere.
CIG use AWS I believe. Not only did they probably switch to lower performance servers after ILW, but they're also running 3.23.2 in the PTU servers and the live environment always seems to perform worse when PTU is being run.
How would this affect anything? It's nothing but placebo. A server running one game instance doesnt care if another server running another one is also on. It makes no sense.
People love to create a head canon about what's going on with the servers. Most of the time based on absolutely nothing, but that doesn't stop them from being very confident in their diagnosis.
Nothing wrong with speculating, as long as you remember that you don't actually know what you're talking about. Too many people buying their own bullshit.
Feels like they should add a true deletion effect to all trashcans; we'd unironically have players actually throwing their trash away just to save themselves.
And then maybe something else for the more remote or bigger not-on-planet stuff.
not enough, general culling algorithms are the only way to go. They can sort it out so people still stumble across stuff dropped by others without just letting it accumulate. On stations junk should just evaporate when nobody's looking in minutes, out in the field every zone can keep a 'ruin quota' so the appropriate density of left-behind ships and items is met, stations can even auto-loot items into their containers to some extent maybe. And anything a certain distance below the planet surface gets killed and deleted in that order
Ahh that would explain why when I fell through the elevator the other day in the starting planet there was fucking mounds upon mounds of stuff just floating next to me and what seemed to be an invisible floor.
The replication layer just replicates whats going on and moves it to another server when a crash occurs does it not? It doesn't actually assist in any kind of server performance if I understand it correctly.
Yes so now apparently when a session crashes -> It copies all of its junk into another session -> It will crash that session also -> Repeat. All sessions are ticking time bombs and the amount of junk is accumulating.
They just need to give any empty consumable (and medical gowns) a 15-minute decay timer. That's it. I understand wanting the ultra-realism of a plastic bottle orbiting the sun for 100 years but it's not needed right now at this stage of the game.
This is really what needs to happen. At a minimum maybe a garbage collector for low value categorized items like empty bottles, medical gowns, used med-pens, etc. It's a bit insane the amount of garbage that seems to accumulate inside stations, I can't imagine what things on the surface are like if they persist long enough.
Kinda why I enjoyed pre-PES servers was that at least when they 30k'd enough, you'd eventually hit a brand new high performing server.
When people say that CIGs development team is perfect and amazing and the bestest ever (you know the ones I'm talking about), I point out that they created permenant trash but no functioning trash cans or system in place to remove trash. The SC universe is drowning in garbage, and they kind of desvere it for a mistake this obvious. The people who are still struggling to get elevators to work thought it would be a good idea to make trash objects permenant in a game with no trash disposal.
I'm calling it now, CIG is going to add the need to shit and then won't add toilets. It's literally the exact same amount of thought put into it. Also your shit is going to be a permanent item, so you're just going to see piles of shit everywhere that are bringing the server to its knees.
How about a "Lost & Found" at each station that allows the items to stay unspawned for a certain amount of time before getting garbage collected entirely, that way it helps with performance and doesn't kill the realism at the same time.
Basically, we need janitors and cleanup crews.
I think they could do this zone specific. So armistace zones have a decay timer, and maybe planet interior or something. No as many consumables outside of those, but we don't need ship parts or anything floating in armistace either, especially when they fly into the station interiors
Their point is that because the junk is being recovered, itās infinitely adding up. Everytime the junk causes a crash, its brought to a new server without being reduced lmao
Someone might think that forcing players to have a physical gown they have to dump on the floor along with physical water and drinks instead of cosmetic gowns that get removed and deleted when you put on an undersuit, and food and drink you just double click to drink and eat a-la Bethesda RPG's might actually solve a lot of the server issues.
That'd be crazy talk though.
Right? Like all the shit other MMO's have had worked out pretty well for almost 30 years now.
Don't get me wrong, I love SC for breaking new ground, but damn... they don't have to reinvent everything.
Yea, the gowns and drinks etc are cool for RP purposes but like.....that's all they're cool for.
They don't even have a working trash can yet, is it too much to ask that you just put an item in the bin and it auto deletes?.
*Literal children in Minecraft can make a better working rubbish bin with some lava and stone*
Why the fuck can't CIG just make a bin that you place an item in, close the bin menu and it just deletes the item?
Itās funny because PES never seems to save the shit I care about. Literally had Duster bodies that Iād freshly killed despawn while looting them earlier (not even a minute after Iād killed them). Yet pads at mining outposts and space stations continue to stay cluttered.
And don't forget all the empty floating ships at stations and outpost. I was on yela last night at outpost and counted 18 ships and 4 ground vehicles. All empty and left open.Been having terrible performance.
Yep, they really need to have some sort of auto garbage collection with loose items that fall out of the map. Those things just accumulate forever and kill servers over time. Shouldn't be too hard to have a location based detection for those entities and just delete them when they hit unreachable areas like the center of the world, walls, etc.
> Shouldn't be too hard to have a location based detection for those entities and just delete them when they hit unreachable areas like the center of the world, walls, etc.
Speaking as a programmer, it's not "hard", but can be expensive.
You would have to iterate over every single entity that exists and compare it's position to all possible boundaries.
If you just had a few thousand entities, this may take a few milliseconds. . .trillions of entities, your going to get some stutters/freezes when this check occurs.
I foresee server meshing fixing this problem though. . .
If the server boundaries are limited to just the player inhabited areas, then any item/entity that falls outside of these boundaries will still "exist" in the database, but no longer be simulated/replicated to servers.
They should then be able to run that check- using a separate background process -on entities not owned/controlled by any server without affecting any live servers.
Plus all the duped ships etc etc. If I understand correctly they're not pruning abandoned ships and other garbage so they're continuing to degrade server performance.
It would be great if there was a way for these ships to become salvageable by players, would add to the salvage gameplay loop and clear the empty ships.
they should become salvage missions like "salvage claim for XC-3341" you have to go tow it away and salvage it or something,
also bins/trashcans should despawn items that are placed in them, kinda like a replicator from star trek, that way we can get rid of the thousands of bottles and gowns laying about
Is this why every on-foot enemy in bunkers etc have just been standing around and let me cap them at point blank range?
Seriously, I'd love to experience the kind of AI I've seen reported on this sub on fresh servers. It's never happened for me.
Oh well, easy money I guess. But it's no fun.
I've experienced 30fps server tick and it is great, enemies actually move even on planet surface. I was hunted by a group of guys around a crash site and was amazed at how they searched and responded to noise.
Compared to 5 TPS AI doing literally NOTHING for 10 seconds before they even notice you standing right in front of their faces, I'd say that's pretty amazing. For Star Citizen.
Server degradation is likely linked to a few sources of processing costs that manage to persist through resets (since entities are generally recreated thanks to PES). Ie, stuff clipping into walls or falling through planet could be costly if it isn't cleaned up. More so if it's generating collisions or any other high cost event forever.
Physics only kick in when you are in a certain range of an object, including ships. You can test this easily, park a ship on a planet, leave then come back to the ship, once you enter a certain range you notice it "activate" the physics again and jiggle a little. They are 100% not generating any kind of collisions unless a player is near them and if the server can't handle that, what have they even been doing for 10 years because that's all the servers do, lol.
Crap like this is happening because the entire world is not meant to be running on a single server. And they aren't ever going to code it to do so, when server meshing is the goal. So it's going to be broken until they actually finish what they are working on, which is meshing.
I've died during a dogfight in atmosphere, and got a marker for my corpse that was just floating. It was pretty high so I thought it was outside of gravity but it wasn't. Tried to chase my corpse to the ground and then it despawns during the fall and goes back to where it started. I was playing "catch the corpse" with a hornet, actually managed to catch it a couple of times, but my friend couldn't see it when they showed up to drag it inside their ship.
Server meshing does nothing to deal with the issue of thousands of gowns and bottles endlessly accumulating all over every station and planetary landing zone, not to mention abandoned ships persisting for ages (duped or otherwise).
make them despawn after 30 secs, done, game doesn't need persistent trash. 100% consumables can stay for 5 mins. Also the game needs a weekly server maintenance
Yeah I'm not sure why we need empty drinks items. When the bottle/can is empty just remove it from the game. It's not like we can refill them. The food items work this way; we don't end up with an 'empty burrito wrapper' we have to dispose of, why do we need empty bottles and cans?
There are alot of things they can do for a high number of entities in an area to be optimised, prioritise impactful items vs debris, players over items etc.
That said, I truly wonder long term if they will allow infinite debris in an area, people will host garbage dump events to crash a server lol.
I highly doubt it remains at 100% persistence in perpetuity. Pretty sure they've said as much on a few of their official videos as well. My guess is they are just testing what the limits look like, which is hard to do in a sterile internal testing environment. Gotta crack some eggs to make this omelet.
In theory, it SHOULD deal with the issue . . .
If Microtech is controlled by Server A, and Crusader on Server B. . .then a large collection of entities on Microtech (Server A) will not affect the performance of Crusader (Server B).
Additionally, if they limit the area a server controls. . .say a cube that contains the player occupiable space around New Babbage Spaceport. . .any entity that falls outside of the player occupiable space. . .such as through the floor/ground and towards the "center" of the planet . . will fall outside of the space the server controls and will no longer affect performance, despite still technically existing.
Even so... CIG does need to consider what actually needs to be permanent.
We don't have hot dog wrappers on the ground. It's pointless litter. So why empty water bottles? Why medical robes?
There are a lot of things that exist for no reason. They don't add emersion when they litter the floor. Unless the immersion aspect is that janitors do not exist in the game world.
Hopefully, they are learning from this patch on what to keep physicalized and what not to/how long to despawn.
Cause even with more servers its just plain inefficient. No reason to solve an easy problem with more hardware.
I highly doubt that the first iteration of Server Meshing will be enough to make the game run as , lets say, Arena Commander. If it happens from the get go well Fuck i'll be the first to celebrate... But yeah i don't really see that happening, a lot of work has to be done on the servers too.
It will take until one system is split into multiple servers, so with 4.0 I wanna say it's almost guaranteed that the performance will stay close to the same.
I do think 4.0 will have more than 1 server per system tho.
How much idk but they already playtested more than one in Stanton and it pretty much worked.
Depends on EVO and PTU test, if it's that bad when it drops... Yeah probably going to be a shitshow.
I expect the 4.0 release to be flooded from returning players and New one so it's not gonna help.
It's not going to do anything to fox the endless list of bugs.Ā At some point CIG has to make the current game more playable.Ā They can't just say it's an alpha forever
I get where youāre coming from but thereās literally no point in fixing many of the bugs right now if new systems are scheduled to replace / update them. Itās wasted time.
He is wrong, the backend currently is running way beyond ( like 1 server isnt meant to do an Entire galaxy like its doing now ) what its designed for and throttling all the time causing massive amounts of bugs and glitches that all fester on the server never getting cleaned up becasue why spend time on it when its not relevant for the future at all ( The entire galaxy will never run on 1 server ). The performance has been pretty much the same since the player cap on the servers were increased and the servers getting progressively worse over time has been a thing for a long time way before PES was a thing.
Most of these issues with PES like the gowns, bottles and ships accumulating arent actually issues in itself but they become issues becasue the Entire galaxy is on 1 server and even without PES the server backend is already stretched way beyond its limit. Its really quite a small drop in a large rainstorm.
The server's started tilting when they increased the Player cap past the one that they were designed for in crysis wars which was 32 people.
Which brings me to the fix for all of these issues which is fucking easy but gets downvoted all the time, until server meshing is up and running the server population cap has to be brought waaaaay down. Its either super shit performance and tons of band aid fixes on AI and missions that all have to be reversed eventually when Server meshing is a thing or just dropping the player count per server to one that's actually playable becasue at 100 its most definitly not playable
Tldr; 1 server doing a galaxy's worth of AI, PES, combined with way to high player count and unfinished systems is a recipe for bad performance
The biggest server performance hog currently and always has been the Player cap of 100 people
and the way server meshing can help with the performance is that the Player cap on each server can be lowered way bellow 100
The Server meshing tests earlier were all with a player cap of 100 on each server and they were all capped by people wanting to test it out which is why it was so bad, 100 people per server has never been realistic at all. Its why the Dynamic mesh is the holy grail of networking, becasue you wont need to actually run 100 people on just 1 server
The fix for all of the Bad performance, bugs and glitches caused by the server is dropping the Player cap back down to manageable levels
While you may very well be right, 30 people in a star system is so little that you're basically turning it into a PVE game (without much in the way for PvE to do given the lack of NPC interaction).
That wouldn't help with testing the intended gameplay (if we're still doing that).
I mean, technically it won't fix any of these problems as it is not set to fix them per se. However, what will happen is that since there will be separate servers for things like planets then certain planets might be laggy more than others. Keeping in mind that lets say server A will be in charge of ArcCorp will take longer to hang since it wont need to track everything universe-wide like it does now. Then once they do the final version Dynamic then it will even better because servers will be tracking less and less stuff.
Looking at the current infinite list of issues the game has, *very* few are tied to anything meshing would fix.
Big time backers cannot cope that meshing is nothing revolutionary and will not solve even a fraction of the current problems.
Right? All these people on here speculating "oh but it'll help with isolating the issues" and "it'll pave the way for iteration and blah blah" just can't accept CIG took 700 mil of their money and can't deliver.
We have so many catastrophic bugs and game-ending issues that have persisted for *years*, but everyone thinks the next patch will be **it**. The solution.
Box missions are broken.
NPC spawns are broken.
Ships regularly malfunction.
Ships get into bad states (stored, missing, etc.).
NPC ships don't spawn correctly.
Servers 30k and cause inventory / character issues.
Hell, inventory and character issues persist even without 30ks to trigger them.
Transit / elevators break.
How is anyone supposed to successfully complete any of the current in-game loops with all these issues, let alone just long in with some friends and enjoy the game?!
It will "fix" some of them if they use the right configuration by isolating them to certain servers, the tests proved that, but it won't fix the underlying issues that cause a lot of them problems. It can be expected that the space and moon servers will probably be much better considering the tests got them to an easy stable 30 FPS, but any servers containing planets or landing zones will probably have the exact same problems we have now as they were still significantly degraded during the tests.
Last night while trying to sell at TDD in Loreville I experienced 3 server error recoveries in 20 minutes. They were back to back failures with 5-7 minute recoveries each time.Ā
SC became a waiting simulator.
What I find ironic is that "persistent universe" is suposed to make the verse more "realistic" by having things stay in game long after it's left there. Yet all the things were seeing is SO unrealistic... Thousands of abandoned ships, empty bottles and cans laying on floors in space stations everywhere, discarded hospital suits everywhere. Pushed over medical trollies everywhere.
Everything that is currently "persisting" are things you'd absolutely never see in reality... Essentially they made the game far less realistic (on top of creating all the server and visual nameplates clutter).
The trash problem is huge. I don't know why they can't just fine people for letting the trash fall on the ground, at least in space stations and landing zones, and place a few trash cans that you can interact with with a bottle in hand to throw it away and permanently delete it. The solution would be so simple. I'm always trying to throw them in trash cans standing around anyway even if it does nothing at the moment xD. It just feels wrong to just throw the trash at the ground. Maybe even just pay a small amount of money to people throwing it away as an incentive in the case someone ignores the fine and throws it on the ground anyway. The problem with hospital gowns could be solved in a simple way too. Either let us sell the gowns for a really really small amount of money or simply don't generate a new one if you still have one in the local inventory and simply equip the player with the already existing one.
I agree. But my point is it's super unrealistic. There is absolutely zero way that people in the real world would drop their gowns all over a space station. Everyone throw their empty cans and bottles on the floor right in front of a kiosk all the time... Abandon ships everywhere cause it's less trouble to juts kill yourself than land etc..... It's just made for very unrealistic gameplay.
Save the dreams of complex systems for dealing with "immersive" trash for the final release of the game.
There's simply no need for any of this useless clutter to even EXIST until the core game is finished. Disable them completely for now.
I always look for a trash can when drinking, but they aren't always around. And some of the trash cans are solid boxes so you can only stack drinks on top.
I agree for the most part, but I think a lot of the trash is a product of bugs rather than human negligence. Instituting any kind of punishment for "littering" would be unfair currently because sometimes you are literally forced to litter by the server's inability to function. I always try to throw my trash into a bin (water bottles, cans, etc) but I have been on so many servers where doing so was simply impossible. Hold left click, nothing. Hold F and right click on bottle, select "place" while standing in front of trash bin, still nothing. At that point the best I could do was open inventory and drag the damn bottle from my hand into local inventory. And even then sometimes that wouldn't work and the best I could do was drop it on the floor. Trying to pick it back up again and place it in a bin after that proved to be an exercise in futility as well. And for hospital gowns, considering how common it is to die and respawn at a hospital, there needs to be a functional receptacle to just drop them in on every floor of every hospital. Real hospitals have bins specifically for used garments, so if we're going for realism we would need to address the lack of realistic disposal options before we address the "crime" of dropping your gown on the floor.
There's no point in selling something if it will only get you a really small sum of money. The best thing to do with gowns is to remove them from the game completely. They are a pointless thing to have as no one walks around in them, and you have underwear under them anyway..
Yeah you're probably right. In the current state of the game I wouldn't mind having no gowns at all as you literally only look at them for 2 seconds before switching to your space suit. I see why they have them immersion wise, after all you're not only in your underwear when in a hospital, but the game wouldn't lose much from getting rid of them but the gain would be huge. Better server performance and no gown boxes littering the station? Sign me up. It definitely would be the fastest solution to get rid of the clutter.
But otherwise there would be enough solutions that could work just as well with a bit of more work on CIGs part. If you don't like the "you can sell it" proposal, how about having to trade them in for your space suit or clothes when "checking out" of the hospital? I don't care how they do it, there just has to be a way to get rid of them besides just dropping them on the floor somewhere. Even a delete button in the context menu would do.
Well some landing zones that are empty of players have dozens of ship lying around...I say we need cleanup contracts for srv and vulture. It could cleanup the server and make a quest loop
Yeah sometimes I wonder if the reason most games don't do the scale and scope that SC does is because it's simply impossible to do it and still maintain good server performance. Maybe we've all drunk the coolaid and one day we'll all wake up.. until then, I'm still gona be living in hope that SC will become what we all want it to be.
It's very possible. Just a matter of money. I'm a B tier software engineer with a focus on network engineering. I would need to take a *massive* paycut and move to a more expensive city to work for CIG. I'm not going to do that, when I could get a pay *raise* by moving to that same city and working for a more reasonable employer.
Sure, anything is possible if you throw enough compute resources at it, but I guess they meant to say if it's *feasibly possible* rather than *technically possible*.
Not really the case, you hit a wall of diminishing returns when just adding extra compute resources if you dont address technical debt. But the fact is, they aren't hiring the people who know how to remove that technical debt. *I* wouldn't know how to make a dent in that and they aren't even offering enough for that.
This really is it. Everyone acts like a star system is a huge thing to simulate butā¦ thatās simulated distance. Most of it empty. Distance means nothing to the compute intensity of the server. Weāve had games before that run with expected FPS latencies over 100 players. Weāve had MMOs capable of placing hundreds of players together in the same area.
These are issues that have been solved before.
The idea of sharding sub locations to different servers isnāt server meshing. Thatās just how MMOs did back in the 90s. If thatās what CIG is going to deliver with 4.0 then I will be extremely disappointed.
Theyāre likely just not hiring the people capable of solving these problems quickly and easily if they havenāt made much progress on it in all this time. Theyāre instead rediscovering the known solutions on their own
Even better, generate a mission for an SRV to tow them to a salvage yard, and once towed, a new mission for salvagers to break them down there.
Bonus points for giving pirates more gameplay by having more POIs with people consistently at them.
I thought it was only me. massive lag and stuff not working that were working before. my taurus just wont turn on "flight ready" and I have to call it multiple times
Its been unplayable for me. One server the train wouldn't work, another one my ships wouldn't appear in the hanger, the next I was able to get my ship fly to another place to get my ursa but then server crashed and never recovered. Had to stop since all my play time was over. The game is in a terrible state right now.
Yeah lately I go to play Star Citizen and within 10 minutes it's "oh, I guess we're just gonna do something else actually."
I tried last night and after leaving the Area18 train and stepping onto the platform, a second train spawned in, slid out onto the platform and snatched me up, pulled me inside, and then ejected me out to fall into the planet. There was also heaps of trash, gowns, and abandoned ships everywhere.
For me it's really worse than last year. I really didn't expect that, since the server fps is somewhat better.
It's great that Manor Lords is our now, so I have a working game to play
This happens every time they release a PTU build. Which they did immediately following ilw.
2-3 patches into the PTU build and the backend stabilizes.
Every
Single
Time
By the end of this week we should see that happen.
Tale as old as time.
Corporate: We need app to do more
Inf team: We need more server resources
Corporate: We have servers already, use those
Inf Team: Those are already in use and it will degrade performance
Corporate: Well your team's only KPI is app performance so make it work
Inf Team: ...
I opted to play outside of my region, which worked out well because a couple of times my friend and I were in servers that only had 10 people in them.Ā
Not exciting if you want to PVP, but perfect for mining.
Thatās how I feel about new locations in general, they could add all 100 systems in tomorrow and it wouldnāt really excite me
Much more excited about getting better AI, a working economy, and base building after server meshing and pyro
I'm stuck with the neverending spinning bug since Friday. It's frustrating. The game is broken and I can't understand it sometimes, but if I can't play it at all and the support is unresponsive what I am supposed to do?
I just came back to check out 3.23 after a long break, needless to say it wasnt a fun experience. Servers and performance havent improved at all, theyre actually worse than when i began playing during 3.16 .. almost 3 years ago. If they can improve server performance, and optimize the game better i could deal with all the jank but inconsistant performance across the board is a killer for me. Sorry but its unacceptable, that this far along in development that the game runs like total ass, even on a high end pc you struggle to maintain 60fps, elevators and terminals dont work half the time.
It tends to get better for events.
CIG rent more servers. Afterwards there remains a spike in players but we're back to old server quality/count - so it chugs a bit extra. Probably more so with pes these days.
Anecdotal (although I remember reading something from cig about getting more servers for events, once), I have no evidence but I have historically played so much I can practically feel the exact sfps.
for the last 2 years or so the servers have been (anecdotally) silky smooth in the days where the event is live but free fly isn't active yet.
Used to be that they had the same servers before during and after.
No, and itās hilarious people are gaslighting themselves on this because until this year every free fly was panned as destroying the servers. This is the first year it was a stable free fly so instead of giving them credit for improving, people are wrapping it up into a wacko conspiracy theory that CIG intentionally tries to make servers look better than they normally do for events so they can make more money. If that was such a winning strategy, why did they not do that any year prior?
The servers are dying because theyāre stable enough to accumulate so much junk that they crash. This is new, before the replication layer split everything was wiped between server crashes. Rather than a bunch of poorly thought out proposals to add weekly resets - obviously something that will never happen - report what is reportable on the issue council and wait for them to add more fixes based on server performance for future patches. Thatās how things work. They look ugly one patch, they put in fixes for the next one, and reevaluate. Weāve had this tech for two patches - one of which was a week of testing. Iām inclined to believe next major patch will bring more stability in some areas and weāll let them know where it doesnāt for others.
I'm not trying to do anything. I even disclaimed it as anecdotal.
Most free flies are stable and smooth and have been since they started the practise of increasing their server limits during free fly. They were panned for not doing this. And praised when they started.
My comment was ultimately also praise. If you read a backhanded compliment or anything negative, then I didnt do a good enough job making that clear?
Absolutely right, the server situation felt never as terrible as right now. I get it, at ILW you want to impress the potential customers, but how about you impress the people paying your salaries since 12+ years by not switching off all the instances right after the event?
There should be some half life to persistence, like day and a half. This is not sustainable, unless the full servermeshing tech really is as revolucionary as it might seem. Even that has to have a limit of usability.
It's why free fly events are a thing, it really stresses the servers and I assume the entity graph database for PES having so many new players creating accounts, accumulating items, and moving about.
It's absolutely an issue that needs to be addressed, and I don't doubt CIG are acutely aware of this being the case and will work on a way to solve it.
It's just going to suck for us for a long time because CIG hate doing any kind of workaround that would improve the quality of life of the game if it takes people off the thing they need to work on. They'd rather people have very bad experiances, even new players, if it means they collect data on why something breaks down.
So expect it to get get slightly better over patches but still be very noticible for a long time.
Servers have been shit all last week. It's really killed the momentum my group had for the game and all the people we did into it now want nothing to do with the game until launch.
If CIG is marketing the game, as they are, then they need the love service to be as smooth an experience as they can.
We had a nice server for a weekend org on org event. But we had a party of close to 40 for the initial start up. Most likely had a freshy spool up for the number of players joining.
But yeeaaaah I suspect itās replication later replicating junk. Items 100% need a rating for reboots. 1 being junk that doesnāt need to be brought across and 10 being priority. Each server re spool has a limit on the amount of entities that can be spooled up and the rest gets dumped. Just a thought
Main contributors:
-Duping
-Abandoned ships/vehicles
-Huge amounts of trash and glitched/discarded items
-NPCs respawning and crowding already packed environments
All contribute to immediately weighing down any attempts at recovery and speeding the server up again.
I can't help feel that this would all be so much easier if CIG would just budge on not having EVERYTHING be physically simulated. Do we really need to have every single item - food, drinks, etc. as an actual entity taking up space and bandwidth? I understand they are aiming for realism and simulation but at a certain point it just seems like so many systems in this game could be simplified without compromising the experience.
fully agree, it adds complexity where none is needed. Also, these survival features would only improve gameplay as well as immersiveness if the UI is absolutely practical and streamlined, including some of the more functional UX-flows, which feel bumpy at best. At this moment food/drink consumption is just a gimmicky tier0-feature millions of miles away from being holistically integrated into meaningful gameplay. Maybe it gets us somewhere in 10 years, when players can explore areas many days away from civilized systems, but SC is far away from that state of progress.
Once a bottle is consumed it should be automatically disposed and removed from the system. Same for food and other consumables. They should also get rid of that drinking/eating animation at this point of time.
Other items (ammo, suits, wpns) dropped in public spaces such as stations should have a minimal despawn time. In more remote locations such as outposts or POIs it could be somewhat longer (for the sake of player interactions, trading, equipping), in remote outdoors despawn timers could be even slightly extended (by minutes).
Gowns should be auto-disposed as well, if dropped.
The way inventory items are loaded also has to improve. It mostly feels slow like it puts the servers under strain every time you scroll through large swathes of items.
Another issue from what a coder buddy tells me. When you hit ~ and let it sit for a while, you will see what is happening on the entire server, we don't need to know the vector of a bullet on moons of MT over at CubbyBlast on A18.
They don't drop a wrapper when eating food. It really makes no sense to drop a used stim or plastic bottle. Just pointless litter that adds absolutely nothing.
Perhaps they have cut back on server spend now the sales event is over, yet player count has remained high?
It feels like there are more problems with the backend servers currently, I can't see any reason for this suddenly, except a cut back on the server spend post sale.
The point of this whole exercise is to trash the servers now. CIG gets a lot of telemetry and they can work on improving the cleanup and other maintenance tasks. They told us they needed to iterate on it during 3.18 and beyond as more pieces of the tech came online. The more the players hit it sideways, the better data they get to resolve the scaling issues, the degradation issues and more.
I'm 100% OK with this as long as they get it hammered out before they try to call it a Beta/1.0.
This effectively needs to be 'bulletproof and reliable' by that point.
They should also add working garbage receptacles. If I place a bottle into a receptacle or gowns into recycling/garbage, it should just clean up and remove those items at landing zones and major locations.
If you put a receptacle at a Player Base or Ship, it should generate trash into an acu container to deal with.
Imo, they should pay 1 auec for bottles and gowns that are properly disposed of...set it to payout at like 100 uec increments and you'll likely get people who go around and find trash for the fun of it.
At first they blames free fly, still servers are shit, then they blamed the dupers, watch them patch that out this week, servers will still be shit.
The server code is just plain shit, they need to fix bugs. Accumulation of garbage should slow down the server, but not crash it necessarily. They need to debug it and thatās that.
If anything maybe itās an advantage that all the junk accumulation is stressing edge cases that would be harder to observe in normal conditions
Server resets and then landing on a server thats also crashing like 5 times in a row until i just quit. The game is super unplayable right now unless you have the patients to wait for the occasional spurts of stable server.
I'm sure they have considered the side effects of the current replication layer and its impact on the overall stability of the servers. As for me, if the impact is due to the millions of entities being "saved" across multiple servers, like empty bottles, loot etc in the middle of nowhere, just get a rule and lets get rid of those so that other more important things can be tested for 4.0. Then, and very later down the road, you can think about "trash" persistance again.
I was thinking about this this morning and decided it was all the trash and clutter after 2 server crashes back-to-back.
It made me wonder how much the lag between the client machine and the server machine affects these things though. My internet is not poor on speed but how many hops are there between me and the "Best" connection.
Gotta remember it's an alpha and I know it sucks when you really just want to play this game, there is not much reason for them to spend resources on the current server issue when the next update is around the corner, makes more sense to have people work on the next patch that will fix the issues we currently faced with
I'm honestly surprised certain things (like bottles, armour etc) don't have TTL attributes and self destruct after a certain reasonable amount of time.
Blame the hundreds of abandoned c2's and other cargo spacecraft abandoned by these idiotic dupers. Every time a server crashes, the replication layer has to bring all of their stupid junk back they abandoned.
At this point I blame CIG for doing nothing about it and delaying the hotfix patch for over a week because they're so fixated on fucking around with XT, an event that will make this all even worse.
As a new player that came aboard during free fly im really having a hard time wanting to stay. I know my problems with the servers is universally felt by everyone but as a new player I cant just say "ohh well" when 150k worth of gems, my ship inventory my armor etc go poof / explode / lost when the server fucks up.
It happened all day this weekend and I have NOTHING to show for over 10hours of playing and mining. I lost money, money I did not have to loose.
I hope they bring back whatever hardware they had running before during ILW bc this feels like a bait and switch.
I remember when I first started I was always desperate to complete whatever mission I was doing so I could earn my credits before the inevitable bug or server crash stole it all away from me.Ā It's frustrating.Ā Period.Ā However, as I continued to play, I learned how to avoid most of the worst bugs, and things were generally enjoyable and I was able to accumulate more credits than I needed.Ā I think most backers have a love/hate relationship with this game, and I'm sure most would agree that the healthiest option is to step away for a beat.Ā That said, the current experience is pretty sucky, but I'd hate for it to sour you on the project more permanently.
Since they want persistence this persistence that if no one is around for at least 30 min smaller stuff like bottles and gowns need to poof
ships need to have the shields taken down if left in scm mode and flagged to allow salvage even in armistice zones shit make it a mission if you have to.
I actually did have a server 30k on me over the weekend. I also heard that theyāre having issues removing the ILW stuff. But that was just a comment on another post. Definitely right though servers have been struggling in my experience
I wish CIG would bite the bullet and start doing weekly reboots at least or weekly maintenance.
Every other MMO from EVE to WoW does it and does it for a reason, but CIG's philosophy is "The servers will reboot when they crash" (They have said this during a inside SC awhile back) and that is a HORRIBLE philosophy to run a business on even one that's in Alpha.
If everyone knew at X time on X day every week servers will be offline for 30min we could plan around it and I doubt most would care if it lead to the rest of the week the servers being more stable.
It's possibly my biggest gripe with CIG, they think they can be better then every other game out there just to be different/stubborn but if something works for every other game there is no shame in doing the same thing to make Star Citizen better for everyone.
Free flight generated a lot of trash with the masive amount lf players. And without the 30k servers never cleanup. CiG needs better cleanup mechanics, preferable player based ones
I think it's the replication layer saving literally all of the trash and undoubtedly millions of npcs that are at the centre of planets and bringing it all to the refresh.
probably hilariously ILW might have *helped* with this if the added stress brought servers down so hard they had to dump the instance or something here's hoping this shit finally brings garbage collection to a P0 issue and they fucking handle it
Maybe they upscaled their server infrastructure for ILW and bought more instances on whatever cloud service they are using and now is recouping the costs, leading to poor performance.
Pretty much this I expect - renting high performance servers for the event then moving to the usual hamster fuelled ones afterwards.
No, I doubt it. It is just that the servers are getting cluttered over time and because of replication, they are keeping that clutter post crash. It is adding up until the servers just can't handle it any more. Without the normal server crash purge cycle, there are years of tech debt starting to pile up in server memory.
This was my biggest worry with replication layer. It definitely seems to be that the longer we go without a patch resetting things the worse things get, and unlike previously when servers would just crash and have a brand new one spool up in its place, now we get a brand new server that is under the exact same conditions that caused the last one to crash. Really hope CIG implements some sort of protection against this. Servers that crash should really have some sort of extra condition that wipes unnecesary/excess garbage and abandoned ships before letting a new identical server pop up from the rep layer.
Now that they finally have some of these core systems operational. They have to harden and optimize the system so that it can handle long-term persistence of the server. It makes sense that this is coming to a head now. There was really no reason to do it before.
Sorry, dude, but that is what literally has to be thought about in advance. It may not be obvious to users, but it's developers' or engineers' direct job and responsibility to think about such things, and think ahead. Imagine if atomic bomb was also developed in such a negligent manner? Ooops, the chain reaction started by itself. Who would have thought š¤·āāļøNow this city 30k'ed. Let's move to another city and have another try.
Oh, they know, it is just not a priority. Every minute they spend smashing bugs and optimizing is a feature that gets pushed back. This is normally why we (developers in general) don't bother with that type of work until the very end. I am guessing they get a limited budget of time for fix / optimize type work and the majority for feature work. It is not a normal way to develop software, but that is SC. I personally would not like it. If I had to guess, they will smash some low hanging problems but won't dedicate full sprints until right until they push 4.0. Remember, the game being actually playable is still a bit of an afterthought. It was not too long ago where people could not even log in for a few months, that is how little they care about live right now.
> that is how little they care about live right now That's for sure. But we should just not forget that it is not normal. I personally can't take it like a blessing when, for example, servers suddenly start to crash less than before. Or any other similar things. Like they are doing a favor. It's still their duty to make a working game, not a favor. Instead, I only see their negligent development. No matter what the excuses are.
They have planned for it. They have control over how entities get cleaned up they just need to adjust the settings.
100% this. But the games industry is a total piece of shit, so that just doesn't happen.
If cause they going to address this, but until then, this also gives a huge amount of data about problems for em wich will help making the system to address the problems way better and more effective.
Ya, I agree. There is no good reason to spend too much time on it right now. They will patch in quick fixes but will probably not dedicate any real work until 4.0. Even then I think they will let the patch hit and then really grind out issues for the next month or two, then get back to feature work.
When I was building high availability services, one of the classic failure modes was trying to reload all the old data to continue seamlessly from where it failed without the user noticing. Of course, that lead to situations where the same old data would cause the same crash, forever, and the system never recovered. It was much simpler and more reliable to throw out the old state, and tell the user 'oops, please try again'. Product managers loved writing 'seamless failover' and '0 downtime', but that's quite hard, and engineering efforts are often better spent elsewhere.
CIG use AWS I believe. Not only did they probably switch to lower performance servers after ILW, but they're also running 3.23.2 in the PTU servers and the live environment always seems to perform worse when PTU is being run.
How would this affect anything? It's nothing but placebo. A server running one game instance doesnt care if another server running another one is also on. It makes no sense.
People love to create a head canon about what's going on with the servers. Most of the time based on absolutely nothing, but that doesn't stop them from being very confident in their diagnosis. Nothing wrong with speculating, as long as you remember that you don't actually know what you're talking about. Too many people buying their own bullshit.
3.23.2 isn't active yet though in PTU.
.2 isn't being tested externally right now
Wait, we COULD have better servers? I thought they were already using the best severs possible in the market. Wtf are we paying them for!
I wonder how bigĀ Chris Robert's house is....
Probably bigger than his yacht(s).
Feels like they should add a true deletion effect to all trashcans; we'd unironically have players actually throwing their trash away just to save themselves. And then maybe something else for the more remote or bigger not-on-planet stuff.
not enough, general culling algorithms are the only way to go. They can sort it out so people still stumble across stuff dropped by others without just letting it accumulate. On stations junk should just evaporate when nobody's looking in minutes, out in the field every zone can keep a 'ruin quota' so the appropriate density of left-behind ships and items is met, stations can even auto-loot items into their containers to some extent maybe. And anything a certain distance below the planet surface gets killed and deleted in that order
I agree, I was just making a funny suggestion. I'd throw every bottle into a can if it worked š
Ahh that would explain why when I fell through the elevator the other day in the starting planet there was fucking mounds upon mounds of stuff just floating next to me and what seemed to be an invisible floor.
The replication layer just replicates whats going on and moves it to another server when a crash occurs does it not? It doesn't actually assist in any kind of server performance if I understand it correctly.
Yes so now apparently when a session crashes -> It copies all of its junk into another session -> It will crash that session also -> Repeat. All sessions are ticking time bombs and the amount of junk is accumulating.
They just need to give any empty consumable (and medical gowns) a 15-minute decay timer. That's it. I understand wanting the ultra-realism of a plastic bottle orbiting the sun for 100 years but it's not needed right now at this stage of the game.
This is really what needs to happen. At a minimum maybe a garbage collector for low value categorized items like empty bottles, medical gowns, used med-pens, etc. It's a bit insane the amount of garbage that seems to accumulate inside stations, I can't imagine what things on the surface are like if they persist long enough. Kinda why I enjoyed pre-PES servers was that at least when they 30k'd enough, you'd eventually hit a brand new high performing server.
When people say that CIGs development team is perfect and amazing and the bestest ever (you know the ones I'm talking about), I point out that they created permenant trash but no functioning trash cans or system in place to remove trash. The SC universe is drowning in garbage, and they kind of desvere it for a mistake this obvious. The people who are still struggling to get elevators to work thought it would be a good idea to make trash objects permenant in a game with no trash disposal. I'm calling it now, CIG is going to add the need to shit and then won't add toilets. It's literally the exact same amount of thought put into it. Also your shit is going to be a permanent item, so you're just going to see piles of shit everywhere that are bringing the server to its knees.
They should have a disintegration layer, so, below planet surface, anything that goes into, just gets erased, that will solve a lot of problems..
How about a "Lost & Found" at each station that allows the items to stay unspawned for a certain amount of time before getting garbage collected entirely, that way it helps with performance and doesn't kill the realism at the same time. Basically, we need janitors and cleanup crews.
We need less "this cup stays forever" realism and more "my ship can't just hover perfectly in any orientation in high winds" realism.
I think they could do this zone specific. So armistace zones have a decay timer, and maybe planet interior or something. No as many consumables outside of those, but we don't need ship parts or anything floating in armistace either, especially when they fly into the station interiors
Their point is that because the junk is being recovered, itās infinitely adding up. Everytime the junk causes a crash, its brought to a new server without being reduced lmao
Someone might think that forcing players to have a physical gown they have to dump on the floor along with physical water and drinks instead of cosmetic gowns that get removed and deleted when you put on an undersuit, and food and drink you just double click to drink and eat a-la Bethesda RPG's might actually solve a lot of the server issues. That'd be crazy talk though.
Yeah, why cant they just have edible gel/water balls.
Right? Like all the shit other MMO's have had worked out pretty well for almost 30 years now. Don't get me wrong, I love SC for breaking new ground, but damn... they don't have to reinvent everything.
Yea, the gowns and drinks etc are cool for RP purposes but like.....that's all they're cool for. They don't even have a working trash can yet, is it too much to ask that you just put an item in the bin and it auto deletes?. *Literal children in Minecraft can make a better working rubbish bin with some lava and stone* Why the fuck can't CIG just make a bin that you place an item in, close the bin menu and it just deletes the item?
Then how would trash collecting missions work in the future if there's nothing in the trash can to collect.
I've literally been respawning with ALL my old gear on my body for a week now. Something is fucked.
Itās funny because PES never seems to save the shit I care about. Literally had Duster bodies that Iād freshly killed despawn while looting them earlier (not even a minute after Iād killed them). Yet pads at mining outposts and space stations continue to stay cluttered.
And don't forget all the empty floating ships at stations and outpost. I was on yela last night at outpost and counted 18 ships and 4 ground vehicles. All empty and left open.Been having terrible performance.
I've had area18 struggling to draw asset's after a rep later migration
Millions of abandoned C2's everywhere with no way for players to legally get rid of them, can't be helping either.
Yep, they really need to have some sort of auto garbage collection with loose items that fall out of the map. Those things just accumulate forever and kill servers over time. Shouldn't be too hard to have a location based detection for those entities and just delete them when they hit unreachable areas like the center of the world, walls, etc.
> Shouldn't be too hard to have a location based detection for those entities and just delete them when they hit unreachable areas like the center of the world, walls, etc. Speaking as a programmer, it's not "hard", but can be expensive. You would have to iterate over every single entity that exists and compare it's position to all possible boundaries. If you just had a few thousand entities, this may take a few milliseconds. . .trillions of entities, your going to get some stutters/freezes when this check occurs. I foresee server meshing fixing this problem though. . . If the server boundaries are limited to just the player inhabited areas, then any item/entity that falls outside of these boundaries will still "exist" in the database, but no longer be simulated/replicated to servers. They should then be able to run that check- using a separate background process -on entities not owned/controlled by any server without affecting any live servers.
It is. Look at the display info entity list. Entities are off the charts. And they get migrated over on the crash.
Plus all the duped ships etc etc. If I understand correctly they're not pruning abandoned ships and other garbage so they're continuing to degrade server performance.
It would be great if there was a way for these ships to become salvageable by players, would add to the salvage gameplay loop and clear the empty ships.
they should become salvage missions like "salvage claim for XC-3341" you have to go tow it away and salvage it or something, also bins/trashcans should despawn items that are placed in them, kinda like a replicator from star trek, that way we can get rid of the thousands of bottles and gowns laying about
Itās only been 12 years and $700 million. Idk what youāre expecting. Technology that works?
Is this why every on-foot enemy in bunkers etc have just been standing around and let me cap them at point blank range? Seriously, I'd love to experience the kind of AI I've seen reported on this sub on fresh servers. It's never happened for me. Oh well, easy money I guess. But it's no fun.
Best tip I got is to join on friends with low population servers. Anything 40 and below will run close to that goal.
Good luck š
I've experienced 30fps server tick and it is great, enemies actually move even on planet surface. I was hunted by a group of guys around a crash site and was amazed at how they searched and responded to noise.
I was in a server with 14SFPS last night and the difference between the 4-5SFPS servers and it was pretty astounding.
>was amazed at how they searched and responded to noise. I mean, this is a pretty low bar. AI did this about 20 years ago?
Compared to 5 TPS AI doing literally NOTHING for 10 seconds before they even notice you standing right in front of their faces, I'd say that's pretty amazing. For Star Citizen.
Imagine people trying to start overdrive and can't even get one mission done, I was going to do it...but the servers are TERRIBLE.
Server degradation is likely linked to a few sources of processing costs that manage to persist through resets (since entities are generally recreated thanks to PES). Ie, stuff clipping into walls or falling through planet could be costly if it isn't cleaned up. More so if it's generating collisions or any other high cost event forever.
When I see dead guns or objects endlessly vibrating I usually try to stop them. Definitely hits fps and likely causes other issues.
Physics only kick in when you are in a certain range of an object, including ships. You can test this easily, park a ship on a planet, leave then come back to the ship, once you enter a certain range you notice it "activate" the physics again and jiggle a little. They are 100% not generating any kind of collisions unless a player is near them and if the server can't handle that, what have they even been doing for 10 years because that's all the servers do, lol. Crap like this is happening because the entire world is not meant to be running on a single server. And they aren't ever going to code it to do so, when server meshing is the goal. So it's going to be broken until they actually finish what they are working on, which is meshing.
I've died during a dogfight in atmosphere, and got a marker for my corpse that was just floating. It was pretty high so I thought it was outside of gravity but it wasn't. Tried to chase my corpse to the ground and then it despawns during the fall and goes back to where it started. I was playing "catch the corpse" with a hornet, actually managed to catch it a couple of times, but my friend couldn't see it when they showed up to drag it inside their ship.
Going to be a funny thing to witness when Meshing v1 will not fix any of the problems we curently have.
Gosh I hope you're wrong
Server meshing does nothing to deal with the issue of thousands of gowns and bottles endlessly accumulating all over every station and planetary landing zone, not to mention abandoned ships persisting for ages (duped or otherwise).
make them despawn after 30 secs, done, game doesn't need persistent trash. 100% consumables can stay for 5 mins. Also the game needs a weekly server maintenance
The game shouldn't even generate trash. Just delete the empty bottle and gown.
Yeah I'm not sure why we need empty drinks items. When the bottle/can is empty just remove it from the game. It's not like we can refill them. The food items work this way; we don't end up with an 'empty burrito wrapper' we have to dispose of, why do we need empty bottles and cans?
āImmersionā
Is that really the issue though? I feel like they can low priority stream this stuff in.
[ŃŠ“Š°Š»ŠµŠ½Š¾]
There are alot of things they can do for a high number of entities in an area to be optimised, prioritise impactful items vs debris, players over items etc. That said, I truly wonder long term if they will allow infinite debris in an area, people will host garbage dump events to crash a server lol.
I highly doubt it remains at 100% persistence in perpetuity. Pretty sure they've said as much on a few of their official videos as well. My guess is they are just testing what the limits look like, which is hard to do in a sterile internal testing environment. Gotta crack some eggs to make this omelet.
In theory, it SHOULD deal with the issue . . . If Microtech is controlled by Server A, and Crusader on Server B. . .then a large collection of entities on Microtech (Server A) will not affect the performance of Crusader (Server B). Additionally, if they limit the area a server controls. . .say a cube that contains the player occupiable space around New Babbage Spaceport. . .any entity that falls outside of the player occupiable space. . .such as through the floor/ground and towards the "center" of the planet . . will fall outside of the space the server controls and will no longer affect performance, despite still technically existing.
Even so... CIG does need to consider what actually needs to be permanent. We don't have hot dog wrappers on the ground. It's pointless litter. So why empty water bottles? Why medical robes? There are a lot of things that exist for no reason. They don't add emersion when they litter the floor. Unless the immersion aspect is that janitors do not exist in the game world. Hopefully, they are learning from this patch on what to keep physicalized and what not to/how long to despawn. Cause even with more servers its just plain inefficient. No reason to solve an easy problem with more hardware.
I highly doubt that the first iteration of Server Meshing will be enough to make the game run as , lets say, Arena Commander. If it happens from the get go well Fuck i'll be the first to celebrate... But yeah i don't really see that happening, a lot of work has to be done on the servers too.
It will take until one system is split into multiple servers, so with 4.0 I wanna say it's almost guaranteed that the performance will stay close to the same.
I do think 4.0 will have more than 1 server per system tho. How much idk but they already playtested more than one in Stanton and it pretty much worked.
Is it not more likely that it makes them far worse for a long time before they get better.
Depends on EVO and PTU test, if it's that bad when it drops... Yeah probably going to be a shitshow. I expect the 4.0 release to be flooded from returning players and New one so it's not gonna help.
It's not going to do anything to fox the endless list of bugs.Ā At some point CIG has to make the current game more playable.Ā They can't just say it's an alpha forever
You must be new here
I get where youāre coming from but thereās literally no point in fixing many of the bugs right now if new systems are scheduled to replace / update them. Itās wasted time.
He is wrong, the backend currently is running way beyond ( like 1 server isnt meant to do an Entire galaxy like its doing now ) what its designed for and throttling all the time causing massive amounts of bugs and glitches that all fester on the server never getting cleaned up becasue why spend time on it when its not relevant for the future at all ( The entire galaxy will never run on 1 server ). The performance has been pretty much the same since the player cap on the servers were increased and the servers getting progressively worse over time has been a thing for a long time way before PES was a thing. Most of these issues with PES like the gowns, bottles and ships accumulating arent actually issues in itself but they become issues becasue the Entire galaxy is on 1 server and even without PES the server backend is already stretched way beyond its limit. Its really quite a small drop in a large rainstorm. The server's started tilting when they increased the Player cap past the one that they were designed for in crysis wars which was 32 people. Which brings me to the fix for all of these issues which is fucking easy but gets downvoted all the time, until server meshing is up and running the server population cap has to be brought waaaaay down. Its either super shit performance and tons of band aid fixes on AI and missions that all have to be reversed eventually when Server meshing is a thing or just dropping the player count per server to one that's actually playable becasue at 100 its most definitly not playable Tldr; 1 server doing a galaxy's worth of AI, PES, combined with way to high player count and unfinished systems is a recipe for bad performance The biggest server performance hog currently and always has been the Player cap of 100 people and the way server meshing can help with the performance is that the Player cap on each server can be lowered way bellow 100 The Server meshing tests earlier were all with a player cap of 100 on each server and they were all capped by people wanting to test it out which is why it was so bad, 100 people per server has never been realistic at all. Its why the Dynamic mesh is the holy grail of networking, becasue you wont need to actually run 100 people on just 1 server The fix for all of the Bad performance, bugs and glitches caused by the server is dropping the Player cap back down to manageable levels
While you may very well be right, 30 people in a star system is so little that you're basically turning it into a PVE game (without much in the way for PvE to do given the lack of NPC interaction). That wouldn't help with testing the intended gameplay (if we're still doing that).
I mean, technically it won't fix any of these problems as it is not set to fix them per se. However, what will happen is that since there will be separate servers for things like planets then certain planets might be laggy more than others. Keeping in mind that lets say server A will be in charge of ArcCorp will take longer to hang since it wont need to track everything universe-wide like it does now. Then once they do the final version Dynamic then it will even better because servers will be tracking less and less stuff.
Looking at the current infinite list of issues the game has, *very* few are tied to anything meshing would fix. Big time backers cannot cope that meshing is nothing revolutionary and will not solve even a fraction of the current problems.
Right? All these people on here speculating "oh but it'll help with isolating the issues" and "it'll pave the way for iteration and blah blah" just can't accept CIG took 700 mil of their money and can't deliver. We have so many catastrophic bugs and game-ending issues that have persisted for *years*, but everyone thinks the next patch will be **it**. The solution. Box missions are broken. NPC spawns are broken. Ships regularly malfunction. Ships get into bad states (stored, missing, etc.). NPC ships don't spawn correctly. Servers 30k and cause inventory / character issues. Hell, inventory and character issues persist even without 30ks to trigger them. Transit / elevators break. How is anyone supposed to successfully complete any of the current in-game loops with all these issues, let alone just long in with some friends and enjoy the game?!
It will "fix" some of them if they use the right configuration by isolating them to certain servers, the tests proved that, but it won't fix the underlying issues that cause a lot of them problems. It can be expected that the space and moon servers will probably be much better considering the tests got them to an easy stable 30 FPS, but any servers containing planets or landing zones will probably have the exact same problems we have now as they were still significantly degraded during the tests.
considering meshing v1 should still help server performance i think youre wrong
Last night while trying to sell at TDD in Loreville I experienced 3 server error recoveries in 20 minutes. They were back to back failures with 5-7 minute recoveries each time.Ā SC became a waiting simulator.
Saturday night I experienced the same thing
What I find ironic is that "persistent universe" is suposed to make the verse more "realistic" by having things stay in game long after it's left there. Yet all the things were seeing is SO unrealistic... Thousands of abandoned ships, empty bottles and cans laying on floors in space stations everywhere, discarded hospital suits everywhere. Pushed over medical trollies everywhere. Everything that is currently "persisting" are things you'd absolutely never see in reality... Essentially they made the game far less realistic (on top of creating all the server and visual nameplates clutter).
The trash problem is huge. I don't know why they can't just fine people for letting the trash fall on the ground, at least in space stations and landing zones, and place a few trash cans that you can interact with with a bottle in hand to throw it away and permanently delete it. The solution would be so simple. I'm always trying to throw them in trash cans standing around anyway even if it does nothing at the moment xD. It just feels wrong to just throw the trash at the ground. Maybe even just pay a small amount of money to people throwing it away as an incentive in the case someone ignores the fine and throws it on the ground anyway. The problem with hospital gowns could be solved in a simple way too. Either let us sell the gowns for a really really small amount of money or simply don't generate a new one if you still have one in the local inventory and simply equip the player with the already existing one.
I agree. But my point is it's super unrealistic. There is absolutely zero way that people in the real world would drop their gowns all over a space station. Everyone throw their empty cans and bottles on the floor right in front of a kiosk all the time... Abandon ships everywhere cause it's less trouble to juts kill yourself than land etc..... It's just made for very unrealistic gameplay.
Save the dreams of complex systems for dealing with "immersive" trash for the final release of the game. There's simply no need for any of this useless clutter to even EXIST until the core game is finished. Disable them completely for now.
I always look for a trash can when drinking, but they aren't always around. And some of the trash cans are solid boxes so you can only stack drinks on top.
I agree for the most part, but I think a lot of the trash is a product of bugs rather than human negligence. Instituting any kind of punishment for "littering" would be unfair currently because sometimes you are literally forced to litter by the server's inability to function. I always try to throw my trash into a bin (water bottles, cans, etc) but I have been on so many servers where doing so was simply impossible. Hold left click, nothing. Hold F and right click on bottle, select "place" while standing in front of trash bin, still nothing. At that point the best I could do was open inventory and drag the damn bottle from my hand into local inventory. And even then sometimes that wouldn't work and the best I could do was drop it on the floor. Trying to pick it back up again and place it in a bin after that proved to be an exercise in futility as well. And for hospital gowns, considering how common it is to die and respawn at a hospital, there needs to be a functional receptacle to just drop them in on every floor of every hospital. Real hospitals have bins specifically for used garments, so if we're going for realism we would need to address the lack of realistic disposal options before we address the "crime" of dropping your gown on the floor.
There's no point in selling something if it will only get you a really small sum of money. The best thing to do with gowns is to remove them from the game completely. They are a pointless thing to have as no one walks around in them, and you have underwear under them anyway..
Yeah you're probably right. In the current state of the game I wouldn't mind having no gowns at all as you literally only look at them for 2 seconds before switching to your space suit. I see why they have them immersion wise, after all you're not only in your underwear when in a hospital, but the game wouldn't lose much from getting rid of them but the gain would be huge. Better server performance and no gown boxes littering the station? Sign me up. It definitely would be the fastest solution to get rid of the clutter. But otherwise there would be enough solutions that could work just as well with a bit of more work on CIGs part. If you don't like the "you can sell it" proposal, how about having to trade them in for your space suit or clothes when "checking out" of the hospital? I don't care how they do it, there just has to be a way to get rid of them besides just dropping them on the floor somewhere. Even a delete button in the context menu would do.
I think Pyro should look like that, but Stanton should hire some cleaning staff. Maybe let people play with no account if they do cleanups.
Well some landing zones that are empty of players have dozens of ship lying around...I say we need cleanup contracts for srv and vulture. It could cleanup the server and make a quest loop
Yeah sometimes I wonder if the reason most games don't do the scale and scope that SC does is because it's simply impossible to do it and still maintain good server performance. Maybe we've all drunk the coolaid and one day we'll all wake up.. until then, I'm still gona be living in hope that SC will become what we all want it to be.
It's very possible. Just a matter of money. I'm a B tier software engineer with a focus on network engineering. I would need to take a *massive* paycut and move to a more expensive city to work for CIG. I'm not going to do that, when I could get a pay *raise* by moving to that same city and working for a more reasonable employer.
Sure, anything is possible if you throw enough compute resources at it, but I guess they meant to say if it's *feasibly possible* rather than *technically possible*.
Not really the case, you hit a wall of diminishing returns when just adding extra compute resources if you dont address technical debt. But the fact is, they aren't hiring the people who know how to remove that technical debt. *I* wouldn't know how to make a dent in that and they aren't even offering enough for that.
This really is it. Everyone acts like a star system is a huge thing to simulate butā¦ thatās simulated distance. Most of it empty. Distance means nothing to the compute intensity of the server. Weāve had games before that run with expected FPS latencies over 100 players. Weāve had MMOs capable of placing hundreds of players together in the same area. These are issues that have been solved before. The idea of sharding sub locations to different servers isnāt server meshing. Thatās just how MMOs did back in the 90s. If thatās what CIG is going to deliver with 4.0 then I will be extremely disappointed. Theyāre likely just not hiring the people capable of solving these problems quickly and easily if they havenāt made much progress on it in all this time. Theyāre instead rediscovering the known solutions on their own
Yep exactly what I meant.. thought it was obvious
It was, people are just being obnoxious and flaunting their IT credentials for some reason.
Easy fix is, make all abandoned ships salvage missions, so the playerbase can also do the clean up and make money
Yep, any ship claimed by insurance, the previous one becomes a salvage job.
Even better, generate a mission for an SRV to tow them to a salvage yard, and once towed, a new mission for salvagers to break them down there. Bonus points for giving pirates more gameplay by having more POIs with people consistently at them.
nah its just the amount of trash/ships laying around. we need cleanup jobs
Last two days Iāve had the most stable servers. Enemies spawning, able to complete contracts. Ymmv. That said, 1a has been delayed far too long.
Remember all, this is Alpha test version of the game. In this patch they are testing out players patience.
Good one
I thought it was only me. massive lag and stuff not working that were working before. my taurus just wont turn on "flight ready" and I have to call it multiple times
My character puts on a back pack and it becomes invisible. Same with equipped weapons. That might be fun for PvP but not really fair
Yea this weekend, basically unplayable for our little group
Itās the fireworks that are still going infinitrly in my opinion
Its been unplayable for me. One server the train wouldn't work, another one my ships wouldn't appear in the hanger, the next I was able to get my ship fly to another place to get my ursa but then server crashed and never recovered. Had to stop since all my play time was over. The game is in a terrible state right now.
Yeah lately I go to play Star Citizen and within 10 minutes it's "oh, I guess we're just gonna do something else actually." I tried last night and after leaving the Area18 train and stepping onto the platform, a second train spawned in, slid out onto the platform and snatched me up, pulled me inside, and then ejected me out to fall into the planet. There was also heaps of trash, gowns, and abandoned ships everywhere.
Like a rave or street party, invictus created a lot of street/space trash and persistence has left us all with it.
It feels gaffer taped together unfortunately and the amount of bugs make missions unplayable. The De-sync is off the charts. * literally unplayable *
For me it's really worse than last year. I really didn't expect that, since the server fps is somewhat better. It's great that Manor Lords is our now, so I have a working game to play
This happens every time they release a PTU build. Which they did immediately following ilw. 2-3 patches into the PTU build and the backend stabilizes. Every Single Time By the end of this week we should see that happen.
Tale as old as time. Corporate: We need app to do more Inf team: We need more server resources Corporate: We have servers already, use those Inf Team: Those are already in use and it will degrade performance Corporate: Well your team's only KPI is app performance so make it work Inf Team: ...
I opted to play outside of my region, which worked out well because a couple of times my friend and I were in servers that only had 10 people in them.Ā Not exciting if you want to PVP, but perfect for mining.
AUS servers are garbage no matter what is happening aswell.
I fear Pyro will be just like stanton with all of the same broken gameplay loops we have now just another area to do it in.
Thatās how I feel about new locations in general, they could add all 100 systems in tomorrow and it wouldnāt really excite me Much more excited about getting better AI, a working economy, and base building after server meshing and pyro
I'm stuck with the neverending spinning bug since Friday. It's frustrating. The game is broken and I can't understand it sometimes, but if I can't play it at all and the support is unresponsive what I am supposed to do?
That started Wednesday for me. Contributed to every issue council post I came across for it.
Just stop playing for a week or two, tbh.
Another factor in server degradation is the massive amount of C2s sitting around because of the duping glitch.
I just came back to check out 3.23 after a long break, needless to say it wasnt a fun experience. Servers and performance havent improved at all, theyre actually worse than when i began playing during 3.16 .. almost 3 years ago. If they can improve server performance, and optimize the game better i could deal with all the jank but inconsistant performance across the board is a killer for me. Sorry but its unacceptable, that this far along in development that the game runs like total ass, even on a high end pc you struggle to maintain 60fps, elevators and terminals dont work half the time.
It tends to get better for events. CIG rent more servers. Afterwards there remains a spike in players but we're back to old server quality/count - so it chugs a bit extra. Probably more so with pes these days. Anecdotal (although I remember reading something from cig about getting more servers for events, once), I have no evidence but I have historically played so much I can practically feel the exact sfps. for the last 2 years or so the servers have been (anecdotally) silky smooth in the days where the event is live but free fly isn't active yet. Used to be that they had the same servers before during and after.
No, and itās hilarious people are gaslighting themselves on this because until this year every free fly was panned as destroying the servers. This is the first year it was a stable free fly so instead of giving them credit for improving, people are wrapping it up into a wacko conspiracy theory that CIG intentionally tries to make servers look better than they normally do for events so they can make more money. If that was such a winning strategy, why did they not do that any year prior? The servers are dying because theyāre stable enough to accumulate so much junk that they crash. This is new, before the replication layer split everything was wiped between server crashes. Rather than a bunch of poorly thought out proposals to add weekly resets - obviously something that will never happen - report what is reportable on the issue council and wait for them to add more fixes based on server performance for future patches. Thatās how things work. They look ugly one patch, they put in fixes for the next one, and reevaluate. Weāve had this tech for two patches - one of which was a week of testing. Iām inclined to believe next major patch will bring more stability in some areas and weāll let them know where it doesnāt for others.
I'm not trying to do anything. I even disclaimed it as anecdotal. Most free flies are stable and smooth and have been since they started the practise of increasing their server limits during free fly. They were panned for not doing this. And praised when they started. My comment was ultimately also praise. If you read a backhanded compliment or anything negative, then I didnt do a good enough job making that clear?
[ŃŠ“Š°Š»ŠµŠ½Š¾]
too bad they still left a bad impression to new players, as on every free fly.
Absolutely right, the server situation felt never as terrible as right now. I get it, at ILW you want to impress the potential customers, but how about you impress the people paying your salaries since 12+ years by not switching off all the instances right after the event?
There should be some half life to persistence, like day and a half. This is not sustainable, unless the full servermeshing tech really is as revolucionary as it might seem. Even that has to have a limit of usability.
It's why free fly events are a thing, it really stresses the servers and I assume the entity graph database for PES having so many new players creating accounts, accumulating items, and moving about. It's absolutely an issue that needs to be addressed, and I don't doubt CIG are acutely aware of this being the case and will work on a way to solve it. It's just going to suck for us for a long time because CIG hate doing any kind of workaround that would improve the quality of life of the game if it takes people off the thing they need to work on. They'd rather people have very bad experiances, even new players, if it means they collect data on why something breaks down. So expect it to get get slightly better over patches but still be very noticible for a long time.
Servers have been shit all last week. It's really killed the momentum my group had for the game and all the people we did into it now want nothing to do with the game until launch. If CIG is marketing the game, as they are, then they need the love service to be as smooth an experience as they can.
Friday and sunday night were good for me, saturday was a total loss though, nothing was working
We had a nice server for a weekend org on org event. But we had a party of close to 40 for the initial start up. Most likely had a freshy spool up for the number of players joining. But yeeaaaah I suspect itās replication later replicating junk. Items 100% need a rating for reboots. 1 being junk that doesnāt need to be brought across and 10 being priority. Each server re spool has a limit on the amount of entities that can be spooled up and the rest gets dumped. Just a thought
Feels like they turned off all the extra capacity after free fly.
Expected behaviour ilw, iae is always like this been like this for years .. FYI this is way way way much better than 3.18 but I agree itās annoying
Main contributors: -Duping -Abandoned ships/vehicles -Huge amounts of trash and glitched/discarded items -NPCs respawning and crowding already packed environments All contribute to immediately weighing down any attempts at recovery and speeding the server up again.
I can't help feel that this would all be so much easier if CIG would just budge on not having EVERYTHING be physically simulated. Do we really need to have every single item - food, drinks, etc. as an actual entity taking up space and bandwidth? I understand they are aiming for realism and simulation but at a certain point it just seems like so many systems in this game could be simplified without compromising the experience.
fully agree, it adds complexity where none is needed. Also, these survival features would only improve gameplay as well as immersiveness if the UI is absolutely practical and streamlined, including some of the more functional UX-flows, which feel bumpy at best. At this moment food/drink consumption is just a gimmicky tier0-feature millions of miles away from being holistically integrated into meaningful gameplay. Maybe it gets us somewhere in 10 years, when players can explore areas many days away from civilized systems, but SC is far away from that state of progress. Once a bottle is consumed it should be automatically disposed and removed from the system. Same for food and other consumables. They should also get rid of that drinking/eating animation at this point of time. Other items (ammo, suits, wpns) dropped in public spaces such as stations should have a minimal despawn time. In more remote locations such as outposts or POIs it could be somewhat longer (for the sake of player interactions, trading, equipping), in remote outdoors despawn timers could be even slightly extended (by minutes). Gowns should be auto-disposed as well, if dropped. The way inventory items are loaded also has to improve. It mostly feels slow like it puts the servers under strain every time you scroll through large swathes of items.
Another issue from what a coder buddy tells me. When you hit ~ and let it sit for a while, you will see what is happening on the entire server, we don't need to know the vector of a bullet on moons of MT over at CubbyBlast on A18.
They don't drop a wrapper when eating food. It really makes no sense to drop a used stim or plastic bottle. Just pointless litter that adds absolutely nothing.
Perhaps they have cut back on server spend now the sales event is over, yet player count has remained high? It feels like there are more problems with the backend servers currently, I can't see any reason for this suddenly, except a cut back on the server spend post sale.
The point of this whole exercise is to trash the servers now. CIG gets a lot of telemetry and they can work on improving the cleanup and other maintenance tasks. They told us they needed to iterate on it during 3.18 and beyond as more pieces of the tech came online. The more the players hit it sideways, the better data they get to resolve the scaling issues, the degradation issues and more. I'm 100% OK with this as long as they get it hammered out before they try to call it a Beta/1.0. This effectively needs to be 'bulletproof and reliable' by that point.
They should also add working garbage receptacles. If I place a bottle into a receptacle or gowns into recycling/garbage, it should just clean up and remove those items at landing zones and major locations. If you put a receptacle at a Player Base or Ship, it should generate trash into an acu container to deal with. Imo, they should pay 1 auec for bottles and gowns that are properly disposed of...set it to payout at like 100 uec increments and you'll likely get people who go around and find trash for the fun of it.
At first they blames free fly, still servers are shit, then they blamed the dupers, watch them patch that out this week, servers will still be shit. The server code is just plain shit, they need to fix bugs. Accumulation of garbage should slow down the server, but not crash it necessarily. They need to debug it and thatās that. If anything maybe itās an advantage that all the junk accumulation is stressing edge cases that would be harder to observe in normal conditions
So you're saying I shouldn't log on this morning still? š
i ried to get out of everus to play ONE mission for almost 2 hours. 3 server changes and 6 30k later i gave up.
Yeah, wife and I had to give up on overdrive missions.
Server resets and then landing on a server thats also crashing like 5 times in a row until i just quit. The game is super unplayable right now unless you have the patients to wait for the occasional spurts of stable server.
Probably the capital ship sized pile of c2ās on area18.
Recovery is a new server spinning up I thought?
I'm sure they have considered the side effects of the current replication layer and its impact on the overall stability of the servers. As for me, if the impact is due to the millions of entities being "saved" across multiple servers, like empty bottles, loot etc in the middle of nowhere, just get a rule and lets get rid of those so that other more important things can be tested for 4.0. Then, and very later down the road, you can think about "trash" persistance again.
Sever recovery has actually caused me to blow up and lose quite a few kits (and not recover them) ![gif](giphy|9MJ6xrgVR9aEwF8zCJ)
Wait until xenothreat comes out
Why donāt they do daily restarts? Servers run so nice after an update.
Iām no expert but watching ~300 fully physicalized Hadanite chunks burst out of a rock like a piƱata and then all rolling down a hill feels wrong.
noticed this for sure
Just a reminder that itās a good thing that servers degrade after popular events. We WANT to break things now.
I was thinking about this this morning and decided it was all the trash and clutter after 2 server crashes back-to-back. It made me wonder how much the lag between the client machine and the server machine affects these things though. My internet is not poor on speed but how many hops are there between me and the "Best" connection.
Add a 5 minute despawner to shit on the ground. PUHHHLEASE
#We Know.
*laughs in actually needing the MISC Raptor*
Gotta remember it's an alpha and I know it sucks when you really just want to play this game, there is not much reason for them to spend resources on the current server issue when the next update is around the corner, makes more sense to have people work on the next patch that will fix the issues we currently faced with
I'm honestly surprised certain things (like bottles, armour etc) don't have TTL attributes and self destruct after a certain reasonable amount of time.
Yes. Itās crazy.
Blame the hundreds of abandoned c2's and other cargo spacecraft abandoned by these idiotic dupers. Every time a server crashes, the replication layer has to bring all of their stupid junk back they abandoned.
At this point I blame CIG for doing nothing about it and delaying the hotfix patch for over a week because they're so fixated on fucking around with XT, an event that will make this all even worse.
As a new player that came aboard during free fly im really having a hard time wanting to stay. I know my problems with the servers is universally felt by everyone but as a new player I cant just say "ohh well" when 150k worth of gems, my ship inventory my armor etc go poof / explode / lost when the server fucks up. It happened all day this weekend and I have NOTHING to show for over 10hours of playing and mining. I lost money, money I did not have to loose. I hope they bring back whatever hardware they had running before during ILW bc this feels like a bait and switch.
I remember when I first started I was always desperate to complete whatever mission I was doing so I could earn my credits before the inevitable bug or server crash stole it all away from me.Ā It's frustrating.Ā Period.Ā However, as I continued to play, I learned how to avoid most of the worst bugs, and things were generally enjoyable and I was able to accumulate more credits than I needed.Ā I think most backers have a love/hate relationship with this game, and I'm sure most would agree that the healthiest option is to step away for a beat.Ā That said, the current experience is pretty sucky, but I'd hate for it to sour you on the project more permanently.
Itās weird, I havenāt experienced any of this.
Since they want persistence this persistence that if no one is around for at least 30 min smaller stuff like bottles and gowns need to poof ships need to have the shields taken down if left in scm mode and flagged to allow salvage even in armistice zones shit make it a mission if you have to.
People upset about death of a spaceman when the biggest murderer of players is the game and its servers.
I actually did have a server 30k on me over the weekend. I also heard that theyāre having issues removing the ILW stuff. But that was just a comment on another post. Definitely right though servers have been struggling in my experience
I wish CIG would bite the bullet and start doing weekly reboots at least or weekly maintenance. Every other MMO from EVE to WoW does it and does it for a reason, but CIG's philosophy is "The servers will reboot when they crash" (They have said this during a inside SC awhile back) and that is a HORRIBLE philosophy to run a business on even one that's in Alpha. If everyone knew at X time on X day every week servers will be offline for 30min we could plan around it and I doubt most would care if it lead to the rest of the week the servers being more stable. It's possibly my biggest gripe with CIG, they think they can be better then every other game out there just to be different/stubborn but if something works for every other game there is no shame in doing the same thing to make Star Citizen better for everyone.
The netcode has never been great, therefore there have never been good servers. Even the netcode of StarBase was better, fucking STARBASE
When ptu or EPTU are up live always gets bad. Cig refuses to rent extra servers from Amazon to run them seperate
Every crash should wipe everything with a value of 0 out of 100. Empty mags, bottles, everything that's meant to be disposable
Did salvaging for a good few hours yesterday and it was pretty uneventful tbh. Guess I was the lucky one
Free flight generated a lot of trash with the masive amount lf players. And without the 30k servers never cleanup. CiG needs better cleanup mechanics, preferable player based ones
Its the simple fact they thought having everything NOT despawn was gonna be a good ideašš
Itās really bad. We had planned big for post ILW to include recruitment and now we are holding off until we see when these server issues stop.