Battle of Teams 2.1b Potential Memory Leak Problem

[Kers]

When I host this map for a long time, it takes more and more RAM. Eventually it will use 1g and crash the server. I wonder if anyone who is good at coding can fix this map? Thanks in advance.

[JakSparro98]

When I host this map for a long time, it takes more and more RAM. Eventually it will use 1g and crash the server. I wonder if anyone who is good at coding can fix this map? Thanks in advance.

Can you provide me the map in order to investigate on the cause?

[Kers]

http://sfdmaps.at.ua/load/maps/custom_m ... 8-1-0-1468

Thanks for your help.

[Kers]

Anyone can help?

[JakSparro98]

Anyone can help?

So far I didn't found something harmful in the code, for how many hours do the server load the map?

[Kers]

It starts with 110 mb ram and slowly increase to around 1gb ram which takes about 200-300 games. I tried crates, official maps, and melee island all of them only take 100 mb ram even after about 500 games.

I don't know the code but the creator of the map tell me that there is some bug in the code.

Now I can confirm the root of the issue is battle of teams or NF3 maps. Is there anyway to fix them?

I can say that BoT code is really bad, but I can only say it for the 3 first maps, the other ones weren't made by me...

BoT has lots of fails in performance and can create a lot of objects due to some bugs (and my stupidity), BUT as I already said, I'm talking about BoT 1, 2 and 3...

[Dumby Eggy]

200-300 GAMES?!! Too many games, like a 24/7 server. Do you normally run scripts when you host the BoT map ?

[JakSparro98]

It starts with 110 mb ram and slowly increase to around 1gb ram which takes about 200-300 games. I tried crates, official maps, and melee island all of them only take 100 mb ram even after about 500 games.

I don't know the code but the creator of the map tell me that there is some bug in the code.

Now I can confirm the root of the issue is battle of teams or NF3 maps. Is there anyway to fix them?

I can say that BoT code is really bad, but I can only say it for the 3 first maps, the other ones weren't made by me...

BoT has lots of fails in performance and can create a lot of objects due to some bugs (and my stupidity), BUT as I already said, I'm talking about BoT 1, 2 and 3...

I rewrote a part of the code, changed something and deleted another part that weren't never executed, I changed the version to 2.2 to distinguish both maps.

Download

If it doesn't solve your problem... well I think we have to take seriously the fact that maybe the script appdomain is unable to collect properly all the garbage, other headaches for Gurt.

[Gurt]

If it doesn't solve your problem... well I think we have to take seriously the fact that maybe the script appdomain is unable to collect properly all the garbage, other headaches for Gurt.

Most likely the case and something I will look deeper into in late beta of the game. It works good enough right now.

[Kers]

200-300 GAMES?!! Too many games, like a 24/7 server. Do you normally run scripts when you host the BoT map ?

Yes it is a 24/7 server. I don't run any script with that map though.

[Kers]

It starts with 110 mb ram and slowly increase to around 1gb ram which takes about 200-300 games. I tried crates, official maps, and melee island all of them only take 100 mb ram even after about 500 games.

I don't know the code but the creator of the map tell me that there is some bug in the code.

I can say that BoT code is really bad, but I can only say it for the 3 first maps, the other ones weren't made by me...

BoT has lots of fails in performance and can create a lot of objects due to some bugs (and my stupidity), BUT as I already said, I'm talking about BoT 1, 2 and 3...

I rewrote a part of the code, changed something and deleted another part that weren't never executed, I changed the version to 2.2 to distinguish both maps.

Download

If it doesn't solve your problem... well I think we have to take seriously the fact that maybe the script appdomain is unable to collect properly all the garbage, other headaches for Gurt.

Thanks a lot. Gonna try this tonight.

[JakSparro98]

Thanks a lot. Gonna try this tonight.

Sorry but due for the 1.3.0 update the map camera was a mess, I updated it again, download the map from here.

[Kers]

Thanks a lot. Gonna try this tonight.

Sorry but due for the 1.3.0 update the map camera was a mess, I updated it again, download the map from here.

I tried your 2.2 version and after 248 games it takes up 485mb ram so the problem may still exists. Also there is error message about the crate in the middle of the map but I don't know how to fix it. I'll keep running the server to see if it will crash or not. So far no crash yet. Thanks again. Will try the new version later.

[JakSparro98]

Thanks a lot. Gonna try this tonight.

Sorry but due for the 1.3.0 update the map camera was a mess, I updated it again, download the map from here.

I tried your 2.2 version and after 248 games it takes up 485mb ram so the problem may still exists. Also there is error message about the crate in the middle of the map but I don't know how to fix it. I'll keep running the server to see if it will crash or not. So far no crash yet. Thanks again. Will try the new version later.

Yeah I also fixed the crate now we only have to wait to find out if the memory leak is solved or not, if not... well, as I already posted, it could not depend of the script code.

[Kers]

Sorry but due for the 1.3.0 update the map camera was a mess, I updated it again, download the map from here.

I tried your 2.2 version and after 248 games it takes up 485mb ram so the problem may still exists. Also there is error message about the crate in the middle of the map but I don't know how to fix it. I'll keep running the server to see if it will crash or not. So far no crash yet. Thanks again. Will try the new version later.

Yeah I also fixed the crate now we only have to wait to find out if the memory leak is solved or not, if not... well, as I already posted, it could not depend of the script code.

Well after 531 games the 2.2 version takes 850mb ram while crates/melee island 1.7 after 1700 games takes only 68mb ram. But at least the battle of teams server no longer crash as often as before. I'm going to host your updated one now see if that makes a difference.

[Kers]

The bug still exists in your updated version so I guess it's not the script's problem. Gonna manually restart the server every day then. BTW stop server running and start it again doesn't release the ram. I must close the dedicated server window and reopen a new one to fix that.

[Kers]

Now I am hosting nuj's tennis map. It has the same memory leak problem. I checked the f11 log and there is constantly "can't find effect hit". Could that be the reason why the servers eventually crash?

[JakSparro98]

Now I am hosting nuj's tennis map. It has the same memory leak problem. I checked the f11 log and there is constantly "can't find effect hit". Could that be the reason why the servers eventually crash?

I think not, it happens every time the game calls sounds and effects not implemented yet (or not found due for messing up with the game data by the user).

[Kers]

So after about 8 hours of game, the tennis map takes 1.5GB memory and causes horrible lag. Time to upgrade to my server I guess.

After some testing, the cause of rapid increasing memory consumption in the tennis server is caused by throwing the baseball bat at the ground, The bat is destroyed upon touching the ground. The memory consumption will increase at a rate of 0.1MB/s. During the testing, when the memory usage reached 140MB it got reduced to 115Mb so i guess the garbage collector worked but then the next time it hit 140MB or higher nothing happened. I don't have time to do testing on the battle of teams yet since it is much more complex. My guess now is the destroy process of the corpse causes the problem.

[JakSparro98]

So after about 8 hours of game, the tennis map takes 1.5GB memory and causes horrible lag. Time to upgrade to my server I guess.

After some testing, the cause of rapid increasing memory consumption in the tennis server is caused by throwing the baseball bat at the ground, The bat is destroyed upon touching the ground. The memory consumption will increase at a rate of 0.1MB/s. During the testing, when the memory usage reached 140MB it got reduced to 115Mb so i guess the garbage collector worked but then the next time it hit 140MB or higher nothing happened. I don't have time to do testing on the battle of teams yet since it is much more complex. My guess now is the destroy process of the corpse causes the problem.

I don't' want to be bringer of bad news but this seems not to be a script problem, it's instead a problem in the script engine. There is no way to solve this by ourselves because (and also Gurt said that) it's a likely an issue in the program code.

Furthermore since the C# implements an automatic memory handling, even if we are sure the GC fails cleaning the references, we cannot manually alloc and dealloc objects.
I think we can opt for another solution, maybe even simplier for the developers to implement, the possibility to auto restart a server after some time or after a number of rounds, as a temporary solution and also as a standard way to preserve the server from unknown bugs caused by long term execution.