Page 1 of 2

Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Sat Apr 22, 2017 11:46 pm
by Kers
When I host this map for a long time, it takes more and more RAM. Eventually it will use 1g and crash the server. I wonder if anyone who is good at coding can fix this map? Thanks in advance.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Sun Apr 23, 2017 12:43 am
by JakSparro98
Kers wrote:When I host this map for a long time, it takes more and more RAM. Eventually it will use 1g and crash the server. I wonder if anyone who is good at coding can fix this map? Thanks in advance.
Can you provide me the map in order to investigate on the cause?

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Mon Apr 24, 2017 5:13 am
by Kers

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Mon May 01, 2017 11:26 pm
by Kers
Anyone can help?

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Mon May 01, 2017 11:32 pm
by JakSparro98
Kers wrote:Anyone can help?
So far I didn't found something harmful in the code, for how many hours do the server load the map?

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Thu May 04, 2017 1:33 am
by Kers
It starts with 110 mb ram and slowly increase to around 1gb ram which takes about 200-300 games. I tried crates, official maps, and melee island all of them only take 100 mb ram even after about 500 games.

I don't know the code but the creator of the map tell me that there is some bug in the code.
ShutDownMan wrote:
Kers wrote:Now I can confirm the root of the issue is battle of teams or NF3 maps. Is there anyway to fix them?
I can say that BoT code is really bad, but I can only say it for the 3 first maps, the other ones weren't made by me...

BoT has lots of fails in performance and can create a lot of objects due to some bugs (and my stupidity), BUT as I already said, I'm talking about BoT 1, 2 and 3...

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Thu May 04, 2017 5:56 am
by Dumby Eggy
200-300 GAMES?!! Too many games, like a 24/7 server. Do you normally run scripts when you host the BoT map ?

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Sat May 06, 2017 10:36 pm
by JakSparro98
Kers wrote:It starts with 110 mb ram and slowly increase to around 1gb ram which takes about 200-300 games. I tried crates, official maps, and melee island all of them only take 100 mb ram even after about 500 games.

I don't know the code but the creator of the map tell me that there is some bug in the code.
ShutDownMan wrote:
Kers wrote:Now I can confirm the root of the issue is battle of teams or NF3 maps. Is there anyway to fix them?
I can say that BoT code is really bad, but I can only say it for the 3 first maps, the other ones weren't made by me...

BoT has lots of fails in performance and can create a lot of objects due to some bugs (and my stupidity), BUT as I already said, I'm talking about BoT 1, 2 and 3...
I rewrote a part of the code, changed something and deleted another part that weren't never executed, I changed the version to 2.2 to distinguish both maps.

Download

If it doesn't solve your problem... well I think we have to take seriously the fact that maybe the script appdomain is unable to collect properly all the garbage, other headaches for Gurt.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Sat May 06, 2017 11:28 pm
by Gurt
JakSparro98 wrote:If it doesn't solve your problem... well I think we have to take seriously the fact that maybe the script appdomain is unable to collect properly all the garbage, other headaches for Gurt.
Most likely the case and something I will look deeper into in late beta of the game. It works good enough right now.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Sun May 07, 2017 11:23 pm
by Kers
Dumby Eggy wrote:
Thu May 04, 2017 5:56 am
200-300 GAMES?!! Too many games, like a 24/7 server. Do you normally run scripts when you host the BoT map ?
Yes it is a 24/7 server. I don't run any script with that map though.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Sun May 07, 2017 11:24 pm
by Kers
JakSparro98 wrote:
Sat May 06, 2017 10:36 pm
Kers wrote:It starts with 110 mb ram and slowly increase to around 1gb ram which takes about 200-300 games. I tried crates, official maps, and melee island all of them only take 100 mb ram even after about 500 games.

I don't know the code but the creator of the map tell me that there is some bug in the code.
ShutDownMan wrote:
I can say that BoT code is really bad, but I can only say it for the 3 first maps, the other ones weren't made by me...

BoT has lots of fails in performance and can create a lot of objects due to some bugs (and my stupidity), BUT as I already said, I'm talking about BoT 1, 2 and 3...
I rewrote a part of the code, changed something and deleted another part that weren't never executed, I changed the version to 2.2 to distinguish both maps.

Download

If it doesn't solve your problem... well I think we have to take seriously the fact that maybe the script appdomain is unable to collect properly all the garbage, other headaches for Gurt.
Thanks a lot. Gonna try this tonight.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Mon May 08, 2017 12:02 am
by JakSparro98
Kers wrote: Thanks a lot. Gonna try this tonight.
Sorry but due for the 1.3.0 update the map camera was a mess, I updated it again, download the map from here.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Tue May 09, 2017 12:45 am
by Kers
JakSparro98 wrote:
Mon May 08, 2017 12:02 am
Kers wrote: Thanks a lot. Gonna try this tonight.
Sorry but due for the 1.3.0 update the map camera was a mess, I updated it again, download the map from here.
I tried your 2.2 version and after 248 games it takes up 485mb ram so the problem may still exists. Also there is error message about the crate in the middle of the map but I don't know how to fix it. I'll keep running the server to see if it will crash or not. So far no crash yet. Thanks again. Will try the new version later.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Tue May 09, 2017 2:01 am
by JakSparro98
Kers wrote:
Tue May 09, 2017 12:45 am
JakSparro98 wrote:
Mon May 08, 2017 12:02 am
Kers wrote: Thanks a lot. Gonna try this tonight.
Sorry but due for the 1.3.0 update the map camera was a mess, I updated it again, download the map from here.
I tried your 2.2 version and after 248 games it takes up 485mb ram so the problem may still exists. Also there is error message about the crate in the middle of the map but I don't know how to fix it. I'll keep running the server to see if it will crash or not. So far no crash yet. Thanks again. Will try the new version later.
Yeah I also fixed the crate ;) now we only have to wait to find out if the memory leak is solved or not, if not... well, as I already posted, it could not depend of the script code.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Tue May 09, 2017 11:59 pm
by Kers
JakSparro98 wrote:
Tue May 09, 2017 2:01 am
Kers wrote:
Tue May 09, 2017 12:45 am
JakSparro98 wrote:
Mon May 08, 2017 12:02 am

Sorry but due for the 1.3.0 update the map camera was a mess, I updated it again, download the map from here.
I tried your 2.2 version and after 248 games it takes up 485mb ram so the problem may still exists. Also there is error message about the crate in the middle of the map but I don't know how to fix it. I'll keep running the server to see if it will crash or not. So far no crash yet. Thanks again. Will try the new version later.
Yeah I also fixed the crate ;) now we only have to wait to find out if the memory leak is solved or not, if not... well, as I already posted, it could not depend of the script code.
Well after 531 games the 2.2 version takes 850mb ram while crates/melee island 1.7 after 1700 games takes only 68mb ram. But at least the battle of teams server no longer crash as often as before. I'm going to host your updated one now see if that makes a difference.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Thu May 11, 2017 4:39 am
by Kers
The bug still exists in your updated version so I guess it's not the script's problem. Gonna manually restart the server every day then. BTW stop server running and start it again doesn't release the ram. I must close the dedicated server window and reopen a new one to fix that.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Wed Jun 14, 2017 10:24 am
by Kers
Now I am hosting nuj's tennis map. It has the same memory leak problem. I checked the f11 log and there is constantly "can't find effect hit". Could that be the reason why the servers eventually crash?

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Wed Jun 14, 2017 5:37 pm
by JakSparro98
Kers wrote:
Wed Jun 14, 2017 10:24 am
Now I am hosting nuj's tennis map. It has the same memory leak problem. I checked the f11 log and there is constantly "can't find effect hit". Could that be the reason why the servers eventually crash?
I think not, it happens every time the game calls sounds and effects not implemented yet (or not found due for messing up with the game data by the user).

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Thu Jun 15, 2017 1:32 am
by Kers
So after about 8 hours of game, the tennis map takes 1.5GB memory and causes horrible lag. Time to upgrade to my server I guess.

After some testing, the cause of rapid increasing memory consumption in the tennis server is caused by throwing the baseball bat at the ground, The bat is destroyed upon touching the ground. The memory consumption will increase at a rate of 0.1MB/s. During the testing, when the memory usage reached 140MB it got reduced to 115Mb so i guess the garbage collector worked but then the next time it hit 140MB or higher nothing happened. I don't have time to do testing on the battle of teams yet since it is much more complex. My guess now is the destroy process of the corpse causes the problem.

Re: Battle of Teams 2.1b Potential Memory Leak Problem

Posted: Thu Jun 15, 2017 2:23 pm
by JakSparro98
Kers wrote:
Thu Jun 15, 2017 1:32 am
So after about 8 hours of game, the tennis map takes 1.5GB memory and causes horrible lag. Time to upgrade to my server I guess.

After some testing, the cause of rapid increasing memory consumption in the tennis server is caused by throwing the baseball bat at the ground, The bat is destroyed upon touching the ground. The memory consumption will increase at a rate of 0.1MB/s. During the testing, when the memory usage reached 140MB it got reduced to 115Mb so i guess the garbage collector worked but then the next time it hit 140MB or higher nothing happened. I don't have time to do testing on the battle of teams yet since it is much more complex. My guess now is the destroy process of the corpse causes the problem.
I don't' want to be bringer of bad news but this seems not to be a script problem, it's instead a problem in the script engine. There is no way to solve this by ourselves because (and also Gurt said that) it's a likely an issue in the program code.

Furthermore since the C# implements an automatic memory handling, even if we are sure the GC fails cleaning the references, we cannot manually alloc and dealloc objects.
I think we can opt for another solution, maybe even simplier for the developers to implement, the possibility to auto restart a server after some time or after a number of rounds, as a temporary solution and also as a standard way to preserve the server from unknown bugs caused by long term execution.