[Solved] CPU utilization with 350 seekers

#1

Hi @aron_granberg, first of all I want to thank you for such an amazing job you have done here.

Now, this is my issue: I’ve managed to get around 350 seekers in standby mode (with everything enabled less the AILerp script) with a little under 6% CPU utilization for 2 days in a row BUT when I enabled the AILerp script it behaves exactly the same for around 40 minutes until CPU util. starts to go up and up until it hits 100%.

It took me around 2 weeks to find which script was failing (profiler didn’t help much), so after compiling multiple server builds with different scripts disabled, I narrowed it down to the AILerp script.

I wouldn’t mind if this operation costed me 30% of CPU utilization or even 70% but I would like it to be stable.

At this point I just dont know what else to do. I have attached my modified AILerp script + a pic from my amazon server utilization, hoping you can give me an Idea of whats happening here.

AILerp Mod
With and Without AILerp

Thanks.

#2

Hi

Are you disabling and enabling objects with the AILerp component a lot?
There is currently a bug that has gone unnoticed for some time. Currently when the AILerp component is enabled it will start a coroutine to search for paths periodically, however when it is disabled that coroutine is not stopped. I thought originally that coroutines were automatically stopped when the object is disabled, but that doesn’t seem to be the case.
This can after some time lead to an increase in memory usage and CPU time as more and more coroutines are started.

Try to add a call to StopAllCoroutines() in the OnDisable method of the AILerp script.
This is already fixed in the beta, but since you seem to have made some changes to the script I suppose it is easier for you to fix it in your version.

#3

Hi,

I am not disabling this script at all, when I need my seekers to go somewhere else, i call the function cleanStart() which basically does everything that OnDisable() does with the exception of the last two lines which cancels the callbacks.

I can still give StopAllCoroutines() a shot thou.

#4

Hm… Ok. Do you think you could monitor the memory usage? A high CPU utilization might just be a side effect of a high memory usage.

Also a few things just to make sure:

  • Try to enable the ASTAR_NO_POOLING option under the Optimizations tab. This will increase memory allocations, but if some code is using pooling incorrectly it will not affect anything.
  • Make sure you have logging disabled (A* Inspector -> Settings -> Logging/Path Log Mode). Logging a message every time a path is calculated shouldn’t be that bad, but maybe you are doing it so often that it over time makes the log file really large (might be relevant if your are using an amazon server with a small-ish amount of hard-drive space).
#5

It is worth mentioning the only two things I modified were:

  1. Changed the “target” variable from transform to Vector3 and I search for new paths every second if target changed.
  2. Added a cleanStart() function with the OnDisable() code in it in an attempt to solve the issue.

I hit the 65k paths searched warning after 15 mins or so (could.this be related to the issue?)

I have logs disabled, not even a single debug log, just the one above.

I will update to the beta version and get back to you with the memory readings I’m about 10 hours.

#6

I am still trying to fix this, haven’t got the time to look into it but yesterday I left the server running on Windows instead of Ubuntu and to my surprise it kept CPU util. under 8% winth 60Mb - 75Mb of memory used (goes up and then back down)

I dont understand how AILerp can only affect builds on Ubuntu/Linux.

Is this a known issue?

EDIT: same issue on Windows… it just takes longer, RAM util. is now 120Mb instead of 60Mb

#7

Done…

Adding StopAllCoroutines() seems to have fixed the issue.

Thank you @aron_granberg

done.PNG909×487 27 KB

#8

Nice!
I’m glad it solved the issue.

1 Like