I’ve got an RTS-type scenario where hundreds of objects are moving around a map and when they encounter each other they start attacking. I’m using RVO on all of these objects but when they’re trying to encircle an enemy they spend a LOT of time running into each other instead of going around each other.
In my head, once they begin attacking they essentially become an obstacle that other objects need to move around instead of trying to move through so I started using NavmeshCut but I’m wondering if this was a good idea or not. I assumed that using “Lock when not moving” on RVOController would do something similar but it doesn’t appear to be.
Every object has a disabled NavmeshCut on them and when they begin attacking it becomes enabled. When they’re done attacking and start pathing again it becomes disabled. It seems to be a bit of a performance hit, but has improved the running-into-each-other thing a bit! I’m looking for tips to see if this is the correct approach or not. Here’s a short video of the difference between using NavmeshCut and not.
However, I am randomly running into some errors and warnings when using this approach. RVOController is saying I’m hitting the “Limit of how many neighbors to consider”, the console is getting “PointOnEdgeException, perturbating vertices slightly”, and somewhat randomly I’m getting an error about GUI Tree (or something like that? 
Can’t seem to recreate it right now). This is what’s making me think there’s got to be another way.
Also: do the RichAI settings WallForce and WallDistance come into play when using the NavmeshCut approach? Here’s a screenshot of my settings for reference.
Update: wondering if trying to utilize something like “Stop when crowded” could be useful as a trigger to have units move to a random nearby position and then recalculate their path to the enemy’s opening. Also, changing the acceleration to 100 made everything much “snappier” 