RVO vs. Rigidbodies?

I’ve got an RTS-type game where dozens of units move around a map and hit each other. The problem I’m running into now is that they all just walk on top of each other. I’m using the free version of A* and plan to upgrade to the paid plan if this project gets more serious, but I wanted to understand more about RVO and if a similar goal could be achieved through other means.

Right now I’ve got a Rigidbody on the parent level of all my units marked with IsKinematic. If I toggle that off, the units just push each other around the map but if I toggle it on they all walk through each other. Is there some other setting I’m missing or is RVO the solution to all my problems?