Sorry for the late answer.
I think your agents have a layer mask used for ground detection that includes the layer that the agents are on themselves. It should not include the layer of the agents since then they may detect other agents as being part of the ground.