Units cannot travel to path.destination

When I first started my project I would set path.destination = rallyPoint (assigned by touching screen) and units would go almost to the destination then walk in place and the transform position in the inspector would show numbers constantly changing and just a few decimal points different than rallyPoint. I fixed that by implementing:
if (distanceToRallyPoint <= 1f)
{
transform.position = rallyPoint;
}
That made it work 95% of the time but my soldiers still get stuck walking in place sometimes.

I think I even know the cause but am not sure how to fix it. My Recast Graph makes triangles that closely match the rolling hills of my terrain, but the Recast graph is slightly off from the terrain in some spots. I made a stay above ground function that calculates the height of the terrain and keeps the character exactly there. The heigh of terrain and graph might not be exactly the same. I need the stay above terrain function because my soldiers will walk through hills without it.

Does anyone have a game where path.destination works every time and how did you set that up?

Hi

Which movement script are you using?
Handling undulating terrains is actually surprisingly tricky when combined with all other movement features.
The FollowerEntity script has the best support for this.

I attached the AIPath and the Raycast Modifier. What file is the FollowerEntity script saved in? Can it work stand alone or does it need another script? Should I remove AIPath and the Raycast Modifier?

The FollowerEntity script is in a file called FollowerEntity.cs.
It works alone, without the AIPath/Raycast modifier/Seeker.
Take a look at the example scenes. Most of them use the FollowerEntity.