Follower entity with local avoidance causes agent to move in circles

Hello,
I am using follower entity , it works fine by itself but when I try to add to local avoidance the agents starts to go to the target in spirals.

check the video : https://youtu.be/cQM40p9IaYg

I am using a destination property to make the agent move:

        this.followerEntity.updateRotation = false;
        this.followerEntity.SetDestination(location);