Additionally, I read another link and tried checking the option AIPath.constrainInsideGraph as true, which can constrain the movement within the mesh.
https://forum.arongranberg.com/t/how-do-you-push-a-character-in-a-direction-but-stay-in-the-navmesh/11863/4
It works very well on a flat surface, but it doesn’t seem to work for different heights in 3D. For example, at the same height level, it’s impassable on the first floor but passable on the second floor, which results in characters on the first floor being incorrectly allowed to reach, as shown in the image below: