So you’re method doesn’t work for my scenario but you got me on the right path. In my case, my planets have mountains, ocean floors, and then clamped areas in the middle. The clamped areas are the ‘safe zones’ where the pathfinding/player can move. Since I happen to know these vertices and my navmesh is 1:1 with the model anyway I can simply iterate through these and then set the matching node position to Walkable since I can be sure the Vector3s are exactly the same. So far this appears to mostly work. Comparing against the distance of the ocean inwards to the center will only work for areas under the seabed.
for (int i = 0; i < verts.Count; i++)
{
NNInfoInternal nnii = graph.GetNearest(verts[i]);
if ((Vector3)nnii.node.position != verts[i])
nnii.node.Walkable = false;
}