Unfortunately, not very much can be done about this with the current system. The edges of triangles can be limited by Max Edge Length in the graph settings, but since vertices should not occur in the middle of the mesh (or at least that is quite bad usually), they cannot be limited very much for large open areas.
That it generates incorrect paths in some cases is a symptom of having small, thin triangles next to large ones, it is one of the main drawbacks of using navmesh based graphs. Currently, not much can be done about that either. Most approaches basically boils down to trying a lot of alternative paths and choosing the best one (something which you might be able to do using AlternativePath modifier and some tricks).
A grid graph might be a better option in this case.
For the random path, what you could do instead is simply to choose a random point and path to that, if your world has many large empty areas, that would work well I think.
I could take it as a feature request to make the RandomPath choose a random point inside a node as target.