Dont know why I didnt think of it previously. But I found a workaround by adding RecastNavmeshModifiers on select colliders, to force NavMesh seams. This seem to remove basically all issues mentioned above.
Example of the very first post without and with a modifier on the slope/stair walkable collider
It also fixed the mysterious random gap issues in inclines near max slope angle mentioned in this thread.