Stack Overflow hard crash upon starting to use navmesh cuts, and editing Poly2Tri

General Recast graph
#1

Since implementing Navmesh Cuts in our game, we’ve started to see a hard crash from a stack overflow in Poly2Tri.FlipEdgeEvent/Poly2Tri.FlipScanEvent, similar to Update Navmesh Cut cause stack overflow

We are using a recast graph, so switching to DynamicGridObstacles is not an option for us for performance reasons. The navmesh cuts are also dynamically placed by the player, so we cannot guarantee that they don’t have any vertices that touch each other.

2 questions here:

  1. I’ve seen this is a known issue - has there been any progress on investigating this one?
  2. I’d like to try editing Poly2Tri myself to avoid this scenario. I see the Poly2Tri is an open source library, but the dependencies.txt in the navmesh cutting library says that the version included in A* Pathfinding has been modified to reduce size and improve platform compatibility. Given that the library seems to just include the DLL (unless I’m missing something), is there somewhere I could grab the modified Poly2Tri source so I could modify it further? Or is GitHub - HalfVoxel/poly2tri-cs: A port of poly2tri to C# already the modified version and I’m misunderstanding the note in dependencies.txt

We are using the pro version of this library 4.2.15

#2

Hi

I’m currently working on switching out Poly2Tri for another package (GitHub - andywiecko/BurstTriangulator: 2d Delaunay triangulation with mesh refinement for Unity with Burst compiler). I think making Poly2Tri robust enough to handle the edge cases is basically impossible. The author of Poly2Tri is very clear about not caring much about edge case like these.

I believe v5 of the package handles some of these edge cases better, however, so you could try that out.

#3

In v5, full graph updates are a lot faster too.

#4

Yeah my thought was modifying Poly2Tri to where if we get close to a stack overflow (via just counting how deep we’ve recursed), throw an exception, and then have A* Pathfinding catch that + perturbate + try again. If I modify the HalfVoxel version of the repo to try that, does it have all the modifications that you’ve done to the library for A* already?

#5

V5 does sound like it comes with a lot of improvements, but it’s outside of our timeline to upgrade it at this exact moment if it doesn’t 100% fix the issue (just with testing and stuff, etc., since we’re already a live game). Hopefully in the future we’ll find the time to switch over though, especially if you switch from Poly2Tri to this new burst library, I do like that you’re now using burst/ECS and that you’ve also added better 3D shapes for navmesh cuts (which I did on my own too separately not knowing it’d be coming soon in v5, which hadn’t released yet at the time, since I hadn’t been tracking the beta package versions lol).

#6

I think so? But tbh it was so long ago that I built the dll, so I cannot recall exactly how I did it.