Hi,
We’re using a recastgraph on terrain, but edges where the terrain is too steep are very jagged (see images below), leading to weird behaviour when trying to walk there
Digging through the code, the resolution of the recast mesh for the terrain is directly influenced by the heightmap resolution, turning it up helps a little, but is still not really satisfactory
with 513:
Is there any way to get a smooth edge?
Setting the max edge error higher with the higher heightmap resolution helps with the worst “spikes”, but it’s still not really good and reduces detail in other areas where we need it
I did try briefly messing about with the terrain rasterization to make a fake high res mesh aswell, but that just broke the navmesh generation completely (good chances I messed something up) and seems more like a bandaid than a proper solution aswell.
The solution we we with for the time being is having one of our team members drawing a custom mesh collider around the terrain edge where we don’t want people to walk - this is of course error prone, since it is manual, but does give us nicely aligned edges (same as the fences in the screen shots)
I’d still be happy if anyone could give some feedback on if this is just how the system works or if there’s ideas/plans on improving it
The mesh the recast graph uses is subsampled from the actual heightmap. You can set the Terrain Sample Size field in the recast graph settings to a lower value which might improve things.
You may also want to increase the ‘max edge error’ field to allow it to simplify the border more (but make sure it is still acceptable in other areas of your world).
We’re already using a terrain sample size of 1 I’m afraid, that’s why I briefly tried to hack the terrain sampling code to generate a higher detail (smoothed) mesh which I gave up on though after only getting broken results
Since the nav mesh is prebaked we’re not too concerned about bake times/memory usage during baking for a high quality mesh anyways and it seems like that’d be the only downside of having an artifically high quality height mesh for the terrain
