Problems with AI Navigating corners when using tagging

I have a top down game where some enemies in my game can climb over tables and waist high obstacles, and other can’t. I’m currently using a modified version of GraphUpdateScene.cs to tag the tables on startup so that certain enemies won’t traverse them. Unfortunately, this would cause them to get stuck on corners really badly and also move extremely slowly when moving alongside tables. To rectify this, I made an update to GraphUpdateScene.cs that expands the bounding box used in the calculation of what nodes to tag. This has helped a lot, but I’ve pushed this as far as I can go.

If you see the screenshot, if I go any further with the size of the tagged area (the blue squares), the creature wont be able to navigate into the middle of the central area because the tagged nodes will close the gap. Can you recommend any other approach to handle this situation of having certain enemies not use certain parts of the navmesh whilst still not having them get stuck on corners? I don’t really want to have 2 separate grids as it’s harder to maintain.

Hi

If they get stuck on corners, I would guess that they follow the path too losely. Have you tried reducing the Pick Next Waypoint Distance on the AIPath script?

Yeah I tested Pick Next Waypoint Distance with different values and settled at 0.7. If I go too long then the movement looks very wooden and adjusting it doesn’t help the problem very much anyway.

Another further issue is if I increase the size of the bounding box more to get the “dont go here” tagged area larger, then the player can just move into that area fully and the AI won’t path to them, which is even worse.

Basically I’m looking for a way to keep the AI further from the non pathable areas unless they really need to go near them to get through a small gap.

Hi

Looking at your original graph, the grid graph’s resolution might be raised to allow for a more consistent tag margin. If you look at the bottom of that building vs the top, the margin which is tagged is much greater at the top. Increasing the grid resolution should help with this.

Thanks for the response. I’ve bumped up the navmesh resolution and it helps definitely, but there’s still a couple of problems.
First image shows the navmesh upped:


Second image shows the tags. I’ve draw a red arrow to show a situation where the gap is big enough for an enemy to get through (i.e. it should fit through based on it’s collision size) but the enemy won’t navigate through that gap because the tagged area completely covers it and closes the gap.

But the tagged area needs to be that big to avoid the enemy’s getting stuck on corners, as explained before.
Second issues is that the sheer amount of space I need to have around the obstacles and walls means that the player can move into these ‘no go zones’ i.e. places off the navgrid, and then the creatures can’t path to them at all.