Recast graphs will not ignore colliders excluded in the layer mask as well as colliders with “Is Trigger” enabled if they are attached to a Rigidbody2D that has another attached collider that is not ignored during graph generation.
How to Recreate
Create game objects with the following hierarchy:
GameObject (layer: Default): Rigidbody2D, CircleCollider2D with radius 0.1.
---- GameObject (layer: Water): CircleCollider2D with radius 10.
GameObject (layer: Default): Rigidbody2D, CircleCollider2D with radius 0.1.
---- GameObject (layer: Default): CircleCollider2D with “Is Trigger” enabled and radius 10.
Scan a recast graph using a layer mask that excludes all layers except “Default” and you’ll find that the resulting mesh has used both the excluded layer and trigger when generating the graph.
Specifics
In ColliderMeshBuilder2D.GenerateMeshesFromColliders(...) collider shapes are collected using Rigidbody2D.GetShapes(...), but this will cause collider shapes for ignored layers and triggers to be processed.
In my project it was easy enough to fix this by tweaking the code to always get the shapes from the collider (as it already does when there isn’t an attached rigidbody). Notably I don’t get any of the annoying Unity error logging that is mentioned in the comments of that code, but I haven’t tested this much yet. Figured it’s worth mentioning just in case.
Hope your eye operation went well, cheers.