Multiple Agents with Layer Masks

I’m trying to implement a system similar to the one described in the Multiple Agent Types tutorial. I have two recast graphs, call them A and B. The only differences between them are that B has a larger Character Radius and an additional layer selected in the Layer Mask field (call it “Field”). Lastly, I have a Seeker with Traversable Graph set to Graph A.

The problem is that this Seeker is instead adopting all the properties of Graph B. My scene is set up with the Seekers placed inside of a force field, on the “Field” layer. When I generate the nav meshes, I get an unwalkable buffer area around both the inside and outside of the field. I expected this area to be unwalkable for B, but not for A. When I set a destination outside of the field, Seeker A moves to the edge of the buffer and stops there, even though the path-to-destination line continues through it. If I remove the Field layer from B’s Layer Mask (so now both A and B are ignoring the Field layer), the buffer vanishes and Seeker A can walk through the field as intended. However, Seeker A still can only get as close to other obstacles as B’s radius, not its own, smaller one. Adding a Graph C with an even larger radius confirms my findings, as now Seeker A can’t get any closer to obstacles than C’s radius.

TLDR: it seems that setting a Seeker’s Traversable Graph has no effect, as it’s always restricted to the most restrictive one. I thought I was doing something wrong with the layer masking, but even by disabling that and just using different radiuses (as the tutorial shows) causes the same problem.


How are you requesting the path? Are you doing it using some custom code or are you using one of the included movement scripts and setting the ai.destination property?

My script extends AIPath and sets GetComponent<AIPath>().destination to a Vector3. My AI also have RVO Controllers attached, if that makes a difference.

Actually, there’s one custom thing I’m doing which I’m just now wondering if it has an effect. My map is generated at runtime with a variable size out of a bunch of square tile prefabs. Each of these prefabs includes one or more of the AI. As soon as I’m done generating the map, I use this bit of code:

foreach (NavGraph graph in
            ((RecastGraph) graph).SnapForceBoundsToScene();
        foreach (Progress progress in
            Debug.Log("Scanning... " + progress.description + " - " + (progress.progress * 100).ToString("0") + "%");
            yield return null;

        // Tell the loading screen that the map and NavMesh are ready

Which successfully generates both the A and B nav meshes. But what I noticed was that the AI was going crazy, because since it was instantiated with the tile prefabs, it existed before the nav mesh was generated. To solve this problem, I start with my AI script disabled, and only enable it when I get the trigger message from the nav mesh finishing.

Could having the AI component disabled at start and then turning it on later be affecting it weirdly? Should I also wait to activate the RVO Controller and the Seeker (it has no checkbox in the inspector, can it still be toggled in code?), or just keep all the AI stuff on a different sub-game object that I can toggle separately?


I don’t think that should cause any issues.
However note that if you are loading/saving graphs during runtime, the graph indices might change and thus the graph mask might mean something else (the graph mask just indicates which graph indices are traversable). Check the seeker’s inspector during runtime to see if it still refers to the correct graph.

No luck. The Seeker’s Traversable Graphs field has the correct graph listed both before and after the graph recalculation. I also tried setting it in code using the GraphMask.FromGraphName() method, after the graph recalculation completes. And you were right - having the AIPath component set on at the beginning makes no difference either.

The next thing I tested was using the GraphMask.FromGraphName() method to set the Seeker to purposely follow the wrong nav mesh. When I try to lure my AI out of the Force Field in that case, the path-to-destination gizmo actually stops at the edge of the Force Field, and when the AI reaches the edge, it registers as reaching the destination and calculates a new path. To me, this proves that setting the masking is working properly. At least when the Seeker was following Graph A, it would draw the gizmo to the correct destination, the AI just couldn’t physically get there. And I still think the rest of my setup is good too, because everything behaves correctly when recast graph A is the only one in play.

Any other ideas?

Weird… I’m not sure what could be happening. I have tried to replicate it, but it seems to work fine for me.

One thing. Note that the graph mask only limits the graphs that the agent considers when finding the closest nodes to the start and end point. If there is say a path from a valid graph to a valid graph that passes through an invalid graph (say using an off-mesh link) then the other graph can be traversed as well.

I created a test scene with as much simplification as possible, and I’m still observing the problem. Maybe there’s an interaction happening that I didn’t think was worth noting before? (Note that I’ve switched to using the stuff’s actual names so it’s easier to talk about.)

-Ground game object. Tagged “Ground”, Default layer. Just a box collider serving as the ground of the scene
-Various “small” environment obstacles. All untagged. “SmallObstacle” layer. Models with capsule or spherical colliders.
-Various “tall” environment obstacles. All untagged. A parent game object with model/collider on the “SmallObstacle” layer and a child game object with model/collider on the “TallObstacle” layer. Think of a tree, where I want the ground AI to avoid the trunk and the air AI to avoid the entire radius of the leaves. The TallObstacle collider reaches all the way to the ground to intersect the navmesh.
-AStar object. RVO Simulator, RVO Navmesh, AStar Path scripts. Two Recast graphs on the AStar Path, one named “Cow Graph” and one named “UFO Graph”. Both set to Rasterize Colliders. Cow Graph has a Character Radius = 1 and Layer Mask only “Default” and “SmallObstacle”. UFO Graph has a Character Radius = 2 and Layer Mask only “Default”, “TallObstacle” and “ForceField”. (ForceField tag is not used in this test scene)
-Ground AI. Untagged, “Cow” layer. Capsule collider and rigidbody. RVO Controller and Funnel Modifier. Seeker with Traversable Graphs set to only “Cow Graph”. SpaceCowController script which extends AIPath. Radius is set to 1.3. (This is so the cows give each other a wider berth than how close they can get to obstacles.) My added code:

  • OnCollisionEnter with the “Ground” tag, call WanderWithinRadius() passing the cow’s position.

  • WanderWithinRadius picks a point inside the unit sphere, multiplies by a given radius, and adds to the cow’s current position. This is wrapped in a do-while loop that executes until the new point is confirmed to be on the recast graph. I set the .destination property to this point.

  • Additional added code involves trigger/collision interactions with objects that don’t exist in this test scene.

-AirAI - not implemented yet, but as you can imagine, its Seeker will follow the UFO Graph
-Physics layer collision matrix: “Cow” cannot collide with “TallObstacle” and “TallObstacle” cannot collide with “SmallObstacle”

In my main game, I programmatically generate the obstacles and rescan the graphs at runtime. Here, I have static scene set up and scan before I run the game. The cows start hovering; their rigidbodies make them fall to the ground, OnCollisionEnter, and start to wander. All the gizmos in the scene appear correct, cow’s Seeker is still set to Cow Graph, but then this happens:

It’s not the best picture, but the cow has picked a valid destination (assuming purple is “no access”, blue is both UFO and Cow Graph, and the in-between is Cow Graph only), only when it gets to the point where it is in the image, it stops moving. I checked the various physical colliders and none of them are touching (Cow should ignore TallObstacle anyway), and the image clearly shows that the center of the cow’s AI component is at the navmesh edge. It just can’t cross the border, even though everything is set correctly.

OK, I built a brand new project from the ground up, and I discovered that the component which broke the system was the RVO Navmesh. I suppose that makes sense; the documentation says that it “Adds a navmesh as RVO obstacles”, but it’s combining my multiple Recast graphs into a single navmesh, the most restrictive one.

When I disable the RVO Navmesh component, the different Seekers obey their layer masks, and I even noticed a general improvement in local avoidance between them. They do seem to cut corners now (Constrain Inside Graph is not an option for my situation), but I already had a generous agent radius so it’s not a big deal.