Hi
You can subclass the PointGraph and override the IsValidConnection method. That will allow you to control which connections are added and which ones are not.
See also this thread which has a few more details about a similar case Tagging procedural 2D tiles
You can also construct a completely custom graph. Here is a tutorial for that: https://arongranberg.com/astar/docs/writing-graph-generators.php