A voxel rasterization grid of 10000*10000 is huge, which is why you are getting the OOM exception. You could try downloading the latest version which has support for tiled recast graphs, they were especially made to solve the issue of large worlds. With a tiled recast graph, only one small tile will be calculated at once instead of the whole world at once, this reduces the memory usage A LOT.