I haven't. That would work, but we have a lot of houses, and I'd rather avoid the clutter. I'd also have to teach the level designer to place out the box colliders, and I kind of want moving houses around be minimal work, so the process for the designer is as smooth as possible.
The houses all have an editor script already, so I could make that script create and then destroy the box colliders when the recast bake happens. Or the script could simply hook into the build process itself and do something like what the RecastMeshObject is doing.
I was looking for a way to do this that's a bit more lazy, since we're prototyping and I don't want to go too deep into coding up solutions. Having a large walkable height was exactly the lazy I need. Now my 2 meter tall npcs need 20 meters of space above them to walk, but it works for our use case.