Pathfinding Crashing Editor and Standalone

Wrymn · June 8, 2024, 10:29am

We have critical issue where for some reason, pathfinding is crashing in rare cases editor, as well as built player (which is even worse). By crashing we mean hard crash of application, not just error.

The crashes happened both during gameplay, as well as when the game was just being loaded.

Unity: 2023.2.18
Platform: MacOS M3 Sonoma
AStar version: 5.1.1
AStar usage: RichAI, RVO, Recast Graph

I have checked the editor log file, and it points to pathfinding, as this is the last report before crash:

=================================================================
	Native Crash Reporting
=================================================================
Got a segv while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

=================================================================
	Native stacktrace:
=================================================================
	0x36cf419d4 - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_dump_native_crash_info
	0x36cf03de4 - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_handle_native_crash
	0x36ce8c628 - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_sigsegv_signal_handler_debug
	0x182ec9a24 - /usr/lib/system/libsystem_platform.dylib : _sigtramp
	0x4f38aa1f0 - Unknown
	0x4f38a920c - Unknown
	0x4f388db2c - Unknown
	0x4f388d618 - Unknown
	0x44e923778 - Unknown
	0x44e922fbc - Unknown
	0x5c6c35ed0 - Unknown
	0x5c6c34d14 - Unknown
	0x5c6c34384 - Unknown
	0x5c6c340f0 - Unknown
	0x5c6c33ea4 - Unknown
	0x568565470 - Unknown
	0x36ce8f8fc - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_jit_runtime_invoke
	0x36d014d30 - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : do_runtime_invoke
	0x36d0166e0 - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_runtime_delegate_try_invoke
	0x36d036d84 - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : start_wrapper_internal
	0x36d036b3c - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : start_wrapper
	0x36d0b6038 - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : GC_inner_start_routine
	0x36d0b5fc0 - /Applications/Unity/Hub/Editor/2023.2.18f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : GC_start_routine
	0x182e9a034 - /usr/lib/system/libsystem_pthread.dylib : _pthread_start
	0x182e94e3c - /usr/lib/system/libsystem_pthread.dylib : thread_start

=================================================================
	Telemetry Dumper:
=================================================================
Pkilling 0x30860668928x from 0x28175265792x
Pkilling 0x19688550400x from 0x28175265792x
Pkilling 0x21708959744x from 0x28175265792x
Pkilling 0x21906845696x from 0x28175265792x
Pkilling 0x15242473472x from 0x28175265792x
Pkilling 0x21428514816x from 0x28175265792x
Pkilling 0x21291429888x from 0x28175265792x
Pkilling 0x6148206592x from 0x28175265792x
Pkilling 0x21282975744x from 0x28175265792x
Pkilling 0x21265199104x from 0x28175265792x
Pkilling 0x6149353472x from 0x28175265792x
Pkilling 0x22034018304x from 0x28175265792x
Pkilling 0x6150500352x from 0x28175265792x
Pkilling 0x26719662080x from 0x28175265792x
Pkilling 0x21023977472x from 0x28175265792x
Pkilling 0x22162731008x from 0x28175265792x
Pkilling 0x6151647232x from 0x28175265792x
Pkilling 0x28352999424x from 0x28175265792x
Pkilling 0x6173929472x from 0x28175265792x
Pkilling 0x28347232256x from 0x28175265792x
Pkilling 0x6152794112x from 0x28175265792x
Pkilling 0x14734274560x from 0x28175265792x
Pkilling 0x15601233920x from 0x28175265792x
Pkilling 0x21298704384x from 0x28175265792x
Pkilling 0x15594795008x from 0x28175265792x
Pkilling 0x21967810560x from 0x28175265792x
Pkilling 0x21808443392x from 0x28175265792x
Pkilling 0x6153940992x from 0x28175265792x
Pkilling 0x15588356096x from 0x28175265792x
Pkilling 0x15581917184x from 0x28175265792x
Pkilling 0x6155087872x from 0x28175265792x
Pkilling 0x21325967360x from 0x28175265792x
Pkilling 0x22842437632x from 0x28175265792x
Pkilling 0x14540779520x from 0x28175265792x
Pkilling 0x19690696704x from 0x28175265792x
Pkilling 0x6158528512x from 0x28175265792x
Pkilling 0x28457857024x from 0x28175265792x
Pkilling 0x14860939264x from 0x28175265792x
Pkilling 0x18651770880x from 0x28175265792x
Pkilling 0x30722256896x from 0x28175265792x
Pkilling 0x27679551488x from 0x28175265792x
Pkilling 0x15105601536x from 0x28175265792x
Pkilling 0x23288901632x from 0x28175265792x
Pkilling 0x21305946112x from 0x28175265792x
Pkilling 0x15603380224x from 0x28175265792x
Pkilling 0x21817028608x from 0x28175265792x
Pkilling 0x7942410240x from 0x28175265792x
Pkilling 0x15596941312x from 0x28175265792x
Pkilling 0x21810589696x from 0x28175265792x
Pkilling 0x15590502400x from 0x28175265792x
Pkilling 0x6147633152x from 0x28175265792x
Pkilling 0x21804150784x from 0x28175265792x
Pkilling 0x15584063488x from 0x28175265792x
Pkilling 0x22185897984x from 0x28175265792x
Pkilling 0x6148780032x from 0x28175265792x
Pkilling 0x21640835072x from 0x28175265792x
Pkilling 0x6149926912x from 0x28175265792x
Pkilling 0x30192726016x from 0x28175265792x
Pkilling 0x22844583936x from 0x28175265792x
Pkilling 0x19065565184x from 0x28175265792x
Pkilling 0x15035248640x from 0x28175265792x
Pkilling 0x22001446912x from 0x28175265792x
Pkilling 0x6151073792x from 0x28175265792x
Pkilling 0x21678288896x from 0x28175265792x
Pkilling 0x18611089408x from 0x28175265792x
Pkilling 0x21697212416x from 0x28175265792x
Pkilling 0x24362643456x from 0x28175265792x
Pkilling 0x6152220672x from 0x28175265792x
Pkilling 0x15267033088x from 0x28175265792x
Pkilling 0x30840745984x from 0x28175265792x
Pkilling 0x15240327168x from 0x28175265792x
Pkilling 0x6153367552x from 0x28175265792x
Pkilling 0x28443766784x from 0x28175265792x
Pkilling 0x6154514432x from 0x28175265792x
Pkilling 0x6155661312x from 0x28175265792x
Pkilling 0x20973711360x from 0x28175265792x
Pkilling 0x22167891968x from 0x28175265792x
Pkilling 0x26717515776x from 0x28175265792x
Pkilling 0x28133847040x from 0x28175265792x
Pkilling 0x21477289984x from 0x28175265792x
Pkilling 0x6159101952x from 0x28175265792x
Pkilling 0x15599087616x from 0x28175265792x
Pkilling 0x28128079872x from 0x28175265792x
Pkilling 0x26724036608x from 0x28175265792x
Pkilling 0x15592648704x from 0x28175265792x
Pkilling 0x15586209792x from 0x28175265792x
Pkilling 0x15579770880x from 0x28175265792x
Pkilling 0x26712502272x from 0x28175265792x
Pkilling 0x21510516736x from 0x28175265792x
Pkilling 0x21630103552x from 0x28175265792x
Pkilling 0x14597140480x from 0x28175265792x
Pkilling 0x21699358720x from 0x28175265792x
Pkilling 0x14538633216x from 0x28175265792x
Entering thread summarizer pause from 0x28175265792x
Finished thread summarizer pause from 0x28175265792x.
Failed to create breadcrumb file (null)/crash_hash_0x65d988019
debugger-agent: Unable to listen on 46

Waiting for dumping threads to resume

=================================================================
	External Debugger Dump:
=================================================================
Asset Pipeline Refresh (id=8e72d5baeaac54786a7f1c3a8d090c11): Total: 0.156 seconds - Initiated by RefreshV2(NoUpdateAssetOptions)
Asset Pipeline Refresh (id=93f76e6cd9f154855a7eb0eabfe6dbef): Total: 0.465 seconds - Initiated by RefreshV2(NoUpdateAssetOptions)
	Summary:
		Imports: total=0 (actual=0, local cache=0, cache server=0)
		Asset DB Process Time: managed=0 ms, native=465 ms
		Asset DB Callback time: managed=0 ms, native=0 ms
		Scripting: domain reloads=0, domain reload time=0 ms, compile time=0 ms, other=0 ms
		Project Asset Count: scripts=3623, non-scripts=11145
		Asset File Changes: new=0, changed=0, moved=0, deleted=0
		Scan Filter Count: 1
	InvokeCustomDependenciesCallbacks: 0.000ms
	InvokePackagesCallback: 0.000ms
	ApplyChangesToAssetFolders: 0.190ms
	Scan: 457.118ms
	GatherAllCurrentPrimaryArtifactRevisions: 0.494ms
	UnloadStreamsBegin: 0.002ms
	UnloadStreamsEnd: 0.121ms
	Untracked: 7.090ms


=================================================================
	Basic Fault Address Reporting
=================================================================
Memory around native instruction pointer (0x46832bdc8):0x46832bdb8  38 4f 78 ff 38 4f 77 ff 35 4b 72 ff 34 4a 71 ff  8Ox.8Ow.5Kr.4Jq.
0x46832bdc8  37 4d 74 ff 36 4c 73 ff 34 4a 71 ff 32 4a 70 ff  7Mt.6Ls.4Jq.2Jp.
0x46832bdd8  35 4c 73 ff 34 4c 72 ff 38 4e 75 ff 37 4e 75 ff  5Ls.4Lr.8Nu.7Nu.
0x46832bde8  39 4f 77 ff 38 4f 77 ff 36 4e 74 ff 36 4e 74 ff  9Ow.8Ow.6Nt.6Nt.

=================================================================
	Managed Stacktrace:
=================================================================
=================================================================
Obtained 20 stack frames.
#0  0x0000046832bdc8 in (Unknown)
#1  0x000004f38a920c in  Pathfinding.ABPath:OnFoundEndNode (uint,uint,uint) [{0x6b64fad38} + 0x4dc] [./Library/PackageCache/com.arongranberg.astar@5.1.1/Pathfinders/ABPath.cs :: 485u] (0x4f38a8d30 0x4f38a925c) [0x15ff62a80 - Unity Child Domain]
#2  0x000004f388db2c in  Pathfinding.Path:CalculateStep (long) [{0x6b64f9d28} + 0x4ec] [./Library/PackageCache/com.arongranberg.astar@5.1.1/Core/Pathfinding/Path.cs :: 1037u] (0x4f388d640 0x4f388deec) [0x15ff62a80 - Unity Child Domain]
#3  0x000004f388d618 in  Pathfinding.Path:Pathfinding.IPathInternals.CalculateStep (long) [{0x6b64f9ac0} + 0x70] [./Library/PackageCache/com.arongranberg.astar@5.1.1/Core/Pathfinding/Path.cs :: 1077u] (0x4f388d5a8 0x4f388d63c) [0x15ff62a80 - Unity Child Domain]
#4  0x0000044e923778 in  Pathfinding.PathProcessor:CalculatePathsThreaded (Pathfinding.PathHandler,Pathfinding.BlockableChannel`1/Receiver<Pathfinding.Path>) [{0x500e1b7b8} + 0x730] [./Library/PackageCache/com.arongranberg.astar@5.1.1/Core/Pathfinding/PathProcessor.cs :: 329u] (0x44e923048 0x44e923f64) [0x15ff62a80 - Unity Child Domain]
#5  0x0000044e922fbc in  Pathfinding.PathProcessor/<>c__DisplayClass27_0:<StartThreads>b__0 () [{0x500e1b580} + 0x8c] [./Library/PackageCache/com.arongranberg.astar@5.1.1/Core/Pathfinding/PathProcessor.cs :: 121u] (0x44e922f30 0x44e922fe8) [0x15ff62a80 - Unity Child Domain]
#6  0x000005c6c35ed0 in  System.Threading.ThreadHelper:ThreadStart_Context (object) [{0x371c1c7c0} + 0x120]  (0x5c6c35db0 0x5c6c35f78) [0x15ff62a80 - Unity Child Domain]
#7  0x000005c6c34d14 in  System.Threading.ExecutionContext:RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [{0x371c1d648} + 0x3d4]  (0x5c6c34940 0x5c6c34da0) [0x15ff62a80 - Unity Child Domain]
#8  0x000005c6c34384 in  System.Threading.ExecutionContext:Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [{0x371c1d5f0} + 0x6c]  (0x5c6c34318 0x5c6c343a8) [0x15ff62a80 - Unity Child Domain]
#9  0x000005c6c340f0 in  System.Threading.ExecutionContext:Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object) [{0x371c1d578} + 0x118]  (0x5c6c33fd8 0x5c6c34118) [0x15ff62a80 - Unity Child Domain]
#10 0x000005c6c33ea4 in  System.Threading.ThreadHelper:ThreadStart () [{0x371beadf8} + 0x84]  (0x5c6c33e20 0x5c6c33f38) [0x15ff62a80 - Unity Child Domain]
#11 0x00000568565470 in  (wrapper runtime-invoke) object:runtime_invoke_void__this__ (object,intptr,intptr,intptr) [{0x371c1d410} + 0x260]  (0x568565210 0x5685654c0) [0x15ff62a80 - Unity Child Domain]
#12 0x0000036ce8f8fc in mono_jit_runtime_invoke
#13 0x0000036d014d30 in do_runtime_invoke
#14 0x0000036d0166e0 in mono_runtime_delegate_try_invoke
#15 0x0000036d036d84 in start_wrapper_internal
#16 0x0000036d036b3c in start_wrapper
#17 0x0000036d0b6038 in GC_inner_start_routine
#18 0x0000036d0b5fc0 in GC_start_routine
#19 0x00000182e9a034 in _pthread_start
Launching bug reporter
Attribute Qt::AA_EnableHighDpiScaling must be set before QCoreApplication is created.
e[40me[32minfoe[39me[22me[49m: Microsoft.Hosting.Lifetime[0]
      Application is shutting down...
e[40me[32minfoe[39me[22me[49m: Unity.ILPP.Runner.PostProcessingAssemblyLoadContext[0]
      ALC ILPP context 34 is unloading
2024-06-08 12:26:39.665 UnityBugReporter[17625:8486440] WARNING: Secure coding is not enabled for restorable state! Enable secure coding by implementing NSApplicationDelegate.applicationSupportsSecureRestorableState: and returning YES.

aron_granberg · June 10, 2024, 8:21am

Hi

That particular function has no usage of unsafe code, so if it’s crashing there, then I would think it’s likely a unity bug. I’d try to upgrade to a more recent version of Unity as a first potential solution.

Wrymn · June 10, 2024, 9:58am

We have almost the newest unity version, so I doubt the issue is with their code.
Also I see the method OnFoundEndNode is called from Pathfinding.PathProcessor:CalculatePathsThreaded, so its in different thread, this means there is most likely some race condition or some unsafe memory access that is causing this issue.

Please could you investigate this more, maybe add some locks?

aron_granberg · June 10, 2024, 10:06am

Unfortunately, it is very hard for me to investigate without being able to reproduce the issue.
Can you reliably reproduce the issue? Do you always get the same stack trace?

There’s both Unity 2023.3 and Unity 6. I’d recommend trying those out to see if it fixes the issue.

Wrymn · June 10, 2024, 10:19am

Unity 2023.3 unfortunately does not exist

We cannot use Unity 6 as they have critical issue with UI Elements we are using, so our entire project is broken after upgrade.

I am trying newest non Unity6 version 2023.2.20, but I doubt it will make any change.

I cannot deterministically reproduce the issue, no. As I said, its very random and rare. Only thing I have is the stacktrace from recent crash report.

All I can do is wait for another crash and check the error log again, if it points to the same method.

aron_granberg · June 10, 2024, 10:26am

Pretty sure it does, as I have it installed. Though there may only exist a beta, I’m not sure.

Wrymn · June 10, 2024, 10:48am

But that is just Unity 6 before rename as it seems:
https://forum.unity.com/threads/unity-6-beta-2023-3-feature-highlights.1532590/