RVOSimulator crash on some machines

Hi Aron,

We’re in the middle of investigating a crash that a number of users are experiencing in DEVOUR that seems to stem from the RVOSimulator. We have never been able to replicate it on any of our in house machines. However using a tool for Unity called Backtrace (https://backtrace.io) we have managed to get a Stacktrace that has pointed us in hopefully the right direction (the crash was fully lost when relying on Unity crash reporting and no crash dump was generated). We have RVO simulators running on every scene in the game. The majority are configured with 2 threads and there are a couple that are set to Automatic High Load. Users describe the crash as the game window going white and the cursor becoming a spinner as though the entire process has hung. The crash is able to occur at any time and has happened in scenes with 2 threads and automatic high load and importantly with no enabled RVOControllers. This is consistent with the crash report in Backtrace. Backtrace indicates that a particular thread has hung and become non responsive.

Thread stacktrace:

[ 00 ] NtWaitForSingleObject
[ 01 ] WaitForSingleObjectEx
[ 02 ] RewiredStandaloneInputModule_IsDefaultPlayer_m5CC829EBCAFBC4CFA21C035917B75E1C7C35AE74(RewiredStandaloneInputModule_tD07E9B6060BB6F8005D33B39335F2B3470C5318D *,int,MethodInfo const *)
[ 03 ] WaitForSingleObjectEx
[ 04 ] il2cpp::os::win::WaitForSingleObjectAndAccountForAPCs(void *,unsigned int,bool)
[ 05 ] il2cpp::vm::Monitor::TryWait(Il2CppObject *,unsigned int)
[ 06 ] ManualResetEventSlim_Wait_m56CBDF8A22EB2F5D7245DDB0EF06FF4BE0EC8A7A(ManualResetEventSlim_tDEDF52539E364C425BA581F3AAF42843AFAD366E *,int,CancellationToken_tC9D68381C9164A4BA10397257E87ADC832AF5FFD,MethodInfo const *)
[ 07 ] Simulator_Update_m52268158BFD7621F81A86FEAF37D2C9B792C1AD1(Simulator_tC73D3ABDC4E347B3D85214184B547C50D95AE722 *,MethodInfo const *)
[ 08 ] RVOSimulator_Update_mB5CA3F8DA0B0969A7711DC0B9120CC2275B7C97C(RVOSimulator_t6C3ED3EC65D1B3A6CCD6444868ECF6280E0CB9C3 *,MethodInfo const *)
[ 09 ] RuntimeInvoker_TrueVoid_t700C6383A2A510C2CF4DD86DABD5CA9FF70ADAC5((),MethodInfo const *,void *,void * *)
[ 10 ] il2cpp::vm::Runtime::Invoke(MethodInfo const *,void *,void * *,Il2CppException * *)
[ 11 ] scripting_method_invoke(class ScriptingMethodPtr,class ScriptingObjectPtr,struct ScriptingArguments &,class ScriptingExceptionPtr *,bool)
[ 12 ] public: ScriptingInvocation::Invoke(class ScriptingExceptionPtr *,bool)
[ 13 ] private: MonoBehaviour::CallMethodIfAvailable(int)
[ 14 ] private: MonoBehaviour::CallUpdateMethod(int)
[ 15 ] protected: BaseBehaviourManager::CommonUpdate<class BehaviourManager>(void)
[ 16 ] ExecutePlayerLoop(struct NativePlayerLoopSystem *)
[ 17 ] ExecutePlayerLoop(struct NativePlayerLoopSystem *)
[ 18 ] PlayerLoop(void)
[ 19 ] PerformMainLoop
[ 20 ] MainMessageLoop
[ 21 ] UnityMainImpl(struct HINSTANCE__ *,struct HINSTANCE__ *,wchar_t *,int)
[ 22 ] UnityMain
[ 23 ] __scrt_common_main_seh
[ 24 ] BaseThreadInitThunk
[ 25 ] RtlUserThreadStart

(The entry with Rewired is due to missing symbols/corruption of the stack according to Backtrace so it is almost certainly wrong)

We believe that the number of crashes that are coming from this may have gone up significantly after we upgraded from 2019.4.30f1 to 2020.3.29f1.

Do you have any ideas while we continue to get to the root cause of the issue? Once we have garnered crashes from some more unique users, our plan is to reduce the thread count on the RVO simulators to one and then get them to re-test in order to pin this down as some kind of multithreading issue.

Thanks

A* version: 4.2.15
Unity version: 2020.3.29f1

We went ahead with changing all of our RVO simulators to being single threaded and it does seem to have eliminated the crashing issue. No reports since doing it about 4-5 days ago

Hi

Sorry for the late reply.
That’s unfortunate. If you want, you can try out the beta version (A* Pathfinding Project), the multithreading parts have been completely rewritten, and I think it’s a lot more stable.