visionOS: Moving window with a Map causes app to freeze

I'm developing a map-based app for visionOS. The loads map data from a server, using JSON. It works just fine, but I noticed the following effect: If I move the app's window around, it freezes; either on the first movement, or on one of the subsequent ones. The map cannot be panned anymore, and all other UI elements lose their interactivity as well.

I noticed this issue before, when I was opening the map on app startup (and here it even happened without moving the window). Since I added a short delay, this was resolved. There was no log message in this case.

However, when I noticed that it also happens if I move the window around, I saw that Xcode logs an error:

+[UIView setAnimationsEnabled:] being called from a background thread. Performing any operation from a background thread on UIView or a subclass is not supported and may result in unexpected and insidious behavior. trace=(
	0   UIKitCore                           0x0000000185824a24 __42+[UIView(Animation) setAnimationsEnabled:]_block_invoke + 112
	1   libdispatch.dylib                   0x0000000102a327e4 _dispatch_client_callout + 16
	2   libdispatch.dylib                   0x0000000102a34284 _dispatch_once_callout + 84
	3   UIKitCore                           0x0000000185824ad8 +[UIView(Animation) performWithoutAnimation:] + 56
	4   SwiftUI                             0x00000001c68cf1e0 OUTLINED_FUNCTION_136 + 10376
	5   SwiftUI                             0x00000001c782bebc OUTLINED_FUNCTION_12 + 22864
	6   SwiftUI                             0x00000001c78285e8 OUTLINED_FUNCTION_12 + 8316
	7   SwiftUI                             0x00000001c787c288 OUTLINED_FUNCTION_20 + 39264
	8   SwiftUI                             0x00000001c787c2cc OUTLINED_FUNCTION_20 + 39332
	9   UIKitCore                           0x000000018582fc24 -[UIView(CALayerDelegate) layoutSublayersOfLayer:] + 1496
	10  QuartzCore                          0x000000018a05cf00 _ZN2CA5Layer16layout_if_neededEPNS_11TransactionE + 440
	11  QuartzCore                          0x000000018a068ad0 _ZN2CA5Layer28layout_and_display_if_neededEPNS_11TransactionE + 124
	12  QuartzCore                          0x0000000189f80498 _ZN2CA7Context18commit_transactionEPNS_11TransactionEdPd + 460
	13  QuartzCore                          0x0000000189fb00b0 _ZN2CA11Transaction6commitEv + 652
	14  VectorKit                           0x00000001938ee620 _ZN2md12HoverSupport18updateHoverProxiesERKNSt3__16vectorINS1_10shared_ptrINS_5LabelEEEN3geo12StdAllocatorIS5_N3mdm9AllocatorEEEEE + 2468
	15  VectorKit                           0x0000000193afd1cc _ZN2md15StandardLabeler16layoutForDisplayERKNS_13LayoutContextE + 156
	16  VectorKit                           0x0000000193cf133c _ZN2md16CompositeLabeler16layoutForDisplayERKNS_13LayoutContextE + 52
	17  VectorKit                           0x0000000193abf318 _ZN2md12LabelManager6layoutERKNS_13LayoutContextEPKNS_20CartographicRendererERKNSt3__113unordered_setINS7_10shared_ptrINS_12LabelMapTileEEENS7_4hashISB_EENS7_8equal_toISB_EEN3geo12StdAllocatorISB_N3mdm9AllocatorEEEEERNS_8PassListE + 2904
	18  VectorKit                           0x0000000193cad464 _ZN2md9realistic16LabelRenderLayer6layoutERKNS_13LayoutContextE + 464
	19  VectorKit                           0x0000000193658b54 _ZNSt3__110__function6__funcIZN2md9realistic20RealisticRenderLayer5frameERNS2_13LayoutContextEE3$_0NS_9allocatorIS7_EEFvvEEclEv + 180
	20  VectorKit                           0x00000001936584cc ___ZN3geo9TaskQueue14queueAsyncTaskENSt3__110shared_ptrINS_4TaskEEEPU28objcproto17OS_dispatch_group8NSObject_block_invoke + 80
	21  libdispatch.dylib                   0x0000000102a30f98 _dispatch_call_block_and_release + 24
	22  libdispatch.dylib                   0x0000000102a327e4 _dispatch_client_callout + 16
	23  libdispatch.dylib                   0x0000000102a3aa80 _dispatch_lane_serial_drain + 916
	24  libdispatch.dylib                   0x0000000102a3b7c4 _dispatch_lane_invoke + 420
	25  libdispatch.dylib                   0x0000000102a3c794 _dispatch_workloop_invoke + 864
	26  libdispatch.dylib                   0x0000000102a481a0 _dispatch_root_queue_drain_deferred_wlh + 324
	27  libdispatch.dylib                   0x0000000102a475fc _dispatch_workloop_worker_thread + 488
	28  libsystem_pthread.dylib             0x0000000103b0f924 _pthread_wqthread + 284
	29  libsystem_pthread.dylib             0x0000000103b0e6e4 start_wqthread + 8

I disabled all my withAnimation() statements, and the problem persists. I also thought it might be related to my own network fetches, but I think all apply their changes on the main thread. And when I turn on network logging for my own fetching logic, I do not see any data coming in. I also do not think there should be a reason for it.

How can I debug such a situation, so I know, which call actually threw this message? I'd like to know if it is my code or a bug in the SwiftUI map itself.