Finding device heading with ARKit and ARGeoTrackingConfiguration

I am working on an app where I need to orient a custom view depending on the device heading. I am using ARKit and ARSCNView with the ARGeoTrackingConfiguration in order to overlay my custom view in real world geographic coordinates. I've got a lot of it working, but the heading of my custom view is off.

Once the ARSession gets a ARGeoTrackingStatus.State of .localized, I need to be able to get the devices heading (0-360) so that I can orient my view. I'm having trouble figuring out how to do this missing piece. Any help is appreciated.

Post not yet marked as solved Up vote post of rolson Down vote post of rolson
791 views

Replies

I no longer have a need for this. I misunderstood the worldAlignment property. Now that I understand it, I don't need this specific question answered

We still need the answer to this question.

Our usecase in short:

  • ❓ For ARGeoTrackingConfiguration, we want to set a custom view to have the same heading as the AR camera, so it can provide geo-referenced symbol with correct heading in real world. However, we couldn't find the right way to access AR camera's heading.
  • ✅ For ARWorldTrackingConfiguration, setting the custom view heading to zero and AR config worldAlignment = gravityAndHeading makes their headings aligned.

More details:

  • I tried many approaches for the custom view's heading. Euler Angle seems to be the most promising, but we don't know how to exactly interpret that vector.

    • i) setting heading to zero,
    • ii) setting heading to the most recent compass heading,
    • iii) using AR camera’s eulerAngles,
    • iv) tinkering with the transformation matrix.
  • When I used the recorded data following Recording and Replaying AR Session Data tutorial, I found that the initial heading deflection is almost always the same value, no matter when the AR session becomes localized.

  • My assumption is that - when the geo-tracking gets localized, it provides additional tracking information so the AR frame is somehow geo-referenced. In contrast, the world tracking doesn’t have anchors, thus using zero as initial heading is fine.

  • Apple's WWDC demo in San Francisco seems to have hardcoded the orientation of the 3D text? From the public demo projects in the documentation, I didn't find a way to get the heading for ARGeoTrackingConfiguration.

  • for reference, WWDC SF Ferry Building demo from this video: https://developer.apple.com/videos/play/wwdc2020/10611/ which hardcodes the text orientation.

  • Have you checked out this sample?

  • Yes, I tried that sample and it helped me understand many concepts related to geo-tracking. However, when it adds the blue sphere model entity, it doesn't set the heading of it, which makes sense as a sphere is isotropic. In our usecase, we need to know the heading so that we can align our overlay with the AR view.

Add a Comment

Another finding during our test is that, regardless of geo-tracking or world-tracking configuration, when I first manually adjust the heading of our custom view to be aligned with the AR view, then start to turn quickly around 360 degrees, or several circles consecutively, the custom view's heading alignment will drift drastically.

In order to restore the alignment, I had to reset the session and re-localize, or manually calibrate the heading according to some visual reference POIs.

Is it due to the device electronic compass' intrinsical limitation?

Also, if there isn't a good answer to the original question, we might just end up presenting a toast to guide the user to manually adjust the heading, whenever it is mis-aligned with the AR view.

Thanks.