VisonOS Image tracking help

Hi all, I need some help debugging some code I wrote. Just as a preface, I'm an extremely new VR/AR developer and also very new to using ARKit + RealityKit. So please bear with me :) I'm just trying to make a simple program that will track an image and place an entity on it. The image is tracked correctly, but the moment the program recognizes the image and tries to place an entity on it, the program crashes. Here’s my code:

VIEWMODEL CODE:

Observable class ImageTrackingModel {

    var session = ARKitSession() // ARSession used to manage AR content
    var imageAnchors = [UUID: Bool]() // Tracks whether specific anchors have been processed
    var entityMap = [UUID: ModelEntity]() // Maps anchors to their corresponding ModelEntity
    var rootEntity = Entity() // Root entity to which all other entities are added


    let imageInfo = ImageTrackingProvider(
        referenceImages: ReferenceImage.loadReferenceImages(inGroupNamed: "referancePaper")
    )

    init() {
        setupImageTracking()
    }

    func setupImageTracking() {
        if ImageTrackingProvider.isSupported {
            Task {
                try await session.run([imageInfo])
                for await update in imageInfo.anchorUpdates {
                    updateImage(update.anchor)
                }
            }
        }
    }

    func updateImage(_ anchor: ImageAnchor) {
        let entity = ModelEntity(mesh: .generateSphere(radius: 0.05)) // THIS IS WHERE THE CODE CRASHES

        if imageAnchors[anchor.id] == nil {
            rootEntity.addChild(entity)
            imageAnchors[anchor.id] = true
            print("Added new entity for anchor \(anchor.id)")
        }
        if anchor.isTracked {
            entity.transform = Transform(matrix: anchor.originFromAnchorTransform)
            print("Updated transform for anchor \(anchor.id)")
        }
    }
}

APP:


@main
struct MyApp: App {
    @State var session = ARKitSession()
    @State var immersionState: ImmersionStyle = .mixed
    private var viewModel = ImageTrackingModel()
    var body: some Scene {
        WindowGroup {
            ModeSelectView()
        }
        ImmersiveSpace(id: "appSpace") {
            ModeSelectView()
        }
        .immersionStyle(selection: $immersionState, in: .mixed)
    }
}

Content View:

RealityView { content in
            Task {
                viewModel.setupImageTracking()
            }
        } //Im serioulsy so clueless on how to use this view

Replies

Could you share the console crash log for more context? One idea coming to my mind: updateImage() might be called on a non-main thread. And afaik ModelEntity should be created on the main- tread. So you could try marking the function as @MainActor.

  • Thanks for your response. This is the error messages that shows up where I put the comment in my initial code. Thread 2: EXC_BREAKPOINT (code=1, subcode=0x104d2bbfc)

    I also tired this

    RealityView { content in
                Task { @MainActor in
                    viewModel.setupImageTracking()
                }
            }
    

    But it did not work :(

  • Running the function with @MainActor also did not fix the issue

Add a Comment