Weird spinning glitch from Apple's Demo "DragRotationModifier"

Hello, I have started using the DragRotationModifier from the Hello World demo project by Apple. I have run into a bug that I can't seem to figure out for the life of me where everything seems to work fine for about 3-5 seconds of moment before it starts rapidly spinning for some reason. I took a video but it looks like I am unable to post any link to outside sources like imgur or youtube so ill try to describe it as best I can:

Basically I can spin the sample USDZ Nike Airforce from the Apple sample objects perfectly, but after about 3-5 seconds it seems to rapidly snap between different other rotations and the rotation where the "cursor" is. A couple of additional notes, this only happens when the finger pinch/drag gesture is interacting with the object and this spin only affects the Yaw rotation axis of the object.

I created an "Imported Model Entity" wrapper that does some additional stuff when importing a USDZ model similar to the Hello World demo. Then, within a RealityView I create an instance of this ImportedModelEntity and attach the Drag Rotation Modifier to the view like this:

RealityView { content in
    let modelEntity = await ImportedModelEntity(configuration: modelViewModel.modelConfiguration)
    content.add(modelEntity)
    self.modelEntity = modelEntity
    content.add(BoundsVisualizer(bounds: [0.6, 0.6, 0.6]))
    
    //Scale object to half of the size of Volume view
    let bounds = content.convert(geometry.frame(in: .local), from: .local, to: content)
    let minExtent = bounds.extents.min()
    
    modelViewModel.modelConfiguration.scale = minExtent
    
}
update: { content in
    modelEntity?.update(configuration: modelViewModel.modelConfiguration)
}
.if(modelEntity != nil) { view in
    view.dragRotation(
        pitchLimit: .degrees(90),
        targetEntity: modelEntity!,
        sensitivity: 10,
        axRotateClockwise: axRotateClockwise,
        axRotateCounterClockwise: axRotateCounterClockwise)
}

For reference here is my ImportedModelEntity:

import Foundation
import RealityKit

class ImportedModelEntity: Entity {
    
    // MARK: - Sub-entities
    
    private var model: ModelEntity = ModelEntity()
    private let rotator = Entity()
    
    // MARK: - Internal state
    
    
    // MARK: - Initializers
    
    @MainActor required init() {
        super.init()
    }
    
    init(
        configuration: Configuration
    ) async {
        super.init()
        
        if(configuration.modelURL == nil) {
            fatalError("Provided modelURL is NOT valid!!")
        }
        
        //Load the custom model on main thread
//        DispatchQueue.main.async {
            do {
                let input: ModelEntity? = try await ModelEntity(contentsOf: configuration.modelURL!)
                guard let model = input else { return }
                self.model = model
                
//                let material = SimpleMaterial(color: .green, isMetallic: false)
//                model.model?.materials = [material]
                
                
                
                //Add input components
                model.components.set(InputTargetComponent())
                model.generateCollisionShapes(recursive: true)
                
                // Add Hover Effect
                model.components.set(HoverEffectComponent())
                
                //self.model.components.set(GroundingShadowComponent(castsShadow: configuration.castsShadow))
                
                // Add Rotator
                self.addChild(rotator)
                
                // Add Model to rotator
                rotator.addChild(model)
            } catch is CancellationError {
                // The entity initializer can throw this error if an enclosing
                // RealityView disappears before the model loads. Exit gracefully.
                return
            } catch let error {
                print("Failed to load model: \(error)")
            }
//        }
        update(configuration: configuration)
    }
    
    //MARK: - Update Handlers
    
    func update(
        configuration: Configuration)
    {
        rotator.orientation = configuration.rotation
        
        move(to: Transform(
            scale: SIMD3(repeating: configuration.scale), rotation: orientation, translation: configuration.position), relativeTo: parent)
        
    }
}

Any help is greatly appreciated!