Passkeys in AppClip shows intermittently "Couldn't communicate with a helper application."

I have a demo AppClip which does Passkey registration and assertion in a wizard style page collection. Very often, the assertion would fail with the error "Couldn't communicate with a helper application". Retrying it few times will eventually work.

Full error:

Connection to agent service interrupted with error: Error Domain=NSCocoaErrorDomain Code=4097 "Couldn’t communicate with a helper application."

I don't know exactly what's causing this and how to prevent it? It only seems to happen during assertion, not registration.