Understanding Sandbox Violations for system services

Adding some additional information. It seems like Family, imagent and searchpartyuseragent each do eight (8) tries to interface via XPC (CoreData: XPC: sendMessage: failed #5 up to #7 starting from #0) and then giving up and initiating a round of Diagnostic reporter after that.

Example of a failure:

default	23:23:45.419364+0100	Family	[0x130e0eb70] activating connection: mach=true listener=false peer=false name=com.apple.contactsd.persistence
error	23:23:45.419593+0100	kernel	Sandbox: Family(1316) deny(1) mach-lookup com.apple.contactsd.persistence
default	23:23:45.419765+0100	Family	[0x130e0eb70] failed to do a bootstrap look-up: xpc_error=[159: Unknown error: 159]
default	23:23:45.419812+0100	Family	[0x130e0eb70] invalidated after a failed init
error	23:23:45.419970+0100	Family	Persistent store service connection invalidated: failed at lookup with error 159 - Sandbox restriction
error	23:23:45.420160+0100	Family	Error communicating with persistent store service proxy: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.contactsd.persistence was invalidated: failed at lookup with error 159 - Sandbox restriction." UserInfo={NSDebugDescription=The connection to service named com.apple.contactsd.persistence was invalidated: failed at lookup with error 159 - Sandbox restriction.}
error	23:23:45.420211+0100	Family	Error connecting to remote endpoint: (null)
error	23:23:45.420267+0100	Family	fault: Unable to create token NSXPCConnection.  NSXPCStoreServerEndpointFactory 0x130e0b5f0 -newEndpoint returned nil
fault	23:23:45.420356+0100	Family	CoreData: Unable to create token NSXPCConnection.  NSXPCStoreServerEndpointFactory 0x130e0b5f0 -newEndpoint returned nil
error	23:23:45.420476+0100	Family	error: Failed to create NSXPCConnection
default	23:23:45.420572+0100	Family	[0x132123fb0] activating connection: mach=true listener=false peer=false name=com.apple.contactsd.persistence
default	23:23:45.420708+0100	Family	[0x132123fb0] failed to do a bootstrap look-up: xpc_error=[159: Unknown error: 159]
default	23:23:45.420743+0100	Family	[0x132123fb0] invalidated after a failed init
error	23:23:45.420789+0100	Family	Persistent store service connection invalidated: failed at lookup with error 159 - Sandbox restriction
error	23:23:45.420895+0100	Family	Error communicating with persistent store service proxy: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.contactsd.persistence was invalidated: failed at lookup with error 159 - Sandbox restriction." UserInfo={NSDebugDescription=The connection to service named com.apple.contactsd.persistence was invalidated: failed at lookup with error 159 - Sandbox restriction.}
error	23:23:45.420938+0100	Family	Error connecting to remote endpoint: (null)
error	23:23:45.420979+0100	Family	fault: Unable to create token NSXPCConnection.  NSXPCStoreServerEndpointFactory 0x130e0b5f0 -newEndpoint returned nil
fault	23:23:45.421076+0100	Family	CoreData: Unable to create token NSXPCConnection.  NSXPCStoreServerEndpointFactory 0x130e0b5f0 -newEndpoint returned nil
error	23:23:45.421154+0100	Family	error: Failed to create NSXPCConnection
default	23:23:45.421214+0100	Family	CoreData: XPC: sendMessage: failed #5

Example of given up after eight (8) tries:

error	23:23:54.433727+0100	Family	error: Failed to create NSXPCConnection
error	23:23:54.434130+0100	Family	error: addPersistentStoreWithType:configuration:URL:options:error: returned error NSCocoaErrorDomain (134060)
error	23:23:54.434160+0100	Family	error: userInfo:
error	23:23:54.434185+0100	Family	error: 	Problem : Unable to send to server; failed after 8 attempts.
error	23:23:54.434300+0100	Family	error: storeType: NSXPCStore
error	23:23:54.434323+0100	Family	error: configuration: (null)
error	23:23:54.434345+0100	Family	error: URL: file:///Users/<USER>/Library/Application%20Support/AddressBook/Sources/<UUID>/AddressBook-v22.abcddb
default	23:23:54.434366+0100	Family	annotation: options:
default	23:23:54.434400+0100	Family	annotation: 	NSReadOnlyPersistentStoreOption : 0
default	23:23:54.434422+0100	Family	annotation: 	NSXPCStoreServerEndpointFactory : <CNCDRemotePersistentStoreEndpointFactory: 0x130e0b5f0>
default	23:23:54.434443+0100	Family	annotation: 	skipModelCheck : 1
default	23:23:54.434463+0100	Family	annotation: 	NSPersistentHistoryTrackingKey : 1
error	23:23:54.434545+0100	Family	Unable to open XPC store: Error Domain=NSCocoaErrorDomain Code=134060 "A Core Data error occurred." UserInfo={Problem=Unable to send to server; failed after 8 attempts.} {
    Problem = "Unable to send to server; failed after 8 attempts.";
}
default	23:23:54.434570+0100	Family	Did add prepared store without migration (path: file:///Users/<USER>/Library/Application%20Support/AddressBook/Sources/<UUID>/AddressBook-v22.abcddb)
default	23:23:54.434601+0100	Family	Will add unprepared store without migration (path: file:///Users/<USER>/Library/Application%20Support/AddressBook/Sources/<UUID>/AddressBook-v22.abcddb)
default	23:23:54.434715+0100	Family	Will add XPC store with options: <private> <private>

After this the Diagnostics Reporter runs, which hogs a lot of resources. My current issue is that these continuous Diagnostics Reporter runs severely bogs down my machines. Currently hogging 2 CPUs each on a M1 Macbook pro (13-inch) and M2 Macbook pro (14-inch).

I can only really help with sandbox issues that are triggered by code that you write. If unrelated system components are triggering sandbox violations, that’s something that you should discuss with Apple Support, either directly or via their Apple Support Community.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank, I'll have a look at Apple Support then. There are a couple of thread in the Apple Support Community which are similar, and they all lack proper solutions.

But to be abundantly clear, there is no way for a end-user to alter mach-lookup permissions / Sandbox rules for system services?

Adding to above, both for posterity and hoping that someone will pick it up and poke someone internally at Apple :-)

I went jumping through all the hoops necessary (disabling SIP, remounting, clicking through warning messages about insecure setup etc) to modify sandbox-profiles for system services (i.e. those in /System/Library/Sandbox/Profiles/). Adding explicit allow of mach-lookup for com.apple.contactsd.persistence to com.apple.imagent.sb (assuming this should alter the process sandbox for imagent), com.apple.coredatad.sb (Family-process ought to stem from there?) and searchpartyuseragent.sb (for process searchpartyuseragent) alter their behaviour, but the "Family Framework" seems to be in such a bad state without SIP et al enabled I don't really know if it actually fixes the issue.

Anyhow, I want to explicitly note that neither com.apple.imagent.sb, com.apple.coredatad.sb nor searchpartyuseragent.sb seems to include an allow of mach-lookups for com.apple.contactsd.persistence (however, some of them allow lookups to other contacts related mach services directly or through imports, but not com.apple.contactsd.persistence specifically). This means that the sandbox-violations in the OP are to be expected. Therefore it is very weird that Apple-services are asking for that kind of permission. @eskimo Is this intended or a bug? Can you / do you want to poke around internally at Apple a bit? There are several comments in the shared sandbox specification for the messages app / framework (?) (com.apple.iMessage.shared.sb) indicating that the sandbox profile for the messages-framework is a work-in-progress.

Also potentially interesting, these are the events that seem to start the Diagnostics Reporter:

error	18:02:09.385361+0100	ContactsPersistence	com.apple.contacts	PeopleMessagesAskToBuy	persistence	Error communicating with persistent store service proxy: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.contactsd.persistence was invalidated: failed at lookup with error 159 - Sandbox restriction." UserInfo={NSDebugDescription=The connection to service named com.apple.contactsd.persistence was invalidated: failed at lookup with error 159 - Sandbox restriction.}
error	18:02:09.385407+0100	ContactsPersistence	com.apple.contacts	PeopleMessagesAskToBuy	persistence	Error connecting to remote endpoint: (null)
error	18:02:09.385441+0100	CoreData	com.apple.coredata	PeopleMessagesAskToBuy	error	fault: Unable to create token NSXPCConnection.  NSXPCStoreServerEndpointFactory 0x14214d4c0 -newEndpoint returned nil
fault	18:02:09.385482+0100	CoreData	com.apple.coredata	PeopleMessagesAskToBuy	error	CoreData: Unable to create token NSXPCConnection.  NSXPCStoreServerEndpointFactory 0x14214d4c0 -newEndpoint returned nil
error	18:02:09.385569+0100	CoreData	com.apple.coredata	PeopleMessagesAskToBuy	error	error: Failed to create NSXPCConnection

The above log entries show up with the messages app open, and are related to app approvals within the "Family Framework". The approvals don't properly "resolve" on any of my Macs (all arm-based), but show up fine on iOS-devices. See the attached image below:

Not using the messages-app and changing automatic sync of contacts to once per hour seems to alleviate the problem a lot, but not completely.

I delved a bit deeper and tried messing with everything related to "Family" and "Contacts", and it seems that the culprit is (was?) multifaceted but centered in "The local-part of a mailbox MUST BE treated as case sensitive." from RFC-5321. Apple respects this, but few other contact frameworks do. This lead to:

• collisions with both Nextcloud and Google contacts, which seems to cause a lot of synchronisation activity and caused linked / merged contacts to split up.
• issues with Messages.app since Messages + Contacts respects capitalisation in email-addresses. I.e., if your contact book contains an entry for the person "First Last" with the email First.Last@.. and you receive an iMessage from first.last@.. the message will not be connected to the contact. This cause quite weird behaviour in group chats (Apple Mail seems to handle capitalisation though). This was even worse when a family member had mismatching capitalisation of email-addresses in Contacts vs Messages in a "request to buy" family scenario.

To fix it I looped through all contacts and fixed offending entries. I used the logic below:

for f in ${HOME}/Library/Application\ Support/AddressBook/Sources/*/Metadata/*ABPerson.abcdp;
do
    # Check for the presence of email
    if plutil -convert raw -o - $f |ggrep -q "^Email$"; then
        ## This contact has at least one email
        NREMAIL=$(plutil -extract Email.labels raw -o - $f)
        
        ## Loop over emails etc and check for weird stuff
        ## ... magic sauce ...
    fi
done

I can share the actual script if someone wants it, but I don't want to post it here since it potentially could mess up the contact book.

I still don't understand Sandbox Violations for system services, especially how to debug them, but this seemed to have solved the actual symptoms, i.e. massive CPU-usage, for me.

@flindeberg please share your script, I'm fighting with this for a long time now and really want to try your solution, thank you in advance.