Our product includes a sudo plugin so we can apply user-defined policies to manage privileged access to command line programs. We’ve been getting reports where the plugin sometimes doesn't get invoked and the sudo command falls back to its default behavior. This seems to only be happening intermittently, but when the issue does occur, this message appears in the Console:
Library Validation failed: Rejecting '/usr/local/libexec/sudo/<our_plugin>.so' (Team ID: <OURTEAMID>, platform: no) for process 'sudo(<pid>)’ (Team ID: N/A, platform: yes), reason: mapping process is a platform binary, but mapped file is not
I recall a previous discussion of this message (that I can’t locate now), which explained that although the host process has library validation disabled, the code flow raises an error anyway, so that the host process can detect it and bypass the validation to load the plugin. It looks like that's what sudo is doing: it has the private entitlement com.apple.private.security.clear-library-validation and makes the appropriate system call when the plugin initially fails to load [1] — but apparently this isn't working reliably for our sudo plugin.
We’ve observed that restarting the Mac generally resolves the issue, at least for a while. This resembles the “classic symptom of a code signing oddity” where the signature is cached and the Mach-O image is rewritten rather than replaced (as documented in Updating Mac Software). But our software uses an Installer package for updates as well as initial installation, and the Installer is documented as not having this issue, so I believe the problem lies somewhere else.
I’m running out of ideas; are there any other avenues I should investigate? Thanks for any help.
[1] This is described in an article called "About com.apple.private.security.clear-library-validation"; I can't link to it directly from the developer forums, but it can easily be found by searching for the title.