Library validation failed: problems with dynamic library

I am developing a new pam module in Monterey [12.6], where I have a dynamic library [.so file] (usage external curl & openssl library) which is referenced from PAM.

More specifically, this is a setup to allow Multi factor Authentication to be used for all authentication.it simply calls some apis.

When I added this module for sudo authentication in /etc/pam.d/sudo file as

auth       sufficient     /usr/local/lib/security/pam_google_authenticator.so

It logs "Library Validation failed: Rejecting 'pam_google_authenticator.so' (Team ID: XXXXXXX, platform: no) for process 'sudo(2498)' (Team ID: none, platform: yes), reason: mapping process is a platform binary, but mapped file is not" but it still loads my pam module and everything is working fine.

But when I added this module for lock screen into /etc/pam.d/screensaver same as above, it logs "Library Validation failed: Rejecting 'pam_google_authenticator.so' (Team ID: XXXXXXXX, platform: no) for process 'loginwindow(15839)' (Team ID: none, platform: yes), reason: mapping process is a platform binary, but mapped file is not" and took back to logon window[not sleep window]

I have code signed pam_google_authenticator.so with

codesign --force --deep --sign "Developer ID Application: --------------(XXXXXXX)" /usr/local/lib/securitypam_google_authenticator.so

For your reference I can here are the logs in console app crash report

System Integrity Protection: enabled
Crashed Thread:        3  Dispatch queue: com.apple.loginwindow.auth
Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY
Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   exc handler [25964]

I have found other references to this error but those seem to involve application bundles. In my case I have a single .so library (plus the two others) I wish to invoke.

The library is from https://github.com/google/google-authenticator-libpam

Again, this works fine in the three previous OS versions. What do I need to change to make it work here? Advice most appreciated, please. Thank you!

Up vote post of napitravi3 Down vote post of napitravi3
685 views
Posted by
napitravi3
Reply
Add a Comment

Accepted Reply

Library validation is a feature that, when it’s enabled, means that a process will only load code signed by Apple as part of the OS or signed by their team. Library validation is an opt-in feature for third-party developers but it’s generally enabled for all programs that are built into the system [1].

This obviously causes problems if the system program has to load a third-party plug-in. In that case we sign the system program with an entitlement that explicitly opts out of library validation. For example:

% sudo cp `which sudo` .
% sudo chown quinn:staff ./sudo
% codesign -d --entitlements - ./sudo
Executable=/Users/quinn/sudo
[Dict]
    [Key] com.apple.private.AuthorizationServices
    [Value]
        [Array]
            [String] com.apple.security.sudo
    [Key] com.apple.private.security.clear-library-validation
    [Value]
        [Bool] true

We specifically want sudo to be able to load PAM modules and so it opts out of library validation. The mapping process is a platform binary, but mapped file is not message you’re seeing is non-fatal. The code within sudo catches this error and retries in a way that allows it to load third-party code.

When you add the module to the screensaver service, it ends up being loaded by loginwindow. I’m actually surprised to see that loginwindow also disables library validation:

% codesign -d --entitlements - /System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow
Executable=/System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow
[Dict]
    …
	[Key] com.apple.private.security.clear-library-validation
	[Value]
		[Bool] true
    …

That fact that loginwindow crashes in this case indicates that something else is going wrong. It’s possible that it’s as simple as it not catching the error and retrying in the same way that sudo does, but that’s just speculation on my part.

To learn more I’d like to get a copy of the full crash report. Please post it here, using the instructions in Posting a Crash Report.

Oh, and you wrote:

I am developing a new pam module in Monterey [12.6]

Have you tried this on macOS 13? If not, please do so. If you don’t want to upgrade to macOS 13, you can always spin it up in a VM.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] The exact mechanics of this varies between built-in programs and third-party programs because built-in programs don’t have a Team ID. Hence the message mapping process is a platform binary, but mapped file is not, where platform binary maps to what I’m calling a system program.

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Post marked as solved
Add a Comment

Replies

Library validation is a feature that, when it’s enabled, means that a process will only load code signed by Apple as part of the OS or signed by their team. Library validation is an opt-in feature for third-party developers but it’s generally enabled for all programs that are built into the system [1].

This obviously causes problems if the system program has to load a third-party plug-in. In that case we sign the system program with an entitlement that explicitly opts out of library validation. For example:

% sudo cp `which sudo` .
% sudo chown quinn:staff ./sudo
% codesign -d --entitlements - ./sudo
Executable=/Users/quinn/sudo
[Dict]
    [Key] com.apple.private.AuthorizationServices
    [Value]
        [Array]
            [String] com.apple.security.sudo
    [Key] com.apple.private.security.clear-library-validation
    [Value]
        [Bool] true

We specifically want sudo to be able to load PAM modules and so it opts out of library validation. The mapping process is a platform binary, but mapped file is not message you’re seeing is non-fatal. The code within sudo catches this error and retries in a way that allows it to load third-party code.

When you add the module to the screensaver service, it ends up being loaded by loginwindow. I’m actually surprised to see that loginwindow also disables library validation:

% codesign -d --entitlements - /System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow
Executable=/System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow
[Dict]
    …
	[Key] com.apple.private.security.clear-library-validation
	[Value]
		[Bool] true
    …

That fact that loginwindow crashes in this case indicates that something else is going wrong. It’s possible that it’s as simple as it not catching the error and retrying in the same way that sudo does, but that’s just speculation on my part.

To learn more I’d like to get a copy of the full crash report. Please post it here, using the instructions in Posting a Crash Report.

Oh, and you wrote:

I am developing a new pam module in Monterey [12.6]

Have you tried this on macOS 13? If not, please do so. If you don’t want to upgrade to macOS 13, you can always spin it up in a VM.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] The exact mechanics of this varies between built-in programs and third-party programs because built-in programs don’t have a Team ID. Hence the message mapping process is a platform binary, but mapped file is not, where platform binary maps to what I’m calling a system program.

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Post marked as solved
Add a Comment

After going through the crash report I have found out that error message mapping process is a platform binary, but mapped file is not nothing to do with error [redirect to logon screen from sleep screen after segment fault] I was facing when I am trying pam_google_authenticator.so on /etc/pam.d/screensaver.

In fact the error was because the pam conversation function I was using to show user to get input. Here is the piece of code

int converse(pam_handle_t *pamh, int nargs,
                    PAM_CONST struct pam_message **message,
                    struct pam_response **response) {
    struct pam_conv *conv;
    int retval = pam_get_item(pamh, PAM_CONV, (void *)&conv);
    if (retval != PAM_SUCCESS) {
       log_error("converse error");
        return retval;
    }
    return conv->conv(nargs, message, response, conv->appdata_ptr);
}

below line throws the segment fault

return conv->conv(nargs, message, response, conv->appdata_ptr);

function converse is called by below function

void conv_error(pam_handle_t *pamh,  char* text) {
    PAM_CONST struct pam_message msg = {
            .msg_style = PAM_ERROR_MSG,
            .msg       = text,
    };
    PAM_CONST struct pam_message *msgs = &msg;
    struct pam_response *resp = NULL;
    const int retval = converse(pamh, 1, &msgs, &resp);
    if (retval != PAM_SUCCESS) {
        log_message(LOG_ERR, pamh, "Failed to inform user of error");
    }
    free(resp);
}

Can you guide me if this is the correct way to interact with user on loginWindow? If yes, where I am doing the mistake and if not what is the correct way to interact with user on login window. I have also attached the crash report as you suggested.

Below line no are used in crash report so for your connivance I typed them here. Utilities.c line16 is

int retval = pam_get_item(pamh, PAM_CONV, (void *)&conv);

Utilities.c line 21 is

return conv->conv(nargs, message, response, conv->appdata_ptr);

crash report
Posted by
napitravi3
Up vote reply of napitravi3 Down vote reply of napitravi3
Add a Comment

Your crash report shows that stuff is being inlined, which makes it harder to understand. If you can disable optimisations in the build you’re testing, that’d be grand.

The crash report also shows that you’re crashing because you’ve dereferenced NULL. It’s hard to say exactly which pointer is NULL because of the above. It might be more obvious once you disable inlining.

Can you guide me if this is the correct way to interact with user on loginWindow?

Probably not. macOS uses PAM internally, and it also makes sense when authenticating in a command-line setups, but GUI authentication is done by an entirely different technology, authorisation plug-ins. And if you want this to run during screen unlock, your UI will have to be presented by an SFAuthorizationPluginView subclass.

I’m not sure PAM_CONV does in the screen unlock context but I doubt it’ll do anything useful.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Add a Comment