Large files in PKGs woes

Question

Dear colleagues

No matter how hard I try I can’t get large files ~20Gb to be processed correctly by the productbuild At least that’s what I think.

I use —large-payload option

Product build actively complains with warnings about 12,0 as min version of macOS. The issue here is that in app plist 12.4 is set as min version. PRE-INSTALL REQUIREMENTS PROPERTY LIST should apparently be used for this but no possible combo of options can fix this warning. The warning have far reaching consequences. More on this later
All the utilities even Apple ones like Trandporter unpack the pkg absolutely wrong. When I check temporary expanded archive my 20gb file turns into 8Gb!
i still have to check installer way whether it unpacks correctly.
AppStore submission immediately rejects the pkg because lsmin key is 12.4 while package min is 12.0. The thing is I never ever used 12.0 anywhere.

so to summarize.

Cant submit large payloads archive
Can’t extract it either

what to do ?

Best regards

Developer Tools

585

Posted by

Stauff

Reply

Add a Comment

Answer 1

Ok I figured it out. I still haven't checked the AppStore process but at least all checks pass now It's MUCH more complex than product build, but product build itself is buggy.

So.

Prepare your .app with HUUUGE file (> 8Gb). Sign it correctly, do everything as usual!
pkgbuild --identifier <APP_ID> --large-payload --compression latest --min-os-version 12.4 --root <root_with_app_inside> <APP_ID>.pkg

Make the following 'Distribution' file

<installer-gui-script minSpecVersion="2">
    <pkg-ref id="com.***">
        <bundle-version>
            <bundle CFBundleShortVersionString="1.0.1" CFBundleVersion="23" id="com.***" path="MY.app"/>
        </bundle-version>
    </pkg-ref>
    <product id="com.***" version="1.0.1"/>
    <title>MY</title>
    <options customize="never" require-scripts="false" hostArchitectures="arm64,x86_64"/>
    <volume-check>
        <allowed-os-versions>
            <os-version min="12.4"/>
        </allowed-os-versions>
    </volume-check>
    <choices-outline>
        <line choice="default">
            <line choice="com.***"/>
        </line>
    </choices-outline>
    <choice id="default" title="MY" versStr="1.0.1"/>
    <choice id="com.***" title="MY" visible="false" customLocation="/Applications">
        <pkg-ref id="com.***">com.***.pkg</pkg-ref>
    </choice>
</installer-gui-script>

productbuild --sign "Signer" --distribution ./Distribution --package-path ~ Final.pkg

Something like that

Your package definitely will be 1) compressed fine 2) smaller 3) with super large file

Feed the stuff to the Transporter. Don't forget to adjust Distribution with correct version, id's etc.

Have fun! I certainly had it

Posted by

Stauff

See last post for fully working solution

—
Stauff

Add a Comment

Answer 2

Some more context

the machine I am doing all tests is 12.6 m1 mini

xcode seems to be completely ignorant of large payloads so all is done by shell scripting

the packaging completes successfully albeit with 12,0 warnings.

I used —expand-full to get broken results

I used auto compression

Posted by

Stauff

Add a Comment

Answer 3

Ok I figured it out. I still haven't checked the AppStore process but at least all checks pass now It's MUCH more complex than product build, but product build itself is buggy.

So.

Prepare your .app with HUUUGE file (> 8Gb). Sign it correctly, do everything as usual!
pkgbuild --identifier <APP_ID> --large-payload --compression latest --min-os-version 12.4 --root <root_with_app_inside> <APP_ID>.pkg

Make the following 'Distribution' file

<installer-gui-script minSpecVersion="2">
    <pkg-ref id="com.***">
        <bundle-version>
            <bundle CFBundleShortVersionString="1.0.1" CFBundleVersion="23" id="com.***" path="MY.app"/>
        </bundle-version>
    </pkg-ref>
    <product id="com.***" version="1.0.1"/>
    <title>MY</title>
    <options customize="never" require-scripts="false" hostArchitectures="arm64,x86_64"/>
    <volume-check>
        <allowed-os-versions>
            <os-version min="12.4"/>
        </allowed-os-versions>
    </volume-check>
    <choices-outline>
        <line choice="default">
            <line choice="com.***"/>
        </line>
    </choices-outline>
    <choice id="default" title="MY" versStr="1.0.1"/>
    <choice id="com.***" title="MY" visible="false" customLocation="/Applications">
        <pkg-ref id="com.***">com.***.pkg</pkg-ref>
    </choice>
</installer-gui-script>

productbuild --sign "Signer" --distribution ./Distribution --package-path ~ Final.pkg

Something like that

Your package definitely will be 1) compressed fine 2) smaller 3) with super large file

Feed the stuff to the Transporter. Don't forget to adjust Distribution with correct version, id's etc.

Have fun! I certainly had it

Posted by

Stauff

See last post for fully working solution

—
Stauff

Add a Comment

Answer 4

Ok I finally sorted it all out.

Here's completely working stuff. Proven to be able to be processed by the Transporter. Warning, any other way will fail ;)

Compression that’s not legacy can’t work with Large payload. At all. No matter what tool you use. I used pkgbuild and it also has major issues with large payload and modern compression.

Prepare your .app with HUUUGE file (> 8Gb). Sign it correctly, do everything as usual!
Pkgbuild time

pkgbuild --identifier *** --large-payload --min-os-version 12.4 --root 'MyRoot/Applications' com.XXXX.pkg pkgbuild: Inferring bundle components from contents of Root/Applications pkgbuild: Adding component at ***.app

<?xml version="1.0" encoding="utf-8"?>
<installer-gui-script minSpecVersion="2">
    <pkg-ref id="com.XXXX"/>
    <product id="com.XXXX" version="1.0.1”/>. <—————————— Must be adjusted
    <options customize="never" require-scripts="false" hostArchitectures="x86_64,arm64"/>
    <volume-check>
        <allowed-os-versions>
            <os-version min="12.4.0"/>
        </allowed-os-versions>
    </volume-check>
    <choices-outline>
        <line choice="default">
            <line choice="com.XXXX"/>
        </line>
    </choices-outline>
    <choice id="default"/>
    <choice id="com.XXXX" visible="false" customLocation="/Applications”>. <——— Custom location should be added
        <pkg-ref id="com.XXXX"/>
    </choice>
    <pkg-ref id="com.XXXX" version="0" onConclusion="none">com.XXXX.pkg</pkg-ref>
</installer-gui-script>.

productbuild --sign "Installer" --symbolication Symbols --distribution installer-script --package-path . My.pkg

This produced My.pkg which was accepted by the Transporter.

One last important note As of today it’s impossible to make all this in any XCode as it seems to be unaware of large payloads and productbuild does have a bug with versioning.

Posted by

Stauff

Add a Comment

Large files in PKGs woes

Accepted Reply

Replies