Metrickit opt in rate and actual crash numbers dont match

@edorphy Don't MetricKit depends from this option? Could you give some link about that, pls? I can't find information about it.

@edorphy @dkvijay @andreikrn Anybody figured this out yet? Does MetricKit require user permission - Share with App Developers ?

@andreikrn  @atul3189 @dkvijay

The conversation I had with a MetricKit engineer during WWDC20 or WWDC21 labs was that the user setting to "Share with App Developers" is not used in MetricKit when deciding to generate a metric or diagnostic payload. If you read the footer carefully, "Share with App Developers" is an Apple thing, Apple is sharing it with developers. MetricKit allows YOU, the developer, directly receive the data and then handle it responsibly. I have been operating under this assumption for years. I just wrote up a feedback requesting a docs update for this. Fingers crossed.

FB13618353 - MetricKit / Developer Documentation: Add description to the behavior of MetricKit with respect to "Share with App Developers" setting

Before operating under this assumption, please consider giving your users the option to control data collection. #privacy-by-design

So now that you've thought about letting users choose to participate in this data collection within a feature or setting in your app, it is easy to validate the assumption that MetricKit is not tied to "Share with app developers".

1. Turn off the "Share with App Developers" setting
2. Create a button that crashes your app.
3. Generate logs or save the payload in your subscriber didReceive callback
4. Install this app on a device via Xcode or TestFlight
5. Ensure you are NOT debugging, as the crash will be caught by Xcode
6. Tap the crash button
7. Relaunch the app, your app SHOULD get a callback
8. Verify that your log is printed or the payload is persisted via your process

An alternate path if you don't log or save the MXDiagnosticPayload would be:

1. Set a breakpoint on your didReceive diagnostics method
2. Attach the debugger (run app) on your device with Xcode after the app is crashed
3. Verify during app launch that you didReceive diagnostic callback is invoked.

I just verified this with iOS 17.3.x in one of my apps using the alternate path and Xcode 15.2.

NOTE: For all of this to work, you should be setting the .addSubscriber method somewhere during app startup--my favorite spot is didFinishLaunching. Even for a SwiftUI application, I still have a UIApplicationDelegateAdaptor and in that implementation I put it in didFinishLaunching.

Now a note on privacy and MetricKit data collection.

Because the user can't control my app's MetricKit payload collection via the global setting "Share with app developers", I have explicitly put reference to MetricKit metrics and diagnostics in my privacy statement. Furthermore I have given users the option to enable/disable directly within my app. With that having been said, I do make a decision that, if you use a TestFlight build of my app for beta testing, both diagnostics and metrics will be recorded by default to help improve my app--that is after all the whole purpose of TestFlight right?

P.S. Don't forget to update your App Privacy Label with respect to this data collection too!

I conditionally detect installed from TestFlight and add a disclaimer to the very top of my Privacy Policy.

If the user did in fact install a TestFlight build, they will see this in the settings to enable/disable collection. Again, note the disclaimer.

My apps don't always crash, but when they do MetricKit is there for me to records it. As an added perk, MetricKit can capture events that other off-the-shelf crash reporters just can't do, so I use MetricKit exclusively to build the most privacy preserving crash reporting solution I can.