Detect issues like logic failures, UI problems, and performance regressions by running tests on your app.

Posts under Testing tag

108 Posts
Sort by:
Post not yet marked as solved
0 Replies
423 Views
Hi there, I'm trying to figure out an issue I'm seeing where the xcresult bundle isn't including the crash report after building and running with xcodebuild. I have a simple test app that has a single button that intentionally crashes and an XCUI test to run it. The UI Test also passes which can probably be explained by the lack of any asserts after clicking the button, but I assumed the app crashing would fail the test. The app itself is a Mac OS app with a single button called "Crash" that links to a method: - (IBAction)crash:(id)sender { @throw NSInternalInconsistencyException; } and a test method: XCUIApplication *app = [[XCUIApplication alloc] init]; [app launch]; XCUIElement *crashButton = [[XCUIApplication alloc] init].windows[@"Window"].buttons[@"Crash"]; XCTAssert([crashButton exists]); [crashButton click]; There is an ips file corresponding to the crash in ~/Library/Logs/DiagnosticReports so the crash gets written, but the test passes. After using xcresulttool to dump the Diagnostics folder from the xcresult bundle, I see these lines in the session log: 10:59:20.790 xcodebuild[23184:6634391] Handling Crash: MacAppTest (23189) main. libsystem_c.dylib: abort() called 10:59:20.792 MacAppTestUITests-Runner[23188:6635181] Process crashed with pid:23189 path:/Users/USER/Library/Developer/Xcode/DerivedData/MacAppTest-dxrjhasbuvyffhenipmdemzdpdht/Build/Products/Debug/MacAppTest.app/Contents/MacOS/MacAppTest bundleID:com.company.MacAppTest summary:main 10:59:20.792 MacAppTestUITests-Runner[23188:6635181] Ignoring crash for com.company.MacAppTest:23189 because we are not currently tracking it 10:59:21.371 MacAppTestUITests-Runner[23188:6634605] Returning result XCTWaiterResultCompleted for waiter <XCTWaiter: 0x6000025fd4d0> 10:59:21.371 MacAppTestUITests-Runner[23188:6634605] Wait <XCTWaiterWait: 0x600003e1b540> completed after 1.003s 10:59:21.371 MacAppTestUITests-Runner[23188:6634605] cleaning up delay The Ignoring crash for com.company.MacAppTest:23189 because we are not currently tracking it is what I suspect is the issue. How can I get the UITest to track the app so that when it crashes, the test fails? Some other lines from the logarchive file that may be helpful: 2024-02-14 10:59:20.612346 -0800 osanalyticshelper xpc log creation type 309 result success: /Users/<REDACTED>/Library/Logs/DiagnosticReports/MacAppTest-2024-02-14-105920.ips 2024-02-14 10:59:20.613030 -0800 testmanagerd Calling new file handler <__NSMallocBlock__: 0x6000031761f0> for MacAppTest-2024-02-14-105920.ips 2024-02-14 10:59:20.614240 -0800 ReportCrash client log create type 309 result success: /Users/<REDACTED>/Library/Logs/DiagnosticReports/MacAppTest-2024-02-14-105920.ips 2024-02-14 10:59:20.615417 -0800 testmanagerd Read crash report from /Users/<REDACTED>/Library/Logs/DiagnosticReports/MacAppTest-2024-02-14-105920.ips, got 17327 bytes of data Thanks!
Posted
by
Post not yet marked as solved
4 Replies
394 Views
Hi Team, I am creating an application which is based on tele communications testing where I need to retrieve the SIP logs from the device. How can we achieve? Any approach apple team suggest which will be highly appreciated. Thank You, Vijay
Posted
by
Post not yet marked as solved
1 Replies
339 Views
Hi all, I create apps for Apple and I've been encountering the issue that I cannot upload 4 or 5 apps in a day because they are rejected. It only allows me to upload one a day. Has anyone else encountered this issue/limit? Any advice or ways to resolve this issue? I have multiple apps to complete and upload but this issue has been delaying the process. Thanks in advance.
Posted
by
Post not yet marked as solved
0 Replies
309 Views
I have an app that I built using Expo and its' only external internet-based dependency is the Google Places API, which it uses when the app is loaded and when settings are tweaked. The app requests use of location services then proceeds to use it in an axios query to fetch some bars nearby. I am able to run it without issues on my iPhone and on Android emulators using Expo's local deployment options. Any idea what might be going on here on the testers' end?
Posted
by
Post not yet marked as solved
1 Replies
320 Views
We are facing an issue in IOS version 16.3.1 where the animation based loader is not getting visible, but .svg based loader is working fine and the same animation based loader is working fine in android and windows, and all the remaining versions of IOS as well The html code : <div class="showbox overlayhideclass" id="overlayhideapple" style="display:none;opacity:2.9;"> <div class="loader"> <svg class="circular darkLoader" style="display:none;" viewBox="25 25 50 50"> <!-- Background ring --> <circle class="background-ring" cx="50" cy="50" r="22"></circle> <!-- Loading circle --> <circle class="path" cx="50" cy="50" r="22" fill="none" stroke-width="6" stroke-miterlimit="10"></circle> </svg> <svg class="circular lightLoader" style="" viewBox="25 25 50 50"> <!-- Background ring --> <circle class="background-ring" cx="50" cy="50" r="22"></circle> <!-- Loading circle --> <circle class="path" cx="50" cy="50" r="22" fill="none" stroke-width="6" stroke-miterlimit="10"></circle> </svg> </div> </div> classes: .loader { position: relative; margin: 0 auto; width: 100px; &:before { content: ""; display: block; padding-top: 100%; } } .circular { animation: rotate 1s linear infinite; height: 100%; transform-origin: center center; width: 74.84%; position: absolute; top: 0; bottom: 0; left: 0; right: 0; margin: auto; border-radius: 50%; } .background-ring { fill: none; stroke: rgb(227, 226, 231); stroke-width: 6; } .path { stroke-dasharray: 1, 200; stroke-dashoffset: 0; animation: dash 1.5s ease-in-out infinite, color 6s ease-in-out infinite; stroke-linecap: round; } @keyframes rotate { 100% { transform: rotate(360deg); -webkit-transform: rotate(360deg); } } @keyframes dash { 0% { stroke-dasharray: 100, 200; stroke-dashoffset: -100px; } 30% { stroke-dasharray: 100, 200; stroke-dashoffset: -100px; } 100% { stroke-dasharray: 100, 200; stroke-dashoffset: -100px; } } @keyframes color { 100%, 0% { stroke:var(--loader-icon); } 40% { stroke:var(--loader-icon); } 66% { stroke:var(--loader-icon); } 80%, 90% { stroke:var(--loader-icon); } } body { background-color: var(--white); } .showbox { position: absolute; top: 0; bottom: 0; left: 0; right: 0; width: 100vw; height: 100vh; display: flex; align-items: center; } .overlayhideclass { position: fixed; width: 100%; height: 100%; top: 0; left: 0; text-align: center; opacity: 2.9; background-color: Var(--pg-bg-primary); z-index: 9999; display: flex; justify-content: center; justify-items: center; justify-self: center; overflow:-y:hidden; } .darkLoader { z-index: 100; display: flex; justify-content: center; align-items: center; } .lightLoader{ z-index: 100; display: flex; justify-content: center; align-items: center; }
Posted
by
Post not yet marked as solved
0 Replies
336 Views
I'm currently developing hybrid MacOS app that display some content in WebKit. I wanted to write some UI tests using Appium for it. For this I'm currently using Mac2 driver https://github.com/appium/appium-mac2-driver The problem that I faced is specifying identifiers for HTML elements to easily find them later in tests. It seems Appium uses Accessibility framework to get list of elements. Also it seems that accessibility framework parses HTML and wrap them to XCUIElements in XML Source (I can see this in Appium inspector, check the screenshot) As you can see Appium tries to get accessibility identifier/label from these elements. While searching in web I found, that if we specify aria-label="something" property on HTML element, it will be parsed by accessibility framework as label. So we can use labels for locating elements. However in ideal scenario we should have some HTML property that will be parsed to identifier, as accessibility label is used for accessibility needs, while identifier is allocated for UI testing. So my question is there HTML tag that can be used to specify identifier, and if not, is there plans to add support for it?
Posted
by
Post not yet marked as solved
1 Replies
324 Views
For external testing through testflight, there is app review process. Link: https://developer.apple.com/testflight/ Planning to submit an app for external testing. However, the backend of the App is an test environment. Is that ok for the **external testing app review process **. The test environment is behind IP restriction. a. Can we provide VPN credentials to apple team for app access and complete the **external testing app review process ** OR b. Does apple team have any IP addresses, that can be add it to our testing environment.
Posted
by
Post not yet marked as solved
1 Replies
289 Views
Hi All, Almost at my wits end. I have owned a 2020 Imac now for 2 years . I bought it refurbed and two months in it blew a logic board which I had replaced under warranty. So after that everything worked fine until I updated to OS Sonoma 14.2 after that crashes and kernel panics. I have gone through Mac diagnosis and it says my hardware is fine. Also, all 3rd part plug ins validate in AUVAL. I never experienced anything so frustrating in 15 years of using Logic X, My old power PC never had issues. It happens on any project large or small. I will paste in some truncated crash reports and attach full ones plus ETRA report. Can anyone get to the bottom of this? I brought Imac to the shop and they said everything checked out. I spent $4000 on this POS and upgraded to 64 GBS memory. I am ready to throw it out the window and start over. ANY HELP IS APPRECIATED
Posted
by
Post not yet marked as solved
0 Replies
401 Views
In my iOS project, there is an infrequent crash related to virtual memory problem. Therefore, I plan to use UITest in combination with Product/Perform Action/Profile "TestCaseName" to conduct Game Performance-type testing. This allows the automatic testing to continuously operate until the profile stops recording upon a crash. This enables me to observe the various states of the program at the time of the crash. However, I have found that the UITest using Profile is highly unstable. The UITestCase often terminates unexpectedly during execution, leading to failed tests (Instruments is still working). Sometimes, the app is terminated immediately after startup. It seems that the use of sleep() in the code can easily cause interruption issues, which do not occur during normal UI testing. I am wondering if anyone has experience using Profile for UITest and whether they have encountered the issues I described." Working Environment: XCode14.3.1, iPhone Device iOS17.2
Posted
by
Post not yet marked as solved
1 Replies
438 Views
I am currently working on a SwiftUI project and I am trying to write a UI test for a DatePicker. However, I am having trouble interacting with the DatePicker in the test. Here is the code for the DatePicker in my SwiftUI view: DatePicker("Date", selection: $date, displayedComponents: [.date]) .datePickerStyle(.graphical) .labelsHidden() .frame(width: 150) And here is the code for my UI test: func testSelectingDateChangesMainView() throws { let app = XCUIApplication() app.launch() let dateDatePicker: XCUIElement = app.windows["mainUI-AppWindow-1"].datePickers["Date"] dateDatePicker.click() let col = dateDatePicker.descendants(matching: .any) print("\(dateDatePicker.debugDescription), \(col.debugDescription)") } When I run the test, it seems like the DatePicker does not have any descendants, so I am unable to select a date. Does anyone have any suggestions on how I can interact with a .graphical style DatePicker in a UI test? Any help would be greatly appreciated. Thank you, OnlyForF1
Posted
by
Post not yet marked as solved
1 Replies
1.8k Views
I've created a test plan that includes the below configurations. {"configurations": [{ ... "options": { "defaultTestExecutionTimeAllowance" : 60, "maximumTestExecutionTimeAllowance" : 60, "testTimeoutsEnabled" : true }], "defaultOptions": { "defaultTestExecutionTimeAllowance" : 60, "maximumTestExecutionTimeAllowance" : 60, "testTimeoutsEnabled" : true, .... } ... } If I write a test that intentionally will take longer than 60 seconds, executing the test locally fails as expected with the error message Test exceeded execution time allowance of 1 minute let app = XCUIApplication() app.launch() let missingButton = app.buttons.firstMatch XCTAssertTrue(missingButton.waitForExistence(timeout: 120), "When a timeout is > 60 seconds, the default max setting of 60 seconds is still enforced") However, when this test is run using this test plan in XCode Cloud, it reports that it took 124 seconds to run and there is no message about exceeding execution time allowance. How can the test step be configured in XCode Cloud to ensure that the execution time allowances are respected?
Posted
by
Post not yet marked as solved
0 Replies
639 Views
Hi! I'm trying to run my Flutter code natively on my iPhone, but whenever I try to run it, I keep get the following error message: Error (Xcode): Cycle inside Runner; building could produce unreliable results. Cycle details: → Target 'Runner': ExtractAppIntentsMetadata ○ Target 'Runner' has copy command from '/Users/matthewlee/flutterflow/AppName/build/ios/Debug-iphoneos/ImageNotification.appex' to '/Users/matthewlee/flutterflow/AppName/build/ios/Debug-iphoneos/Runner.app/PlugIns/ImageNotification.appex' ○ Target 'Runner' has compile command with input '/Users/matthewlee/flutterflow/AppName/ios/Runner/Base.lproj/LaunchScreen.storyboard' ○ That command depends on command in Target 'Runner': script phase “Thin Binary” ○ Target 'Runner' has process command with output '/Users/matthewlee/flutterflow/AppName/build/ios/Debug-iphoneos/Runner.app/Info.plist' I've been trying to rearrange my Build Phases in Xcode, but nothing has seemed to work yet. How do I fix this issue preventing my app from running?
Posted
by
Post not yet marked as solved
0 Replies
406 Views
I have an installer downloaded from the App Store a few weeks ago for macOS Sonoma 14.1.2 (I've made copies for testing purposes), but today I have been unable to install it on a volume - an experiment I had been conducting repeatedly last week. Is Apple blocking 14.1.2 from being installed now? (Sonoma 14.2 is the version currently available from the App Store) Have I reached some kind of limit on the number of times I can install it? Any other ideas? Here is the alert: Note: the problem I am testing is an issue with an app installed from TestFlight (that contained a network system extension) that killed networking when the OS was updated from Sonoma 14.1.2 to Sonoma 14.2. (related post)
Posted
by
Post not yet marked as solved
0 Replies
429 Views
Hello everyone! I have a question (issue). I need to write a test for an app, and everything is okay with my tests, but I need to test login with an Apple ID, and I don't want to use my private Apple ID because I plan to post this test on GitHub. Is there any possibility to create, for example, a dummy Apple ID account? Or how do developers typically handle this situation?
Posted
by
Post not yet marked as solved
0 Replies
334 Views
I'm starting to set up a testing environment for macOS applications, and I want to be able to rollback to different versions of the OS (all machines can be scrubbed, so I don't worry about preserving user data). I noticed that I can download the latest versions of Ventura and Sonoma from the Mac App Store, but not past versions of Ventura and Sonoma Is there a way to download earlier versions of these macOSes (or maybe even earlier ones like Monterey)? Alternatively, if I regularly start downloading macOS installers from the Mac App Store and maintain my own local copies so I can use these to rollback to these versions, do I need to download a version for each hardware? For example, I currently have an M1 Mac mini, an M2 Mac mini, and an M1 Mac Studio. Do I need to download different installers for each of these hardware platforms?
Posted
by
Post not yet marked as solved
1 Replies
475 Views
I have noticed that, as of late, the iOS simulator has been incredibly slow. Even simple (and perhaps silly) actions like changing the wallpaper take an unreasonably long time, to the point where it is no longer clear whether the process has hanged. I tried to change the wallpaper to a photo of mine, but it simply would not appear. Even booting up takes several seconds, and home screen animations are laggy. I've given up on testing my build for now. I have tried with an iPhone Xr and an iPhone 15 Pro simulator - both running iOS 17.0. Both are extremely laggy. This is on an M1 MacBook Air running the stable version of macOS Sonoma 14.0. with Xcode version 15.0.1 and Simulator version 15.0.1. I have checked the Activity Monitor for CPU and RAM. There is nothing abnormal. I would appreciate any suggestions.
Posted
by
Post not yet marked as solved
2 Replies
827 Views
Our CI pipeline produces .xcresult bundles that allow developers to diagnose failures by opening them in Xcode. If the test crashes, we used to be able to open the diagnostic folders and inspect the raw test logs. As of Xcode 15, that option is no longer available. I know the Diagnostic data is there because I can view it with Xcode 14. Am I just missing something in the Xcode 15 UI? Or has the capability to browse the Diagnostics actually been removed?
Posted
by
Post not yet marked as solved
3 Replies
977 Views
I'm trying to write performance tests with Xcode 15, but it's not working: even though setting a baseline seems to work (baseline data files are created), Xcode seems to ignore that baseline altogether whenever the test is run. As a result, performance tests never fail, no matter how bad performance gets. The source editor persistently displays a “No baseline average for Time” message. Has anyone else encountered this? Figured out how to sidestep the issue? Of note, the issue is easily reproduced: create a new (macOS in my case) app project with tests, add a dummy performance test; run the test once and save its baseline, and despite this subsequent tests will always succeed, even if you significantly slow the dummy test down. (I've filed this as FB13330139)
Posted
by
Post not yet marked as solved
0 Replies
564 Views
I'm encountering an intermittent issue while trying to run safaridriver on macOS Sonoma. Here are the details of the problem: I ran sudo /usr/bin/safaridriver -p0 --enable in the beginning. After that when I run this multiple times /usr/bin/safaridriver -p0, I sometimes receive an "Operation not permitted" error, but not consistently. This issue seems to occur intermittently. I've checked the sudo logs, and I see the following error message: kernel: (Sandbox) Sandbox: com.apple.WebDriver.HTTPService(2049) deny(1) user-preference-write com.apple.WebDriver.HTTPService cfprefsd: (CoreFoundation) [com.apple.defaults:cfprefsd] rejecting write of key(s) MobileDeviceRemoteXPCEnabled in { com.apple.WebDriver.HTTPService, nimish, kCFPreferencesAnyHost, /Users/nimish/Library/Preferences/com.apple.WebDriver.HTTPService.plist, managed: 0 } from process 2049 (com.apple.WebDriver.HTTPService) because setting these preferences requires user-preference-write or file-write-data sandbox access cfprefsd: (CoreFoundation) [com.apple.defaults:cfprefsd] Couldn't open parent path due to [2: No such file or directory kernel: (Sandbox) Sandbox: com.apple.WebDriver.HTTPService(2049) deny(1) network-bind local:*:7055 com.apple.WebDriver.HTTPService: (WebDriver) [com.apple.WebDriver:WebService] Error starting HTTP server listening on localhost:0: Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted" UserInfo={NSLocalizedDescription=Operation not permitted, NSLocalizedFailureReason=Error in bind() function} com.apple.WebDriver.HTTPService: (WebDriver) [com.apple.WebDriver:XPCService] Client connection invalidated for some reason I am getting this on Mac Os Sonoma Can anybody please help on this?
Posted
by
Post not yet marked as solved
1 Replies
371 Views
I have an animation app that runs fine on all iPhones I've tested but one. Menus show up differently, Tool bars appear differently. And worst of all, the image skews when painting with the fill tool. How can I find out what makes these things different on one iPhone than on others? The phone in question is an iPhone 13 Pro with plenty of memory available.
Posted
by