Create and run unit tests, performance tests, and UI tests for your Xcode project using XCTest.

XCTest Documentation

Posts under XCTest tag

155 Posts
Sort by:
Post marked as solved
6 Replies
2.3k Views
When I try to run unit tests in Xcode 14.3, it's often freezing up. Needs to be "Force Quit"ed. This happens most often the first time I try to run a suite through the GUI. Running a single test sometimes freezes, but not always. I can't pause the test execution, and when I try to exit Xcode it asks if I want to cancel the tests, but then it doesn't cancel them and I need to force quit at that point. Is there any know issue with 14.3 that would cause this?
Posted
by
Post not yet marked as solved
7 Replies
9.2k Views
App launches and crashes immediately with error Details Simulator device returned an error for the requested operation. Domain: NSPOSIXErrorDomain Code: 3 Failure Reason: No such process User Info: { DVTErrorCreationDateKey = "2023-04-06 13:10:58 +0000"; IDERunOperationFailingWorker = IDELaunchiPhoneSimulatorLauncher; SimCallingSelector = "launchApplicationWithID:options:pid:error:"; } -- Application launch for 'com.abc.example' did not return a valid pid nor a launch error. Domain: NSPOSIXErrorDomain Code: 3 Failure Reason: No such process -- Analytics Event: com.apple.dt.IDERunOperationWorkerFinished : { "device_model" = "iPhone10,4"; "device_osBuild" = "16.4 (20E247)"; "device_platform" = "com.apple.platform.iphonesimulator"; "launchSession_schemeCommand" = Run; "launchSession_state" = 1; "launchSession_targetArch" = arm64; "operation_duration_ms" = 666; "operation_errorCode" = 3; "operation_errorDomain" = NSPOSIXErrorDomain; "operation_errorWorker" = IDELaunchiPhoneSimulatorLauncher; "operation_name" = IDERunOperationWorkerGroup; "param_consoleMode" = 0; "param_debugger_attachToExtensions" = 0; "param_debugger_attachToXPC" = 1; "param_debugger_type" = 3; "param_destination_isProxy" = 0; "param_destination_platform" = "com.apple.platform.iphonesimulator"; "param_diag_MainThreadChecker_stopOnIssue" = 0; "param_diag_MallocStackLogging_enableDuringAttach" = 0; "param_diag_MallocStackLogging_enableForXPC" = 1; "param_diag_allowLocationSimulation" = 1; "param_diag_checker_tpc_enable" = 1; "param_diag_gpu_frameCapture_enable" = 0; "param_diag_gpu_shaderValidation_enable" = 0; "param_diag_gpu_validation_enable" = 0; "param_diag_memoryGraphOnResourceException" = 0; "param_diag_queueDebugging_enable" = 1; "param_diag_runtimeProfile_generate" = 0; "param_diag_sanitizer_asan_enable" = 0; "param_diag_sanitizer_tsan_enable" = 0; "param_diag_sanitizer_tsan_stopOnIssue" = 0; "param_diag_sanitizer_ubsan_stopOnIssue" = 0; "param_diag_showNonLocalizedStrings" = 0; "param_diag_viewDebugging_enabled" = 1; "param_diag_viewDebugging_insertDylibOnLaunch" = 1; "param_install_style" = 0; "param_launcher_UID" = 2; "param_launcher_allowDeviceSensorReplayData" = 0; "param_launcher_kind" = 0; "param_launcher_style" = 0; "param_launcher_substyle" = 0; "param_runnable_appExtensionHostRunMode" = 0; "param_runnable_productType" = "com.apple.product-type.application"; "param_testing_launchedForTesting" = 0; "param_testing_suppressSimulatorApp" = 0; "param_testing_usingCLI" = 0; "sdk_canonicalName" = "iphonesimulator16.4"; "sdk_osVersion" = "16.4"; "sdk_variant" = iphonesimulator; } -- System Information macOS Version 13.3 (Build 22E252) Xcode 14.3 (21812) (Build 14E222b) Timestamp: 2023-04-06T09:10:58-04:00
Posted
by
Post not yet marked as solved
1 Replies
952 Views
I am unable to run Unit Test targets for any project on my xcode 14.1 with iOS Simulator version 13.7 Assertions: System: Failed to install or launch the test runner. Underlying Error: Process spawn via launchd failed. Bad file descriptor. (Underlying Error: The operation couldn’t be completed. (Mach error 9 - (os/kern) memory failure)) Details Process spawn via launchd failed. Domain: NSPOSIXErrorDomain Code: 9 Failure Reason: Bad file descriptor User Info: { DVTErrorCreationDateKey = "2023-04-26 21:17:18 +0000"; IDERunOperationFailingWorker = IDELaunchiPhoneSimulatorLauncher; Session = "com.apple.CoreSimulator.SimDevice.32CA512F-AF12-4E75-B34C-D43C9AB75CDE"; } -- The operation couldn’t be completed. (Mach error 9 - (os/kern) memory failure) Domain: NSMachErrorDomain Code: 9 -- Analytics Event: com.apple.dt.IDERunOperationWorkerFinished : { "device_model" = "iPhone12,5"; "device_osBuild" = "13.7 (17H22)"; "device_platform" = "com.apple.platform.iphonesimulator"; "launchSession_schemeCommand" = Test; "launchSession_state" = 1; "launchSession_targetArch" = "x86_64"; "operation_duration_ms" = 59; "operation_errorCode" = 9; "operation_errorDomain" = NSPOSIXErrorDomain; "operation_errorWorker" = IDELaunchiPhoneSimulatorLauncher; "operation_name" = IDERunOperationWorkerGroup; "param_consoleMode" = 0; "param_debugger_attachToExtensions" = 0; "param_debugger_attachToXPC" = 1; "param_debugger_type" = 3; "param_destination_isProxy" = 0; "param_destination_platform" = "com.apple.platform.iphonesimulator"; "param_diag_MainThreadChecker_stopOnIssue" = 0; "param_diag_MallocStackLogging_enableDuringAttach" = 0; "param_diag_MallocStackLogging_enableForXPC" = 0; "param_diag_allowLocationSimulation" = 1; "param_diag_checker_tpc_enable" = 0; "param_diag_gpu_frameCapture_enable" = 3; "param_diag_gpu_shaderValidation_enable" = 0; "param_diag_gpu_validation_enable" = 1; "param_diag_memoryGraphOnResourceException" = 0; "param_diag_queueDebugging_enable" = 1; "param_diag_runtimeProfile_generate" = 0; "param_diag_sanitizer_asan_enable" = 0; "param_diag_sanitizer_tsan_enable" = 0; "param_diag_sanitizer_tsan_stopOnIssue" = 0; "param_diag_sanitizer_ubsan_stopOnIssue" = 0; "param_diag_showNonLocalizedStrings" = 0; "param_diag_viewDebugging_enabled" = 1; "param_diag_viewDebugging_insertDylibOnLaunch" = 0; "param_install_style" = 0; "param_launcher_UID" = 2; "param_launcher_allowDeviceSensorReplayData" = 0; "param_launcher_kind" = 0; "param_launcher_style" = 0; "param_launcher_substyle" = 0; "param_runnable_appExtensionHostRunMode" = 0; "param_runnable_productType" = "com.apple.product-type.tool"; "param_runnable_type" = 1; "param_testing_launchedForTesting" = 1; "param_testing_suppressSimulatorApp" = 0; "param_testing_usingCLI" = 0; "sdk_canonicalName" = "iphonesimulator16.1"; "sdk_osVersion" = "16.1"; "sdk_variant" = iphonesimulator; } -- System Information macOS Version 13.3.1 (Build 22E261) Xcode 14.1 (21534.1) (Build 14B47b) Timestamp: 2023-04-26T23:17:18+02:00
Posted
by
Post not yet marked as solved
5 Replies
3.8k Views
Issue: When I click run unit tests the tests get 'stuck' and in the top bar is says "Testing..." and never completes - I even left it for an hour. I have created new projects with no code written and the same thing happens. I have deleted and re-downloaded code as well. Current Xcode: 14.3 I don't believe it's a project issue since I have tried multiple but rather something with Xcode. and my CPU % for Xcode shoots to 365+ when this is happening. I have a M1 Max with 64GB. Any help would be appreciated.
Posted
by
Post not yet marked as solved
2 Replies
1.1k Views
My tests fail with "Restarting after unexpected exit, crash, or test timeout". This is what I have so far established: Does only happen when building with xcodebuild (14.3) but not when building with Xcode (14.3) Tests that are failing are in a .testTarget within a local swift package and part of a test plan together with tests that are in subprojects there are no crashlogs, so likely these tests are not crashing with Xcode 14.2 there are no problems whatsoever (xcodebuild or Xcode) Any ideas on how to debug this any further? Best Roddi
Posted
by
Post marked as solved
1 Replies
966 Views
Hello, I've started using URLSessionWebSocketTask to perform very simple socket operations. To avoid needing an external connection while testing I've opted to create a URLProtocol (which I've used previously for Data tasks), but it's unclear if this is the direction I should be taking. Within the startLoading() method I'm properly creating the hand-shake response according to RFC6455 (previously I was getting a Bad Response error). override open func startLoading() { let requestKey = request.allHTTPHeaderFields!["Sec-WebSocket-Key"]! let sha1HexString = "\(requestKey)258EAFA5-E914-47DA-95CA-C5AB0DC85B11".sha1() let acceptHeaderValue = Data( fromHexEncodedString: sha1HexString )!.base64EncodedString() let response = HTTPURLResponse( url: url, statusCode: 101, httpVersion: "http/1.1", headerFields: [ "Sec-WebSocket-Accept": acceptHeaderValue, "upgrade": "websocket", "connection": "upgrade" ])! self.client.urlProtocol( self, didReceive: response, cacheStoragePolicy: .notAllowed ) /* `data` is generated elsewhere and doesn't seem relevant*/ self.client.urlProtocol(self, didLoad: data) self.client.urlProtocolDidFinishLoading(self) } I've changed the order of calling didReceive: and didLoad, but everything results in the following two errors in my logs: Connection not set before response is received, failing task Task . finished with error [-1005] Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost." UserInfo={NSErrorFailingURLStringKey=ws://localhost:8080, NSErrorFailingURLKey=ws://localhost:8080, _NSURLErrorRelatedURLSessionTaskErrorKey=("LocalWebSocketTask ."), _NSURLErrorFailingURLSessionTaskErrorKey=LocalWebSocketTask ., NSLocalizedDescription=The network connection was lost.} I've set breakpoints within startLoading, so I know it's using my custom URLProtocol, but I can't seem to figure out how to make it work. Is it possible to use URLProtocol with sockets, or is there an alternative approach to mocking WebSockets? I want to avoid wrapping everything within layers of protocols, but completely replacing URLSession seems like the only course forward. Thank you!
Posted
by
Post not yet marked as solved
2 Replies
1.8k Views
Hello everyone, I am currently building a swift library that has a technical limitation of running tests in iOS Simulator with Rosetta only (since it depends on a library that can only, for now, build it for iOS Simulator x86_64). Currently, I can run this swift library tests in XCode with an iPhone Rosetta target, but I want to understand how to run these tests from the command line. I am not an expert in xcodebuild tool, so if anyone can help me, I would appreciate a lot. I have tried, without success, running: xcodebuild -scheme MyLibrary -destination 'platform=iOS Simulator,name=iPhone 14' -derivedDataPath build -verbose EXCLUDED_ARCHS=arm64 VALID_ARCHS=x86_64 ONLY_ACTIVE_ARCH=NO test In which I get an error similar to: xctest (17305) encountered an error (Failed to load the test bundle. If you believe this error represents a bug, please attach the result bundle at [...] MyLibraryTests' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')))) I suspect it's not running xctest with arch -x86_64. So I have also tried: arch -x86_64 xcodebuild -scheme MyLibrary -destination 'platform=iOS Simulator,name=iPhone 14' -derivedDataPath build -verbose EXCLUDED_ARCHS=arm64 VALID_ARCHS=x86_64 ONLY_ACTIVE_ARCH=NO test But result is the same. I want to understand what is the command that XCode runs the tests in iOS Simulator with Rosetta. Other IDEs prints the command line used, but I could not find any setting that would tell XCode to print it. Any help appreciated. Thanks in advance, --Thiago
Posted
by
Post not yet marked as solved
2 Replies
647 Views
We have four build agents. 2 x Intel, 8GB RAM 1 x M1, 8 GB RAM 1 x M2, 8GB RAM The only differences in the configurations are the processor types. On the Intel systems things work fine. On the M1/M2 systems we see the "Force Quit Applications" dialog, with the message: "Your system has run out of application memory." None of the processes it offers to quit are big, but checking Activity Monitor shows testmanagerd using 9+ GB (yes GIGABYTES) of Memory. It show "Real Memory" of about 1GB. Anyone have an idea of what's going on here? All of the systems are running the most current Ventura build, and Xcode 14.3
Posted
by
Post not yet marked as solved
1 Replies
690 Views
Hi! Xcode 14.3, Swift project, macOS: 13.3.1 (a) I've written a few unit tests for my project after following online instructions on how to add them to existing XCode project. When I run the tests (both as a whole and individually), the tests build fine but never complete. Only when I hit the stop button do I get an output report that says everything OK. I've put knowingly false assertions into my tests so they should fail if they are properly loaded. How do I go about troubleshooting this problem? Thanks, Bobby
Posted
by
Post not yet marked as solved
0 Replies
1k Views
I have an old project Objective-C iOS app I've been updating to Swift. It used to use Carthage, but now uses only SPM for dependencies. One such dependency is TrueTime (which was updated to support SPM by that author). It has two products, TrueTime, a Swift library, and CTrueTime a header-only C library with some packed, byte-aligned structs used when parsing NTP responses. I’m adding unit tests to my project, and when I import the app module with @testable import MyApp, the build fails on that line with @testable import MyApp ^ <unknown>:0: error: missing required module 'CTrueTime' The app itself builds and runs just fine. If I don’t import the app, then I have to expressly add code I want to test to the test target, which is not the right way to do these things. The other dependencies I have all work fine, but none have a C library product. I get a warning about implicitly including a bridging header, but I don’t think that has anything to do with this. Update: I also just tried adding TrueTime to a modern, pure SwiftUI iOS app, and I get the same issue. Is there something more I need to add to my test target config? (I've also posted this to stack overflow https://stackoverflow.com/questions/76284838/when-importing-app-with-testable-get-missing-required-module-ctruetime-usi)
Posted
by
Post not yet marked as solved
1 Replies
639 Views
I wish to switch to and from app switcher while automation for my test case to check the visibility of the content in my app when it's in app switcher stack. While I'm searching I found an option XCUIDevice Button press to go to home screen. XCUIDevice.shared.press(.home) sleep(10) XCUIApplication().launch() Although I can't view my app in app switcher stack, hence can't fulfil my test case. Also, I tried to do some work around like enabling Assistive Touch and try to use the App Switcher option from there. But I can't find the assistive touch element to utilise it. Is there any other way I need to try? 🔎 Thanks in advance...
Posted
by
Post not yet marked as solved
2 Replies
763 Views
I am writing the Unit test cases for the App Extension. The App Extension target is a combination of Swift Code + Objective Code. I just followed the below steps after creating the unit test target for the schema (The app target to be tested)which points out to the App Extension target in the XCode. Select you extension test target in Project settings Go to Build Phases Expand Compile Sources Click + Add source files with your testable code. Build for Testing When I try to build the unit test target from the XCode. I am seeing the below error Cannot find interface declaration for 'XCTestCase', superclass of 'TunnelTests' Please provide help to overcome this Blocker.
Posted
by
Post not yet marked as solved
0 Replies
749 Views
Hi all, I have all latest configurations on my mac with - macOS - Ventura 13.4 xCode - 14.3.1 iOS real device - 16.4.1 Error - Cannot link directly with dylib/framework, your binary is not an allowed client of /Applications/Xcode.app/Contents/Developer/Platforms/ iPhoneOS.platform/Developer/Library/PrivateFrameworks/XCTAutomationSupport.framework/XCTAutomationSupport for architecture armé64 Linker command failed with exit code 1 (use -v to see invocation) RCAs tried - build settings : architecture only : NO (initial yes) Corrected framework search path in build settings Added XCTest.Framework under WebDriverAgentRunner (was added in webdriveragentLib) - couldn’t find XCTAutomationSupport.framework under Link Binary With Libraries in WebDriverAgentRunner to add that as well Tried downloading the latest webdriver agent 4.9 but that too didn't work. also, upon checking the versions of initial webdriver agent under appium server GUI &amp; the new downloaded one i.e. 4.9.0, the versions were same: CFBundleShortVersionString 1.0 CFBundleVersion1 Downloaded support file for 16.4.1 exclusively i.e. different from 16.4 &amp; placed at /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/ Tried building with real device iOS 16.1.2 but didn't work. Please help, thanks !
Posted
by
Post not yet marked as solved
1 Replies
1.3k Views
I've written a Swift package plugin to add the colours provided in the asset bundles in an extension of the SwiftUI Color type but I'm having difficulties testing this since the bundle and/or colour that is in the asset catalogs are not yet loaded when XCTest goes to evaluate the test conditions. Here is my test code: (colours are just random colours, some derive from macOS colour picker; also, the new static method is functionality that I added and is not present in the existing Color APIs) final class TestCase: XCTestCase { func testBanana() { XCTAssertEqual(Color.banana, Color.new(from: "FEFC78")!) } func testAlexsColor() { XCTAssertEqual(Color.alexsColor, Color.new(from: "0432FF")!) } func testMaximumPowerColor() { XCTAssertEqual(Color.maximumPower, Color.new(from: "FF2600")!) } func testLongNameColor() { XCTAssertEqual(Color.reallyLongNameThatJustKeepsGoingOnAndOnAndOnAndOn, Color.new(from: "AAA000")) } } Here is an example error message that I get when I run the test: testBanana(): XCTAssertEqual failed: ("NamedColor(name: "Banana", bundle: Optional(NSBundle &lt;/Users/trevorhafner/Library/Developer/Xcode/DerivedData/TransportBase-cbagdabrompfzofwkimswvlsincu/Build/Products/Debug/TransportBaseTests.xctest/Contents/Resources/TransportBase_TransportBaseTests.bundle&gt; (not yet loaded)))") is not equal to ("#FEFC78FF") Here is the autogenerated code that my Swift Package Plugin creates: #if canImport(SwiftUI) import SwiftUI import Foundation @available(iOS 13.0, macOS 10.15, tvOS 13.0, watchOS 6.0, *) public extension Color { static let banana = Color("Banana", bundle: .module) static let alexsColor = Color("Alex's Color", bundle: .module) static let reallyLongNameThatJustKeepsGoingOnAndOnAndOnAndOn = Color("really long name that just keeps going on and on and on and on", bundle: .module) static let maximumPower = Color("Maximum Power", bundle: .module) } #endif Does anyone know how to somehow instruct the bundle or the Color to load such that the comparison can be made correctly between the two colours?
Posted
by
Post not yet marked as solved
5 Replies
637 Views
This is for MacOS My app posts errors during its operation. Two years ago the following used to be able to test whether the alert was correct (or missing) func isRightAlert(alertImgStr: String, target: String) -> Bool{ let app = XCUIApplication() let ackdialog = app.dialogs["alert"] let imageexists = ackdialog.images[alertImgStr/*"CaptureEclipse alert"*/].exists //let full = app.debugDescription XCTAssert(imageexists) let acklist = ackdialog.children(matching: .staticText) //NSLog( String(acklist.count)) var found = false NSLog("Alert>>>>>>>>>>>") for index in 0...acklist.count - 1{ <see if is is one of the expected ones> } <other tests> } Now that I am on Ventura with XCode 14.3 it is no longer able to detect the alert. ackdialog returns as not found. imageexists is false which triggers the Assert The alert is on the screen when the timeouts I added trigger app.debugDescription justs lists all of my menu entries These are my own alerts (NSAlert) and not system alerts.
Posted
by
Post not yet marked as solved
3 Replies
682 Views
I see this happening in other threads but none of those solutions have worked for me. I'm new to Swift (my apologies) and attempting a simple test to get a string response from a service I have running on my machine. (This service show results from a browser and Postman so I'm sure it's working.) This is my code: let url = URL(string: "http://127.0.0.1:8080/stuff/ping")! var request = URLRequest(url: url) request.httpMethod = "GET" print("creating ping task...") let task = URLSession.shared.dataTask(with: request) { data, response, error in print("processing ping result...") if let data = data { let str = String(decoding: data, as: UTF8.self) print(str) } else if let error = error { print("HTTP Request Failed \(error)") } } print("resuming ping task...") task.resume() When I run that in a playground, it works! The console shows: creating ping task.... resuming ping task... processing ping result... One ping only, Vasily But from within a XCTest, it does not: Test Suite 'NetworkTests' started at 2023-06-02 08:24:46.007 Test Case '-[StuffTests.NetworkTests testPing]' started. creating ping task... resuming ping task... Test Case '-[StuffTests.NetworkTests testPing]' passed (0.002 seconds). Test Suite 'NetworkTests' passed at 2023-06-02 08:24:46.010. Executed 1 test, with 0 failures (0 unexpected) in 0.002 (0.002) seconds I've set the info of the project according to what I've seen in other posts so that it can access a local host (?): I appeal to you for help in understanding and fixing this. thank you!
Posted
by
Post not yet marked as solved
1 Replies
909 Views
I have some trouble to use this API https://developer.apple.com/documentation/xctest/xcuisystem/4108234-open in UI testing. Calling this API that opens my app but a system dialog Open in "MyApp" will show. XCUIDevice.shared.system.open(url) is a sync API, so I can't use my test to tap on the "Open" button on the dialog. This basically makes the API unusable, are there some way to bypass this system dialog?
Posted
by
Post not yet marked as solved
0 Replies
355 Views
Hi all, I'm just switching my app to use a separate synchronized and local CoreData store. Normally, for setting my SQLite as in-memory for utilization in UnitTests, I use the common setup of setting the URL to "/dev/null/": container.persistentStoreDescriptions.first?.url = URL(fileURLWithPath: "/dev/null/") Anyone an idea, how to use this approach when using multiple stores, as I can't link both persistentStoreDescriptions to the same URL? Many thanks for your help. Folox
Posted
by
Post not yet marked as solved
1 Replies
768 Views
I am trying to create a document-based (macOS) app using SwiftUI and SwiftData. I'm trying to set up XCTests to validate my model behaviors. Clearly, I need to set up a container in my test class for my model objects. Failing to do so shows that SwiftData is trying: my app launches into the New Document dialog; if I dismiss it, the app fails with a "failed to find an active container" for my model object. Any pointer on setting up a container and context within XCTest is welcome. TIA
Posted
by
Post not yet marked as solved
9 Replies
1.9k Views
We're seeing this message when trying to run test automation on iOS 17. We're building the app on Sonoma, using Xcode 15, and running on iOS 17 (physical iPhone 13). At one point in the test we want to verify that something is correctly set in the iOS Setting app. We were previously doing this by launching com.apple.Preferences On iOS 17 we're getting a failure because "XCTAutomationSupport couldn’t be loaded because it is damaged or missing necessary resources" I suspect that the error message is not actually representative of the real error Anyone seen this before?
Posted
by