Post not yet marked as solved
Hi,Is functional testing supported in UI Test ?I'd like to check the state of my application is correct in a UI Test function (like I have done in my functional tests in the Unit Test target), however adding `@testable import MyApp` causes a link time error:Undefined symbols for architecture x86_64:
"type metadata accessor for MyApp.MainViewController", referenced from:
MyAppUITests.MyAppUITests.setUp (MyAppUITests.MyAppUITests)() -> () in MyAppUITests.o
type metadata accessor for MyApp.MainViewController? in MyAppUITests.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)I noticed that `UIApplication.sharedApplication().windows` also returns an empty array.Has access to the app been removed from UI Test targets?Cheers,Daniel
Post not yet marked as solved
A couple of days ago Xcode stopped loading our tests into the test navigator. Additionally our test class files stopped showing the gutter test diamonds. We can still run the tests using CMD+U and after a successful run every test case shows up in the test navigator. But, the test diamonds don't return which implies that Xcode doesn't see these files as tests.So, it seems that something in our project has broken Xcode's test case indexing.The project is mainly Obj-C with an growing Swift part. Our test suite contains ~6000 unit tests derived from XCTestCase. This problem is reproducible on El Capitan 10.11.3, both when using Xcode 7.2.1 and Xcode 7.3 beta 5.We've tried all the usual measures for when dealing with bad Xcode state:Clean building.Deleting derived data.Deleting ~/Library/Caches/com.apple.dt.XCodeIs this a known issue? Can anyone help us understand how Xcode discovers the tests so that we could fix the issue with our project?Update:When observing what happens in Console.app when opening Xcode we see the following lines:17/03/16 11:44:44,717 Xcode[89856]: IDEIndexingClangInvocation: Failed to save PCH file: /Users/johannth/Library/Developer/Xcode/DerivedData/MyApp-fmkkuqtykhsstrezzmhrfiedrpcu/Index/PrecompiledHeaders/UnitTests-Prefix-dvmyiewguqjpnbflmvoybdjdec.pch.pch
17/03/16 11:44:53,954 sourcekit-serv[89989]: [1:hashFileReference:8599: 2.1299] failed to stat file: <imports> (No such file or directory)We actually see a lot of the latter error. By running git bisect we've pinpointed that setting `DEFINE_MODULE=YES` introduced the issue and reverting it to `DEFINE_MODULE=NO` fixes it with various weird side effects. The .pch file for UnitTests only contains the standard Foundation and UIKit imports so I don't think there is anything directly in it which breaks the indexing.Anyone have any idea how to better understand the console logs from Xcode?
Post not yet marked as solved
In our application, we are having custom logging. When the project is compiled locally on the XCode, I can see that those logs are appearing properly on Xcode Console (Debug Area). Also, when compiling the UITest scheme, the application logs are appearing as expected. But when I run a uitest, then only test runner logs are showing up in debug area. When I select My application target then it's not having application logs.Is this something expected? Is there any setting that I have to turn ON with in Xcode?Any help will be really appreciated.Thanks
Post not yet marked as solved
Hello,Since Xcode 11, getting value from a UISwitch in Settings app with XCUITest is not working anymore with iOS 12 devices.For example, going on the Notification settings, I would like to check some switch values. let switchSounds = settingsApp.switches["Sounds"]
let expectedValue = isEnabled ? "1" : "0"
XCTAssert(switchSounds.value as? String == expectedValue)The switchSounds is found but its value is nil. It was working before and it still works on the app I run my other tests on.Is there any workaround I can use ? I tried using isSelected but it's not working either.
Post not yet marked as solved
Sometimes when running UI tests, the tests fail on keyboard input because the "Speed up your typing" keyboard tutorial overlay obscures the keys. This happens randomly, but sometimes repeatedly.
Is there a way to turn this off? Checking for this tutorial before any keyboard input would significantly complicate tests just to check for one edge case...
Post not yet marked as solved
Test UI and unit tests are working fine on 14.3 with XCode 13.0 beta4 but it gets stuck on 15.0 and doesn't execute any test case with no error.
Post not yet marked as solved
I have a SwiftUI menu
Menu{
....
}, label : {
Image(...).accessibility(identifier: "cardMenu")
}
I used to be able to bring up the menu (before upgrading to xcode 13 (ios15)) like this
let app = XCUIApplication()
app.launch()
app.buttons["cardMenu"].tap()
But now i am unable to see the identifier in app.buttons. Can't seem to find the identifier anymore. I've tried looking for the identifier in the other app fields and changing to use text instead of identifer. No luck. These tests used to work prior to the upgrade.
Any help would be appreciated
Post not yet marked as solved
I am trying to run my projects unit test targets on Xcode 13. Xcode is not able to run the tests. It always throws me this error
xctest[36188:349217] Failed to create a bundle instance representing '/Users/bvijay16/Library/Developer/Xcode/DerivedData/Build/Products/Debug-iphonesimulator/DataStoreTests.xctest'. Check that the bundle exists on disk.
Program ended with exit code: 1.
Please let me know if anyone of you have faced the same issue. I have tried. clearing derived data, re updating pods and almost all the work arounds.
Post not yet marked as solved
We got the newly issue that our Test devices keeps us asking for the pin code to "Enable UI Automation".
Then it works for some hours or days, but after some time it starts again.
"Enable UI Automation" is already enabled in "Settings" - "Developer" menu.
The devices are located remotely and we can't access them directly, so this is a big issue for us right now.
Is there any way to avoid this?
I can't execute individual tests or test suits on XCode 13.1, I get this message:
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
I can only execute the complete set of tests, is this a bug or am I missing something ?
Post not yet marked as solved
We are having an issue when trying to view the the coveage report in our "merged result bundle"
Running xccov view --report --only-targets merged.xcresult results in:
09:52:54 Error: Error Domain=XCCovErrorDomain Code=0 "Failed to load coverage archive in scheme action '(null)' in result bundle" UserInfo={NSLocalizedDescription=Failed to load coverage archive in scheme action '(null)' in result bundle, NSUnderlyingError=0x7f8ff8714f30 {Error Domain=NSCocoaErrorDomain Code=260 "The file “Metadata.plist” couldn’t be opened because there is no such file." UserInfo={NSFilePath=/tmp/action.xccovarchive/Metadata.plist, NSUnderlyingError=0x7f8ff8714930 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}}}
Checking the man page for xccov I was inspired by the idea of extracting the coverage report from the result bundle to try to understand or work around our error.
The man xccov page states that
If xccov is passed a result bundle directly, it will extract the report or archive implicitly, as part of the action being requested. Alternatively, the raw report/archive files can be extracted from the result bundle using xcresulttool(1) and subsequently passed to xccov.
Checking man xcresulttool and trying out some stuff randomly I was not able to learn how a person might:
Given an xcresult bundle use xcresulttool to create/extract an xccovreport bundle.
Post not yet marked as solved
On an intel mac, my XCUITest test cases which is included swipe actions works perfectly but it doesn't work on my m1 mac. I tried running simulator with Rosetta but it doesnt work out it still gives an error when I run my cases because it clicks instead of swipe.
Does anyone have a solution for swiping problem on the m1?
Post not yet marked as solved
Hi,
I'm currently researching if we are able to migrate to Xcode Cloud in our project. I encountered one issue.
I run unit tests on Xcode Cloud and I see that they finish in ~150 seconds, but the whole step takes ~530s and there is no information in logs why. I even run the same command on my local computer and it doesn't take that long. Any ideas? Any way to optimize it?
Also one more thing: it looks like Xcode Cloud is not running on M1 machines. Do you know guys if there are plans for Xcode Cloud to take advantage of M1/M2?
Post not yet marked as solved
Hi everybody,
given the following case using SwiftUI:
Button("testButton", action: noop)
.accessibilityRepresentation{representation: {
Slider(value: $sliderData.sliderValue)
})
.accessibilityIdentifier("testSlider")
I'm trying to control the slider using XCUITest via slider.adjust(toNormalizedSliderPosition: 0.2)
and receive the following error message:
Failed: Unable to get expected attributes for slider, found {
"XC_kAXXCAttributeMaxScrubberPosition" = {
X = 0;
Y = 0;
};
"XC_kAXXCAttributeMinScrubberPosition" = {
X = 0;
Y = 0;
};
Similarly, print (slider.normalizedSliderPosition) leads to the error message "Failed to determine current position of slider "0 %".
Is there a way to set and read the value of accessibilityRepresentation-Sliders with XCUITest?
Thanks!
Post not yet marked as solved
Hello,
I'm trying to visualize code coverage data in our project but I've faced issues getting the coverage data from the .xcresult bundle produced by xCode Cloud.
On my local machine, I'm using xccov view to preview coverage data and everything looks fine:
/Users/d.kuznetsov/Git/***/***.swift:
1: *
2: *
3: *
4: *
5: *
6: *
7: *
8: *
9: *
10: *
11: *
12: *
13: *
14: *
15: *
16: 4
17: *
18: *
19: *
20: *
21: *
22: *
23: *
24: *
25: *
26: *
27: *
28: *
29: *
30: *
31: 3
32: 3
33: 3
34: 3
35: *
36: 3
37: 3 [
(56, 0, 0)
]
38: 0
39: 0
40: 3 [
(1, 9, 0)
]
41: 3
42: 3
43: 3
44: 3
45: *
46: 3
47: 3
48: 3
49: 3
50: *
But when I tried to use the same approach in ci_post_xcodebuild.sh I couldn't see the coverage data:
#!/bin/sh
xcrun xccov view --archive $CI_RESULT_BUNDLE_PATH
Am I missing something? The schema settings are the same for both Cloud and local machine and code coverage gathering setting is enabled.
Post not yet marked as solved
Hello everyone. There is really annoying regression in xcode 14 and 14.1 beta. We have a large modularized workspace. In workspace there is one ios app target that hosts xctestplan that contains over 50 unit test targets. When we try to run tests with cmd+u xcode immidately freezes and force quit is required. This was not an issue with xcode 13.
I tried to run tests via command line with xcodebuild. This time it stucks at prepare packages stage before building any target. After about 7-10 minutes it continues building targets. So I am guessing there is something wrong with resolving target dependencies in xcodebuild.
Anyway this is super frustrating for us and it holds us back to moving xcode 14. Is there other people experiencing the same issue with xcode 14? Did you find any workaround?
Post not yet marked as solved
Setup
I've been experimenting with migrating to Xcode Cloud for our CI but I'm struggling with getting Snapshot Testing to work.
I've been involved in some discussions here... https://github.com/pointfreeco/swift-snapshot-testing/discussions/553
And I saw this site also which explains a little... https://wojciechkulik.pl/xcode/xcode-cloud-overview-and-setup
But so far I've been unsuccessful in getting it to work at all.
I'm using the swift-snapshot-testing library from PointFree https://github.com/pointfreeco/swift-snapshot-testing and I have overridden the assertSnapshot function to change the snapshot testing directory.
The function looks like this...
func snapshotDirectory(
for file: StaticString,
ciScriptsPathComponent: String = "ci_scripts",
relativePathComponent: String = "Tests"
) -> String {
var sourcePathComponents = URL(fileURLWithPath: "\(file)").pathComponents
if let indexFolder = sourcePathComponents.firstIndex(of: relativePathComponent) {
sourcePathComponents.insert("resources", at: indexFolder)
sourcePathComponents.insert(ciScriptsPathComponent, at: indexFolder)
}
var pathsComponents: [String] = sourcePathComponents.dropLast()
let fileUrl = URL(fileURLWithPath: "\(file)", isDirectory: false)
let folderName = fileUrl.deletingPathExtension().lastPathComponent
pathsComponents.append("__Snapshots__")
pathsComponents.append(folderName)
let directory = String(pathsComponents.joined(separator: "/").dropFirst())
return directory
}
public func assertSnapshot<Value, Format>(
matching value: @autoclosure () throws -> Value,
as snapshotting: Snapshotting<Value, Format>,
named name: String? = nil,
record recording: Bool = false,
timeout: TimeInterval = 5,
file: StaticString = #file,
testName: String = #function,
line: UInt = #line
) {
let failure = verifySnapshot(
matching: try value(),
as: snapshotting,
named: name,
record: recording,
snapshotDirectory: snapshotDirectory(for: file),
timeout: timeout,
file: file,
testName: testName
)
guard let message = failure else { return }
XCTFail("\(message) snap: \(snapshotDirectoryUrl) file: \(file) ", file: file, line: line)
}
Essentially this takes my test file path... repoRoot/Tests/FeatureTests/FeatureTestFile.swift.
And injects some path component into it so that you end up with a directory path... repoRoot/ci_scripts/resources/Tests/FeatureTests/__Snapshots__/FeatureTestFile/.
And then the snapshot file will be located in that directory using the name of the test function with a suffix of .1.txt or .2.txt (etc... for each subsequent snapshot in each function). i.e. testSnapshotStuff.1.txt, testSnapshotStuff.2.txt.
Problem
This all works locally. And all the files are checked into GitHub.
But, when I run this on Xcode Cloud it fails the tests and tells me the files are not there.
Having added some logging in it is writing new snapshot files to where I am expecting them to be so it just looks like those files are not available to the Test environment.
This is where I read about putting them into the ci_scripts file at the root of the repo. Which is what I've done. Files in this directory are supposed to be copied into the test environments so that they can be accessed... but it seems that they're not being.
I have tried using ci_scripts/resources and ci_scripts/Artifacts but it's always the same. The files aren't there and the Xcode Cloud tests write those files there over time.
I'm running out of options of what to do with this now. I just want a way that I can access these snapshot files in the test environment on Xcode Cloud.
Any help would be much appreciated.
Thanks
Post not yet marked as solved
Hello,
I’ve a problem with unit tests. Whenever I run the unit tests individually, they pass. However, if I run them altogether, some fail.
Since the tests are based on shared data (Database), I know this may be the problem.
In order to solve this, I clear the database on setup function in each test. Yet, this doesn’t fix the issue.
So I assume the unit tests work in parallel, which leads to race condition (so some fail & some pass).
My questions are
how unit test order works?
Can I force unit tests to run the tests sequentially? Can I force it not to move to the next test until the current test is done?
What about setup & teardown, do they always work til the end before running test functions?
Thanks! 🙏
Post not yet marked as solved
Hi,
I have an extensive macOS test suite that accesses a lot of files located on an external volume (several Tbytes of databases, images,...).
After moving to Ventura this test suite is broken. More specifically, when trying to access .sqlite databases I receive a SQL error 23: unauthorised access. This is just the symptom of the fact that I do not seem to have the access rights on those files from the test suite. I verified that by moving such a .sqlite file to /tmp, it opens fine and the corresponding test succeeds.
I tried fixing those rights by giving Full Disk Access (FDA) to Xcode, to xctest...but the problem persists.
Anyone out there having the same issue that found a fix ?
I should add that this test suite is associated with a framework (Swift package) and I have no Xcode project associated with it...just a plain Package.swift file. So I can't really fiddle with project level settings.
They surely must exist a way to access any file from a test suite even if it is located on an external disk, etc...
Thanks
Matthieu
Post not yet marked as solved
I'm new to Xcode cloud - working with a Mac OS app, build is working great. Now I am trying to add a Test action; the testing target builds but won't run, and the error indicates it can't find the testing bundle in the expected build output. There's also mention of a code signing error, but I have automatic code signing enabled with the same settings on test target as the app.
I am only running the unit test (XCTest) scheme, not the UI tests. When I run it locally from the IDE it works fine, either selecting the test scheme explicitly or as the test step of the app scheme.
I notice the XCTest target's scheme setup uses Debug builds and expects the test output to be in the Debug .app bundle, I thought perhaps that was the problem (in case only the release app bundle actually gets built in the Xcode Cloud environment). So I created a duplicate scheme and set the build targets to Release - again I can run this fine locally (after creating a release build), but it fails with the same error in Xcode cloud.
I also tried changing the code signing certificate from "Development" to "Sign to run locally" to see if that made a difference, but I get the same error. (It's using my developer account Team, and "Automatically manage signing".)
Can anyone relate the proper way to set up an XCTest scheme so that the tests will actually run in a Mac OS Xcode Cloud workflow? I'm using Xcode 14.0.1.
Here's the full error output, with [AppName] and [TestTargetName] substituted for the actual:
[AppName] (....) encountered an error (Failed to load the test bundle. If you believe this error represents a bug, please attach the result bundle at /Volumes/workspace/resultbundle.xcresult. (Underlying Error: The bundle “[TestTargetName]” couldn’t be loaded. The bundle couldn’t be loaded. Try reinstalling the bundle. dlopen(/Volumes/workspace/TestProducts/Debug/[AppName].app/Contents/PlugIns/[TestTargetName].xctest/Contents/MacOS/[TestTargetName], 0x0109): tried: '/Volumes/workspace/TestProducts/Debug/[TestTargetName]' (no such file), '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/[TestTargetName]' (no such file), '/Volumes/workspace/TestProducts/Debug/[AppName].app/Contents/PlugIns/[TestTargetName].xctest/Contents/MacOS/[TestTargetName]' (code signature in <....> '/Volumes/workspace/TestProducts/Debug/[AppName].app/Contents/PlugIns/[TestTargetName].xctest/Contents/MacOS/[TestTargetName]' not valid for use in process: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?))))
Thanks!
