How ordering of Unit Tests in Xcode works? My tests fail because of the ordering and dependencies

Hello,

I’ve a problem with unit tests. Whenever I run the unit tests individually, they pass. However, if I run them altogether, some fail. Since the tests are based on shared data (Database), I know this may be the problem. 

In order to solve this, I clear the database on setup function in each test. Yet, this doesn’t fix the issue.

So I assume the unit tests work in parallel, which leads to race condition (so some fail & some pass).

My questions are

  1. how unit test order works? 
  2. Can I force unit tests to run the tests sequentially?  Can I force it not to move to the next test until the current test is done?
  3. What about setup & teardown, do they always work til the end before running test functions?

Thanks! 🙏

Up vote post of abc_2022 Down vote post of abc_2022
2.2k views
Posted by
abc_2022
Reply
Add a Comment

Replies

You could probably mock the database so you don't need to rely on it in each test. And yes, as you noticed, unit tests shouldn't depend on one another.

Posted by
darkpaw
Up vote reply of darkpaw Down vote reply of darkpaw
Add a Comment

The fundamental problem here is your tests; they should not depend on being run in a specific order. I recommend that you come up with a plan to fix that in the medium term.

In the short term, you can force tests to run in order using the Configuration > Test Execution > Execution Order option in your test plan.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Post not yet marked as solved Up vote reply of eskimo Down vote reply of eskimo
Add a Comment

I dont agree that in all cases the problem are the tests. For Instance we use OCmock and create partial mocks for objects. and that's cause us a lot of issues when different tests mocking/using the same objects simultaneously. When u have a multiple developers trying to test same region, there is good chance of race conditions between the tests accessing same area of code/ resources what what won't really happen in simple user flow

Posted by
mike sldkcjnsdmhvbsdh
Up vote reply of mike sldkcjnsdmhvbsdh Down vote reply of mike sldkcjnsdmhvbsdh
Add a Comment