We now have about 80 tests and the code that makes tests pass, but do the tests really test all the code? Using TDD, the code coverage of our tests should be quite high. Should.
Instead of guessing, we would rather have numbers that tell us how good the code coverage of our tests really is. Before Xcode 7, it was quite difficult to measure the coverage of a test suite, but with version 7, Apple added this feature to Xcode.
In this chapter, we will measure the code coverage of our tests, and we will take a look at how we can use Xcode Server and fastlane to automate everyday tasks in our lives as iOS developers. The chapter is structured like this:
- Enabling code coverage
- Continuous integration
- Automatic deployment with fastlane
Measuring the code coverage of our tests gives us a feeling of completeness about our test suite. While following the TDD workflow, as we don't write any code without a failing test, the code coverage of our project should be very high. We don't expect it to be 100%, meaning that all the code paths are executed in the tests because the static analyzer forces us to write code that we don't expect to be executed. For example, in the code we wrote, we often used guard to make sure that the value we wanted to access was not nil. We could have written tests for a case where the value was nil. But in my opinion, in most cases these tests give no additional value.
Nevertheless, we will examine the parts of the project without code coverage and discuss whether we need to add tests to cover them.
Xcode has added native support for the measurement of the code coverage of tests with version 7. To enable it, select Edit Scheme... in the Scheme selector in Xcode:
In the following pop-up window, select the Test phase and check Gather coverage data:
That is all! If you have tried to add the gathering of code coverage in Xcode 6, you will most probably be impressed by how easy this is in Xcode 8. Close the window, and run all the tests to measure the code coverage.
After the tests have finished, select the Report Navigator, click on Test, and select the Coverage tab, as shown in the following screenshot:
This opens the Coverage data view. On the left-hand side, you can see the files in the project, and on the right-hand side, the corresponding coverage value is shown. The worst coverage is in AppDelegate.swift. Click on the triangle next to the filename to expand its details. The details show the coverage data for all the methods in the file. It immediately becomes clear why the code coverage in AppDelegate.swift is that low. We left the methods from the template in AppDelegate even though we don't need them.
Let's remove the unused methods. Open AppDelegate.swift. The only really required method here is application(_:didFinishLaunchingWithOptions:). Remove all the other methods, run all the tests, and open the code coverage for the new test run. Now AppDelegate has 100% test coverage. Great!
Let's take a look at another file where the code coverage is not 100%. Open DetailViewController.swift and go to Editor | Show Code Coverage. If you cannot find the menu item and, instead, there is an item called Hide Code Coverage, it means that your editor is already set up correctly. With this setting, Xcode shows the coverage data in the editor next to the code:
The numbers show how often this code block has been executed during the test's run. If the number is 0, it means that this line did not get executed. In the case of DetailViewController.swift, the following line has no code coverage:
guard let itemInfo = itemInfo else { return }
To take a look at what is going on here, let's replace this line with the following equivalent implementation:
guard let itemInfo = itemInfo
else { return }
Run the tests again to collect the coverage data. The code coverage is zero in the line with the else clause. The reason for this is that we did not write a test for the case when itemInfo was nil. Do we need this test? In my opinion, in this case, it does not make sense to add a test for this because we will just return from viewWillAppear(_:) when itemInfo is nil. In addition to this, in our app, the only controller that creates an instance of DetailViewController is ItemListViewController, and we already have a test that this controller sets in the itemInfo dictionary.
In fact, it is a development error if we forget to set itemInfo because then the detail View Controller will not be able to show any useful data. So, instead of adding a test, we'd rather make sure that the app crashes when there isn't itemInfo at this point. Then, such an error would show up as a crash during development. We also find the error faster than in a case where we just return from viewWillAppear(_:), and wonder why the UI is not populated with data.
To make the app crash in case there is no itemInfo, replace the guard statement with the following:
guard let itemInfo = itemInfo
else { fatalError() }
What value for code coverage is enough? This question cannot be answered because it mostly depends on the project and people working on the project. In fact, it is often better to ignore the code coverage data altogether because it only has a limited value to decide whether tests are missing. But if you search the Internet for this question, you will find a lot of different opinions on the topic. You might have to find you own answer to this question.
In my opinion, the one and only measurement to answer whether there are enough tests is your confidence. If you are confident that the code you've written is working because you've tested all the relevant aspects of it, then you have enough tests.
But, nevertheless, the code coverage data can help you figure out whether you have missed something in your test that you thought would have already been tested.