Skip to main content

Is there such a thing as too much unit testing?

Testing, Unit Tests, TDD (Test Driven Development) are one of the first thing that we learn when we are in university and study computer science. A part of us might be lucky enough to have a specific course for TDD.

Code is written by people for machines that needs to resolve problems. People do mistakes and this is why it is so important to test our code. I didn’t have the opportunity to see until now a working application, without bug or issues from the beginning.

I would say that a complex application that is not covered by unit tests and without a testing process will end up in the trash. At the beginning of each project there is a classical discussion related to unit tests and code coverage.

- Do we need unit tests? 
- Yes.

- What is the code coverage target?
- 80%, 60%, 20%, 100%....

It is pretty clear that we need unit test. An engineer needs to be able to test his code and check if what we develop is working as expect or not. The response for the first question is pretty clear. But when we jump to the next question – the discussions can take hours and hours.
The development team will try to push the value as high as possible. In contrast, management might push back. Constraints like time, budget, complexity or quality level might impact the decision.
In situations like this, both parts might have good arguments to sustain their point of view. We are in a gray zone, where people can easily fall in a defense mode. I was very often in situations like this and unfortunately, managers will win. They are the one that takes the final decision.

What we shall do in this situations?

The first thing that we need to do is to map all the risks that might appear if there are no tests or if the code coverage is too low. Once we done this, we need to stay with all the team and identify how this cases could be mitigated.
Mitigation for this kind of problem could be strange, but acceptable at project level. Solution like increase the number of testers, bugs will be solved in production or accept quality level to go down are strange solutions for technical people, but this can be acceptable at project level.
The most important thing is to create the risk map. In this way people that can take a decision can have the whole picture in front of them.

The second thing that we need to do is to try to identify what are the components where complexity is high and the risk to have issues are imminent. Once we identify them we can requests that at least these components to be covered by tests.
In this way we can ensure that the most complex part of the system will be testeed and the development team can write good and working code from the beginning.

The 3rd things that can be done is to try to focus to write tests that covers custom logic (business logic). It is more likely to have an issues on custom code that calculates the discount level than in the one that save the results in a database or make a remote request. Of course also at persistence level we might have a bug, but because this layer is used by many other components also, the risk of having issues there without detecting it at development level is much lower.
When the available time for writing tests is not as much as we want we shall focus to cover the most important stuff with tests. Cover the part of the systems where you know that you will have issue.

Conclusion
In the end, there is trade-off. Getting enough time to have 100% coverage is not often. Even with 100% code coverage, you will still have bugs in production.
Don’t forget that we write code that runs on machines, but needs to be read by people.

Is there such a thing as too much unit testing?
No, but taking into account time, scope and quality, the quantity of unit tests can be impacted, even if there is a direct impact on this triangle (time, scope, quality).

Comments

  1. I think the questions are misleading. What is to have unit testing? Is it an obligation to have the code testable? Is it to have x% coverage (like half of project fully tested and the other not)? Is it related in any way to the possibility of future refactoring? How are the tests checked? Automatically when submitting or occasionally when someone decides they need to check?

    The first thing I realized about unit testing is that it forces you to write things a certain way. Having coverage may be nice, but also not representative. What I would push for is testable code, not tested code. And how do you "test" this? Just define a minimum requirement for one unit test and make every class that has any public methods or code properties have mandatory at least one.

    I've read once that intelligence seems to emerge naturally from attempts to keep as many options open. Without testable code you cannot have unit tests, with it, the choice remains yours.

    ReplyDelete

Post a Comment

Popular posts from this blog

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

Fundamental Books of a Software Engineer (version 2018)

More then six years ago I wrote a blog post about fundamental books that any software engineer (developer) should read. Now it is an excellent time to update this list with new entries.

There are 5 different categories of books, that represent the recommended path. For example, you start with Coding books, after that, you read books about Programming, Design and so on.
There are some books about C++ that I recommend not because you shall know C++, only because the concepts that you can learn from it.

Coding

Writing solid codeCode completeProgramming Pearls, more programming pearls(recommended)[NEW] Introduction to Algorithms

Programming

Refactoring (M. Fowler)Pragmatic ProgrammerClean code[NEW] Software Engineering: A Practitioner's Approach[NEW] The Mythical Man-Month[NEW] The Art of Computer Programming

Design

Applying UML and Patterns (GRASP patterns)C++ coding standards (Sutter, Alexandrescu)The C++ programming language (Stroustrup, Part IV)Object-oriented programming (Peter Coad)P…

Entity Framework (EF) TransactionScope vs Database.BeginTransaction

In today blog post we will talk a little about a new feature that is available on EF6+ related to Transactions.
Until now, when we had to use transaction we used ‘TransactionScope’. It works great and I would say that is something that is now in our blood.
using (var scope = new TransactionScope(TransactionScopeOption.Required)) { using (SqlConnection conn = new SqlConnection("...")) { conn.Open(); SqlCommand sqlCommand = new SqlCommand(); sqlCommand.Connection = conn; sqlCommand.CommandText = ... sqlCommand.ExecuteNonQuery(); ... } scope.Complete(); } Starting with EF6.0 we have a new way to work with transactions. The new approach is based on Database.BeginTransaction(), Database.Rollback(), Database.Commit(). Yes, no more TransactionScope.
In the followi…