Why using Integration tests instead of unit tests is a bad idea?

user_108317 0
Viewed 14171

Let me start from definition:

Unit Test is a software verification and validation method in which a programmer tests if individual units of source code are fit for use

Integration testing is the activity of software testing in which individual software modules are combined and tested as a group.

Although they serve different purposes very often these terms are mixed up. Developers refer to automated integration tests as unit tests. Also some argue which one is better which seems to me as a wrong question at all.

I would like to ask development community to share their opinions on why automated integration tests cannot replace classic unit tests.

Here are my own observations:

  1. Integration tests can not be used with TDD approach
  2. Integration tests are slow and can not be executed very often
  3. In most cases integration tests do not indicate the source of the problem
  4. it's more difficult to create test environment with integration tests
  5. it's more difficult to ensure high coverage (e.g. simulating special cases, unexpected failures etc)
  6. Integration tests can not be used with Interaction based testing
  7. Integration tests move moment of discovering defect further (from paxdiablo)

EDIT: Just to clarify once again: the question is not about whether to use integration or unit testing and not about which one is more useful. Basically I want to collect arguments to the development teams which write ONLY integration tests and consider them as unit tests. Any test which involve components from different layers is considered as integration test. This is to compare to unit test where isolation is the main goal.

Thank you, Andrey

integration-testing tdd unit-testing
15 Answers

It is a bad idea to "use integration tests instead of unit tests" because it means you aren't appreciating that they are testing different things, and of course passing and failing tests will give you different information. They make up sort of a ying and yang of testing as they approach it from either side.

  • Integration tests take an approach that simulates how a user would interact with the application. These will cut down on the need for as much manual testing, and passing tests will can tell you that you app is good to go on multiple platforms. A failing test will tell you that something is broken but often doesn't give you a whole lot of information about what's wrong with the underlying code.

  • Unit tests should be focusing on making sure the inputs and outputs of your function are what you expect them to be in all cases. Passing units tests can mean that your functions are working according to spec (assuming you have tests for all situations). However, all your functions working properly in isolation doesn't necessarily mean that everything will work perfectly when it's deployed. A failing unit test will give you detailed, specific information about why it's failing which should in theory make it easier to debug.

In the end I believe a combination of both unit and integration tests will yield the quickest a most bug-free software. You could choose to use one and not the other, but I avoid using the phrase "instead of".

user_19103550

How I see integration testing & unit testing:

Unit Testing: Test small things in isolation with low level details including but not limited to 'method conditions', checks, loops, defaulting, calculations etc.

Integration testing: Test wider scope which involves number of components, which can impact the behaviour of other things when married together. Integration tests should cover end to end integration & behaviours. The purpose of integration tests should be to prove systems/components work fine when integrated together.

user_34532440

(I think) What is referred here by OP as integration tests are leaning more to scenario level tests.

But where do we draw the line between unit -> integration -> scenario?

What I often see is developers writing a feature and then when unit testing it mocking away every other piece of code this feature uses/consumes and only test their own feature-code because they think someone else tested that so it should be fine. This helps code coverage but can harm the application in general.

In theory the small isolation of Unit Test should cover a lot since everything is tested in its own scope. But such tests are flawed and do not see the complete picture.

A good Unit test should try to mock as least as possible. Mocking API and persistency would be something for example. Even if the application itself does not use IOC (Inversion Of Control) it should be easy to spin up some objects for a test without mocking if every developer working on the project does it as well it gets even easier. Then the test are useful. These kind of tests have an integration character to them aren't as easy to write but help you find design flaws of your code. If it is not easy to test then adapt your code to make it easy to test. (TDD)

Pros

  1. Fast issue identification
  2. Helps even before a PR merge
  3. Simple to implement and maintain
  4. Providing a lot of data for code quality checking (e.g. coverage etc.)
  5. Allows TDD (Test Driven Development)

Cons

  1. Misses scenario integration errors
  2. Succumbs to developer blindness in their own code(happens to all of us)

A good integration test would be executed for complete end to end scenarios and even check persistency and APIs which the unit test could not cover so you might know where to look first when those fail.

Pros:

  1. Test close to real world e2e scenario
  2. Finds Issues that developers did not think about
  3. Very helpful in microservices architectures

Cons:

  1. Most of the time slow
  2. Need often a rather complex setup
  3. Environment (persistency and api) pollution issues (needs cleanup steps)
  4. Mostly not feasible to be used on PR's (Pull Requests)

TLDR: You need both you cant replace one with the other! The question is how to design such tests to get the best from both. And not just have them to show good statistics to the management.

user_83998020
Related