I have faced some perennial questions when developing automated tests are:
- should I write unit tests, integration or functional tests.
- why do I have to fix tests all the time, shouldn't they be catching bugs instead.
- the tests take too long to run, who pays for all the lost time
- what percentage of unit test coverage is good enough
- no matter how many unit tests I write we still have lot of bugs
I have heard some comments on them but not really answers. For example to the last one: "you are not writing the right tests". May be true but not very useful.
While there are lot of subjective factors we can use to categorize our tests, there are three things objective about any test.
Scope: amount of the code under test.
Performance: time to run the test.
Business criticality of the function under test.
Ideally I would like write business critical tests
Test what breaks and not blindly. Automate key business scenarios.