Every now and then I discover test code that cannot fail. There are two main categories:
1) If you comment out the code that performs the action, the asserts still pass.
These tests usually check that something won’t happen. Like let’s say a user disables notifications and then you’re trying to make sure that no notifications go out. The obvious problem is that if you don’t generate notifications, then nothing will go out. If you do generate notifications but they are generated asynchronously and you check too soon, then it will also look like no notifications went out. If your test looked for a specific notification but got the search/filter wrong, then it will also look like no notifications went out. Regardless of whether they did or not. So testing this requires careful thinking and to some degree you need to prove that your test even works. Like maybe before disabling notifications you enable it, generate a notification and look for that notification so that the test that tests that disabling notifications can do the exact same thing when it proves that disabling notifications worked. Then you can have some confidence that the test is valid.
Experience can help anticipate these situations. Peer review hopefully helps if you have good peer reviewers. It helps being extremely skeptical when working on or reviewing this sort of thing.
2) Everything is stubbed to the point where you’re only testing the stubs.
These tests are usually easy to spot on those days where 300 of 320 tests failed because something fundamental broke and you start wondering why the other 20 passed ;)
There is probably a whole separate post about when to stub or mock and when not to, but the TLDR is to only stub as a last resort. Prefer integration tests over unit tests and prefer real data over fake data for tests. If it feels hard or like too much work or something that’s really slowing down test writing or even test execution, then take a step back and work on the infrastructure or the test helpers, maybe try and think outside the box.
In the meantime check out these classic talks on the subject if you’re interested:
