Regression tests
Checking that a change has broken nothing: replaying the suite after every change, writing one test per fixed bug, comparing two reports, choosing the tests to replay and distrusting flaky tests.
- 22 cards
- 3 topics
- Free
The outline, topic by topic
Each topic is weighted the way the exam weighs it.
- Concepts
- Questions
- Sort and link
A few cards from the deck
Regression: what is it?
Show the answer
Behaviour that worked before a change and no longer works after it.
It often shows up far from what was changed, because a shared function serves several places.
A reported bug has just been reproduced by hand. Which step comes next, before touching the code?
- Write a test that reproduces the bug and see it fail
- Fix the code, then write a test that checks the fix
- Delete the failing test so that the suite goes green again
- Rerun the existing suite to see which test already reports the bug
Show the answer
Write a test that reproduces the bug and see it fail
A test written after the fix does not prove that it would have caught the bug. Seeing it fail first shows that it reproduces the bug, and seeing it pass afterwards shows that the fix works. Had the existing suite covered this case, the bug would not have slipped through: that test is exactly what it lacks.
When fixing a bug, one first writes a test that … it, then fixes it, so that this bug cannot come back undetected.
Show the answer
When fixing a bug, one first writes a test that exposes it, then fixes it, so that this bug cannot come back undetected.
The test stays in the suite: it becomes a safeguard for all future changes.
Regression test: what is it?
Show the answer
A test that keeps an already-correct feature under watch and is replayed after every change to make sure it stayed correct.
It is not a special kind of test: any passing test becomes a regression test from the moment it is replayed.
After fixing the VAT calculation, the PDF export shows wrong totals. What is the most likely cause?
- The PDF generator is incompatible with the new version of the code
- The export uses the same calculation function and no test covers it
- The VAT fix was wrong from the start
- The VAT test suite was not run before the fix
Show the answer
The export uses the same calculation function and no test covers it
A shared function fixed for one use also changes the others. The export, never tested, raised no alert. The remedy: a test on the export's totals, replayed at every change to the function.