Pinning Tests

I wrote this on the C2 Wiki, with the hopes that other people would help improve it. But now that site is down, so I'm posting it here:

Definition: A simple-minded automated test that locks down the behavior of existing code that otherwise is not well-tested, as a safety net while refactoring.

Example: Run some code and collect logs as a baseline. Each time you make a change, run the program again and compare the logs against the baseline. As long as there is no difference, you have some confidence that things are still working.

Pinning tests can make it safer to refactor. (Pinning tests can never make refactoring completely safe, because you'll forget important cases in your pinning tests. For safety, use #3 or #4 from Various Definitions of "Refactoring"). Pinning tests are a safety net, just in case.)

The most important features of pinning tests are:

1. Give an obvious, definitive pass or fail result.
2. Good coverage. Professional testers get really good at this; ask them to help.
3. Faster is better, so you can run them often.

Non-requirements for pinning tests:

Robustness. Professional testers get really good at making robust tests that work on different computers, or at different screen resolutions, or across UI changes. Ask them to refrain - these tests are short lived, and the behavior of the system won't be changing (by definition of "Refactoring").

You don't need to run your pinning tests in every environment that you ship. For a GUI, it's fine to record mouse clicks and keystrokes.

Long-lived. The goal is to hold behavior constant for just long enough to ReFactor.

Clean code. Hacking the test together is OK. For example,

• Use the C preprocessor to redirect troublesome API calls to write to a log instead.
• Edit your HOSTS file to hijack accessing a network resource.