Several months ago, James Bach shared the idea of test framing. He identified it as one of the necessary skills tester and done some work to refine this idea by performing tests in support of an exercise with one of their online students.
Recently, we have additionally worked on perfecting the concept. September 30, 2010 I made a short presentation regarding the basis of tests at a meeting of the Association for the software quality Kitchener-Waterloo, and also held a four-hour seminar on this topic EuroSTAR.
The basic idea is as follows. At any point in testing
Test framing is the ability to track and / or build a logical chain that connects the purpose of testing with the tests. Following the natural route, the logical chain will affect the items within the list above. Objective justification test is to be able to answer questions like:
The form, which is used to justify the tests – a sequence of statements and logical connectives linking the test to. Statement – a sentence that may be true or false. We can assume that the statements – it’s affirmative proposals or assumptions. Ligaments – a word or phrase (“and”, “no”, “if”, “consequently”, “thus,” because, “since”, “On the other hand,” but it is possible that ” and so on) that connect the statements with each other to form a new expression.
This is not a strictly formal system, but it heuristically and well structured. That’s pretty obvious example.
Given: (The purpose of the testing:) to detect problems that may threaten the value of the product, for example, the incorrect behavior of a program or data loss.
Saying: There is an input field.
Saying: When the user presses the input text field sends the data to the clipboard.
Saying: Unlimited I can cram a clipboard.
Saying: Buffer overflow erase data or programming code.
Saying: erased data can lead to data loss.
Saying: nipped the code could lead to the observed incorrect program behavior.
A bunch of statements: if this input field is not restricted, and if it is because of this buffer overflows, THEREFORE, there is a risk of data loss or incorrect program behavior.
Statement: The larger set of data sent in a text field, the more likely erase the code or data.
Bundle: The larger the data set, the higher the likelihood of the observed problems.
Bundle: If I place in this field very long line, I’m more likely to be able to see the problem.
Output (test): Therefore I will try to copy a very long line in the text box and watch for signs of damage, such as debris in the records that I’ve seen before in an unchanged form, or a memory leak, or decrease programs or other unusual behavior.
To some this may seem logical though, but too obvious. However, to our surprise, some testers have difficulty tracing the path from the objective test to test, or in the opposite direction from the test to the goal – or a quick and convincing reconstruction of the chain of reasoning.
Therefore, the training we carry out this exercise. Testers available object and purpose of testing. We invite them to describe any test, and then ask to restore their reasoning. Alternatively, we might ask why they chose this test, and ask for an explanation of your choice, trace the logical chain back to the goal.
You can do this exercise on their own. If you have a test that was not subjected to the rationale, try to justify it. You must be able to do it for most of the tests, but if you can not structure the test immediately, maybe all is in order. Why? Because when we test, we not only use the information, we also reveal it. This identified one test information can be used to justify some other tests. Once you follow the well-founded tests, add a few tests, which can not be immediately or completely build a foundation.
One possible reason for the test that is resistant to the rationale, is that we always work with ulterior justifications. Revealing the hidden or unknown justification is the motivation perform a large number of random autotests or stress testing, or use of styles and methods of testing that can lead to unexpected results (but may not give). The fact that you have found something unusual, give retrospective justification as to why you performed this test. This rationale is not based on well-known theory of errors, it is impossible to give in advance. But be that as it may be better, if you’re not a customer, say, “Who would have thought?”
Testing – a narrative consisting of two parallel stories: the history of the product and the history of the test. We are James and believe that the rationale for tests – a key skill that helps to compose, edit, and told the story to justify testing logically connected and quickly.