Paper work on Principles of Software Testing

Theme of testing is Enhancing the satisfaction and safety of software customers, users, and developers By Cem Kaner

Introduction: Software testing is an extremely creative and intellectually challenging task. When testing follows the principles given below, the creative element of test design and execution rivals any of the preceding software development steps. These are not all obvious, but the purpose is to guide testers, and prevent the types of problems.
Principles:
· Testing shows presence of defect
· Exhaustive testing is impossible
· Early testing
· Defect clustering
· Pesticide paradox
· Testing is context dependant
· Absence-of-errors fallacy

Description:
Testing shows presence of defect: Testing can show that defects are present, but cannot prove that there are no defects. Running a test through a software system can only show that one or more defects exist.
Testing cannot show that the software is error free.Eg: the top 10 wanted criminals website was error free. There were no functional defects, yet the website failed. In this case the problem was non- functional and the absence of defects was not adequate as a criterion for release of the website into operation.

Exhaustive testing is impossible: Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Usually, the test data set is infinite or significantly large, which creates a situation that testing each element in the test domain is not pragmatic. If testing finds problems, then surely you would expect more testing to find additional problems, until eventually we would have found them all. Consider a small example where enter a password that can contain up to three characters, the number of possibilities is higher. It is not possible to say whether there are any defects until all possible input combinations have been tried. Even in this small example, there are many possible data combinations to attempt. So exhaustive testing is not possible.
Instead of exhaustive testing, we use risk and priorities to focus testing efforts.

Early testing: Early testing in order to detect flaws early in the software life cycle. Testing activities should start as early as possible in the software or system development life cycle, and should be focused on defined objectives. This principle is important because, as a proposed deployment date approaches, time pressure can increase dramatically. There is a real danger that testing will be squeezed, and this is bad news if the only testing we are doing is after all the development has been completed. The earlier the testing activity is started, the longer the elapsed time available. Testers do not have to wait until software is available to test.
Defect clustering: In a large application, it is often a small number of modules that exhibit the majority of the problems. This is the application of the Pareto principle to software testing: approximately 80 per cent of the problems are found in about 20 per cent of the modules. It is useful if testing activity reflects this spread of defects, and targets areas of the application under test where a high proportion of defects can be found. However, it must be remembered that testing should not concentrate exclusively on these parts. There may be fewer defects in the remaining code, but testers still need to search diligently for them.

The pesticide paradox: Running the same set of tests continually will not continue to find new defects. Developers will soon know that the test team always tests the boundaries of conditions, for example, so they will test these conditions before the
Software is delivered. This does not make defects elsewhere in the code less
Likely, so continuing to use the same test set will result in decreasing effectiveness of the tests. Eventually the same set of test cases will no longer find any new bugs. To overcome this “pesticide paradox”, the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.

Testing is context dependent: Testing is done differently in different contexts. For example, safety-critical software is tested differently from an e-commerce site. For an e-commerce site, we should concentrate on security (password) aspects. We need to test an air traffic control system with more care than an application for calculating the length of a mortgage.
Risk can be a large factor in determining the type of testing that is needed. The higher the possibility of losses, the more we need to invest in testing the software before it is implemented.
Absence of errors fallacy: Finding and fixing defects does not help if the system built is unusable and does not fulfill the user’s needs and expectations. The fact that no defects are outstanding is not a good reason to ship the software. Before dynamic testing has begun, there are no defects reported against the code delivered.
Advantages: 1. Reduce faults cost
2. Reduce faults multiplication.
3. Errors in critical modules are found early.
4. Improve the quality.
5. Save the time.
Conclusion: In my humble opinion early testing is the best principle because if testing activity is started earlier then testers do not have to wait until software is available to test. Due to this we will save time and resources.By using these Principles we can do effective testing and we can get a quality product