Objectives of Testing & Testing Techniques

Objectives of Testing

To find greatest possible number of errors with manageable amount of efforts applied over a realistic time span with a finite number of test cases.


What Does Software testing Reveal?
1. Errors
2. Requirements conformance or the lack of it
3. Performance
4. An indication of quality

Testing Techniques

•Static Testing - Testing a software without execution on a computer. Involves just examination/review and evaluation.
•Dynamic Testing - Testing software through executing it.

Types Of Testing Techniques

Static Testing ::

Review

Code inspection

Walkthrough

Desk check

Dynamic Testing

White box

Black box

Static Testing

•Static Testing is a process of reviewing the work product and reviewing is done using a checklist.
•Static Testing helps weed out many errors/bugs at an early stage
•Static Testing lays strict emphasis on conforming to specifications.
•Static Testing can discover dead codes, infinite loops, uninitialised and unused variables, standard violations and is effective in finding 30-70% of errors

Static Testing Methods
• Self Review
• Code Inspection
• Walk Through
• Desk Checking

Code Review Checklist
•Data Reference Errors
•Data Declaration Errors
•Computation errors
•Comparison errors
•Control Flow errors
•Interface errors
•Input/output errors

Code Inspection
•Code inspection is a set of procedures and error detection techniques for group code reading.
•Involves reading or visual inspection of a program by a team of people , hence it is a group activity.
•The objective is to find errors but not solutions to the errors
•An inspection team usually consists of:
o A moderator
o A programmer
o The program designer
o A test specialist

Code Inspection Procedure

•The moderator distributes the program’s listing and design specification to the group well in advance of the inspection session
•During the inspection
• The programmer narrates the logic of the program, statement by statement
• During the discourse, questions are raised and pursued to determine if errors exist
• The program is analyzed w.r.t a check list of historically common programming errors

Code Inspection Helps in
Detect Defects
Conformance to standards/spec
Requirements Transformation into product

Walkthroughs
•Code Walkthrough is a set of procedures and error detection techniques for group reading
•Like code inspection it is also an group activity
•In Walkthrough meeting, three to five people are involved. Out of the three, one is moderator, the second one is Secretary who is responsible for recording all the errors and the third person plays a role of Test Engineer

Desk Checking
•Human error detection technique
•Viewed as a one person inspection or walkthrough
•A person reads a program and checks it with respect to an error list and/or walks test data through it

Dynamic Testing

White Box Test Techniques
• Code Coverage
• Statement Coverage
• Decision Coverage
• Condition Coverage
• Loop Testing
• Code complexity
•Cyclomatic Complexity
• Memory Leakage

Black Box Test Techniques
• Equivalence Partitioning
• Boundary Value Analysis
• Use Case / UML
• Error Guessing
• Cause-Effect Graphing
• State Transition Testing

White Box Test Techniques

•White box is logic driven testing and permits Test Engineer to examine the internal structure of the program
•Examine paths in the implementation
•Make sure that each statement, decision branch,or path is tested with at least one test case
•Desirable to use tools to analyze and track Coverage
•White box testing is also known as structural, glass-box and clear-box

•White Box Test Techniques
• Code Coverage
Statement Coverage
Decision Coverage
Condition Coverage
Loop Testing
• Code complexity
• Memory Leakage

Code Coverage

•Measure the degree to which the test cases exercise or cover the logic (source code) of the program
•Types
• Statement Coverage
• Decision Coverage
• Conditional Coverage
• Loop Testing

Statement Coverage

•Test cases must be such that all statements in the program is traversed at least once
•Consider the following snippet of code
void procedure(int a, int b, int x)
{ If (a>1) && (b==0)
{ x=x/a; }
If (a==2 x>1)
{ x=x+1; }
}

Decision Coverage

Test cases must be such that each decision has a true and false outcome at least once. If we consider the same example as before, we need at least two test cases to execute the true and false outcome of the decisions at least once

Condition Coverage

•Test cases are written such that each condition in a decision takes on all possible outcomes at least once.

Loop Testing

•Loops testing is a white box testing technique that focuses exclusively on validity of Loop construct
•Types of loops
• Simple Loop
• Nested Loop
• Concatenated Loop
• Spaghatti Loop

Basis Path Testing

•Basis Path Testing is a white box testing technique that enables the test case designer to derive a logical complexity measure of a procedural design and use this measure as a guide for defining the basis set of execution paths.
• Test Cases derived to exercise the basis set are guaranteed to execute every statement in the program at least one time during testing.

Flow Graph

•Main tool for test case identification
•Shows the relationship between program segments , which is the sequence of statements having the property that if the first member of the sequence is executed then all other statements in that sequence will also be executed
•Nodes represent one program segment
•Areas bounded by edges and nodes are called regions
•An independent path is any path through the program that introduces at least one new set of processing statements or a new condition

Cyclomatic Complexity


•Cyclomatic Complexity is a software metric that provides a quantitative measure of logical complexity of a program
•When Used in the context of the basis path testing method, value for cyclomatic complexity defines number of independent paths in basis set of a program
•Also provides an upper bound for the number of tests that must be conducted to ensure that all statements have been executed at least once
•Cyclomatic complexity is often referred to simply as
program complexity, or as McCabe's complexity

Calculating Cyclomatic Complexity

•The cyclomatic complexity of a software module is calculated from a flow graph of the module , when used in context of the basis path testing method
•Cyclomatic Complexity V(G) is calculated one of the three ways:
1.V(G) = E - N + 2 , where E is the number of edges and N = the number of nodes of the graph
2.V(G) = P+1, where P is the number of predicate nodes
3.V(G) = R , where number of region in the graph

Cyclomatic Complexity – Risk Evolution

Cyclomatic Complexity
Risk Evaluation
1-10 A simple program without much risk
11-20 More Complex, Moderate Risk
21-50 Complex, High Risk Program
Greater than 50 Highly complex,Very high risk program

Memory Leak

•Memory leak is present whenever a program loses track of memory.
•Memory leaks are most common types of defect and difficult to detect
•Performance degradation or a deadlock condition occurs
•Memory leak detection tools help to identify
• memory allocated but not deallocated
• uninitialized memory locations

Black Box Test Techniques

•Black box is data-driven, or input/output-driven testing
•The Test Engineer is completely unconcerned about the internal behavior and structure of program
•Black box testing is also known as behavioral, functional, opaque-box and closed-box

Black Box Test Techniques

Tests are designed to answer the following questions:
•How is functional validity tested ?
•What classes of input will make good test cases?
•Is the system particularly sensitive to certain input values?
•What effect will specific combinations of data have on system operations?

Black Box Test Techniques

•Equivalence Partitioning
•Boundary Value Analysis
•Error Guessing
•Cause Effect Graphing
•State transition testing

Equivalence Partitioning

•This method divides the input domain of a program into categories of data for deriving test cases.
•Identify equivalence classes - the input ranges which are treated the same by the software
- Valid classes: legal input ranges
- Invalid classes: illegal or out of range input values
•The aim is to group and minimize the number of test cases required to cover these input conditions

Assumption:
•If one value in a group works, all will work
One from each partition is better than all from one
•Thus it consists of two steps:
Ø Identify the Equivalence class
Ø Write test cases for each class

Examples of types of equivalence classes
•1. If an input condition specifies a continuous range of values, there is one valid class and two invalid classes
Example: The input variable is a mortgage applicant’s income. The valid range is $1000/mo. to $75,000/mo.
- Valid class: {1000 > = income < = 75,000} - Invalid classes: {income <> 75,000}

2. If an input condition specifies that a variable, say count, can take range of values(1 - 999),
Identify - one valid equivalence class (1 <>999)

3. If a “must be” condition is required, there is one valid equivalence class and one invalid class
Example: The mortgage applicant must be a person.
- Valid class: {person}
- Invalid classes:{corporation, ...anything else...}

Example
If we have to test function int Max(int a , int b) the Equivalence Classes for the arguments of the functions will be
Arguments Valid Values Invalid Values
A -32768 <= Value <= 32767 < - 32768 , >32767
B -32768 <= Value <= 32767 < - 32768 , >32767

Boundary Value Analysis

•“Bugs lurk in corners and congregate at boundaries …..”
Boris Beizer
•Boundary Conditions are those situations directly on, above, and beneath the edges of input equivalence classes and output equivalence classes.
•Boundary value analysis is a test case design technique that complements Equivalence partitioning
•Test cases at the boundary of each input Includes the values at the boundary , just below the boundary and just above the boundary

From previous example, we have the valid equivalence class as (1 < count="0," count="1,count=" count="998,count=" count="1000">

Error Guessing

•Based on experience and intuition one may add more test cases to those derived by following other methodologies.
•It is an ad hoc approach
•The basis behind this approach is in general people have the knack of “smelling out” errors

Error Guessing

•Make a list of possible errors or error-prone situations and then develop test cases based on the list.
•Defects’ history are useful. Probability that defects that have been there in the past are the kind that are going to be there in the future.
• Some examples :
• Empty or null lists/strings
• Zero occurrences
• Blanks or null character in strings
• Negative numbers

•Example : Suppose we have to test the login screen of an application. An experienced test engineer may immediately see if the password typed in the password field can be copied to a text field which may cause a breach in the security of the application.
•Error guessing testing for sorting subroutine situations
- The input list empty
- The input list contains only one entry
- All entries in the list have the same value
- Already sorted input list

Cause Effect Graphing

•A testing technique that aids in selecting, in a systematic way, a high-yield set of test cases that logically relates causes to effects to produce test cases.
•It has a beneficial side effect in pointing out incompleteness and ambiguities in specifications.
• Steps:
- Identify the causes and effects from the specification
- Develop the cause effect diagram
- Create a decision table.
- Develop test cases from the decision table.

Insurance policy renewal example
•An insurance agency has the following norms to provide premium to its policy holders
•If age<=30 and no claim made premium increase will be 200 else 500 •For any age if claims made is 1 to 4 then premium increase will be 1000 •If one or more claims made then warning letter, if 5 or more claims made then cancel policy

State Transition Testing

A testing techniques that aids to validate various states when an program moves from one visible state to another.
Menu System Example
•The program starts with an introductory menu. As an option is selected the program changes state and displays a new menu. Eventually it displays some information , data input screen.
•Each option in each menu should be tested to validate that each selection made takes us to the state we should reach next