Test Case Prioritization For Black Box Testing Analysis Essay Example

based system, black box testing, Test Case prioritization technique, Criticality of Component, Fault proneness, Component Dependency Density

Acknoledgement

First, I offer my sincerest gratitude to my supervisor, Associate Professor Richard Lai, who has supported me throughout my thesis with his patience and knowledge. In addition, his constant feedback, encouragement, guidance and support at every phase of writing this thesis from the initial to the final level enabled me to develop an in-depth understanding of the research area. Second, I am thankful to Associate Professor Ben Soh, who has encouraged me for this thesis, provided me the guidance in thesis writing and empowered my critical reading and writing skills.

The greatest challenges in component-based software development are that suitable ready-made components are difficult to find, and if they are found, they are not necessarily of good quality. In addition, components have different dependencies between them, source code is seldom available, and the target context of the components is unknown.

The component developer tests the component for its correct functionality and implementation logic via unit testing. When, system integrator integrates the system combining number of components, the focus of testing is the required functionality supposed to provide by the component.

Integrator tests the functional aspect of subject component. System Integration tester uses integration testing and system testing methods to find the faults.

In CB, testing, main focus is on interfaces, which are only external parts of components. Interface supposedly suffers from component synchronization problems, coupling complexity and runtime events generated by the integration of components. In previous work, a number of techniques were proposed for testing component-based systems, among which most are code based or model based. Same time, testing

is the most time and resource consuming process in any software development.

To reduce the cost of testing without decreasing the effectiveness is the critical management problem for today’s entrepreneurs. Answer is to prioritize the test cases so, which are more important run earlier in test process. There can be different goals of the test case prioritization such as increasing reliability or performance of system, reducing cost per test case. Based on the aim of testing, the prioritization factors are selected and test cases are sort out using those factors. Actually, regression testing is the most time /cost consuming process, but important aspect of testing.

Many techniques were proposed in OO domain to prioritize the test cases for regression testing. Those techniques are code based and difficult to apply in component based development environment. In OO, measure of factors directly related to complexity and fault severity are used, where as in CBS we hardly have such measures, so we have to depend on the indirect measures such as fault proneness of component or complexity of integration. In CB-system, it is obvious that as the number of interfaces used for component integration increases the complexity of system integration increases.

One component couples with more than one component to provide specific functionality.

As the dependency of component is higher, the component is more prone to fault. Fault severity is internal characteristic of component dependency. Here, this paper proposes a test case prioritization technique in component environment. The factors I am using are criticality of component and fault proneness of component.

Based on above factors the metrics/equation of criticality of component is proposed to measure the importance of component for

testing purpose.

Case study is provided to evaluate effectiveness of techniques. The paper is organized as below: section 1 and 2 provides the introduction and background related to object oriented techniques and their limitation compared to component based system development techniques. Section 3 presents survey of previous techniques for test case prioritization based on applied prioritization factors. Section 4 describes the various complexity metrics for component based system and makes comparison between them.

Section 5 compares the various techniques and metrics to measure the fault proneness.

Section 6 presents the proposed technique for test case prioritization in component based black box testing. New matrix called Dependency density of Component (DDcomp) is proposed and based on that proposed technique is described which is based on two prioritization factors namely: 1) Criticality of Component and 2) Fault Proneness of Component interactions. Section 7 presents the experimental case study for validation of proposed technique.

Section 7. 1, 7. 2, and 7. 3 orderly present the system under test, experiment setting/ factor collection process and analysis of result.

At last, section 8 presents the conclusion and future work.

Background

Object oriented technology and software design principals entered in the mainstream software development to mirror /mimic the behaviour of the complex real word systems outside. The root principals of the OO techniques are incorporating states and behaviour of the subject system. Therefore, it heavily rely on the how objects are connected with each other rather than Why? Examples relations are associations, inheritance, polymorphism, and uses, which most of the times, shadows the understating of why object are relates to each other.

In Response to above phenomenon , the Object technology abstract out

the continuous nature of real world systems by incorporating time and space in discrete quantities.

Component based technology, a subset, special case of Object oriented technology views each logical particle of the software system as a unique component. Component can be a class, object, module or subsystem, which functions independently irrespective of the other component of the system, easing the development process of large complex software. We can build the system by combining the components from different venders at reduced cost and time.

However, the views of the both techniques are different by nature: an object technique relay on relation between objects and is white box in nature. Contrast CB systems, which relays on functionality of the component, making the component as black box.

Due to unavailability of source code, the white box testing of COTS –component is nearly impossible that is why developer has to depend on functional black box testing. There are numbers of methods, techniques and, frameworks for testing both OO and CB systems, which are used at various level of software development process such as requirement capturing, design and coding.

Still, the lack of methods and techniques combining both techniques in a single domain create hurdles when testing the software. Here, in this paper, I am trying to contribute to solve the above problem by applying test case prioritization method using the factors influencing the correctness and performance of software.

Limitation Of Object Oriented Techniques

Object oriented software development framework is the general methodology now day people are using to build each type of software products because of its principal of abstractness and modularity.

When using in the special cases such as embedded real-time systems

or component based systems ,which has special problem space and development requirements the limits of OO technique come in to focus and developer has to work keeping in the mind the limitations of Object oriented techniques. The real world behave as continuum[1]. Time and space are the un-abstract-able parameters in real world systems. The status of the system at particular time instance depends on the one or more events occur simultaneously on component /object, so it is relative in nature [2].

Opposite of above fact, the OO technique cannot capture the object’s continuous behaviour at flow of time. For Examples , there are number of observations noted in the papers[1] such as interaction of electric and magnetic field, thermal conductivity in thermodynamics , and gyroscopic effects on spinning objects , which lights the limitations of object techniques. Focusing on all above examples, component based techniques most suites the problem characterization, where the completeness of integration is most important.

CB technique provides us not fully but partial flexibility to define relative time measures at interaction of component, but it is out of focus of the paper here. Adding to this, when modelling the OO-system , OO-models, both static and behavioural, focuses on system modelling issues, but researches and surveys indicated that the development processes are equally as prevalent as modelling difficulties[3].

Again, the Object oriented technique is by nature white box, source code availability allows us to test how object interacts with one another.