Black box vs White box testing

Testing software is the fifth phase in the Software Development Life Cycle(SDLC). We test software to ensure safety, security, product quality, no bugs are present, usability, etc. In this article, we learn about black box and white box testing.

Table of contents.

1. Introduction.
2. Black box testing.
3. White box testing.
4. Comparison: Black box vs White box testing
5. Summary.
6. References.

Introduction.

In black-box testing, we perform tests without any knowledge of the internal functioning of the application. On the other hand in white box testing, the tester/developer understands the underlying logic of the system and therefore creates tests from that knowledge.

A practical example is the following. Picture a car, a mechanic and the car owner. In this case the car is the AUT(Application Under Test), the mechanic is a white box tester since he/she understands the underlying functioning of the application, and the car owner is the white box tester since he/she only knows what the application is supposed to do.

Black box testing.

This is software testing whereby we test an application without having any knowledge of the underlying logic.

Take, for example, using a web application without any knowledge of the logic and functionality. Another example is using a mobile phone or mobile application without knowing how it is built or how the components interact with each other. In this case, we just need to know how to make and receive calls, send messages, power it on and off, and other basic tasks.

In the above two cases the web application, mobile phone, and mobile application are referred to as AUTs, that is Application Under Test.

In black-box testing, we use automated testing tools that record and playback test cases. These tools are developed using scripting languages such as VB script, Perl, Javascript, and Python.

The following are black-box testing techniques:

• Equivalence partitioning
  Here we divide inputs into classes/groups based on the similarity of outcomes. This allows us to choose a single value from a group and test it, the result covers all inputs in that group. Therefore we have test coverage and at the same time reduce work and time spent.

• Error guessing
  The tester uses knowledge about the application to guess areas in the system that might be prone to errors. Examples include division by zero, input forms, and test fields validation.

• State transition testing
  Here we test the different states of the system that is being tested. We look for the changing states of the system that are triggered by different events/conditions and test them.

• Graph-based testing
  A graph is used to model relationships between objects. In software testing, we identify all objects making up a piece of software and use them to prepare a graph. Relationships between objects are identified and test cases are prepared to look for errors.

• Decision table testing
  We use this test when presented with logical relationships such as if-else conditional statements. Given such a statement, we create a decision table and prepare test cases based on the table.

• Comparison testing
  The tester compares different versions of the same application.

• Boundary value analysis
  A boundary refers to the value that is close to a limit which when reached the behavior of the system changes. In this analysis, valid and invalid inputs are tested to check for issues.

Commonly used tools for black-box testing are,

• Selenium
• Applitools eyes
• Coded UI test Builder
• Appium
• QTP

Advantages of black box testing

• No prior knowledge of the system is needed.
• Effective for large-scale applications.
• Testing can start immediately after development is complete.
• Bugs are identified early.

Disadvantages of black box testing

• Possible chances of missing edge cases due to lack of programming knowledge.
• Test coverage is impossible for large-scale applications.
• Given a limited time frame we might skip all possible inputs.

White box testing

Unlike black-box testing, in this type of testing, we need to have in-depth knowledge about the application. Aspects like data flow, programming syntax, etc.
Once we know the workings of the application, we can derive test cases based on that knowledge.

In most cases, the testers are mostly developers or professionals with programming experience.

An example of a white box testing scenario is taking a car to the garage. In this case, the mechanic is the whit box tester since he/she understands the system well enough to design test cases. The user is the black box tester since he/she only knows what the car is supposed to do and not how it is built. The car is the AUT(Application Under Test).

We perform white box testing to ensure all logical decisions are verified, all loops are executed within their bounds and all paths within a module are executed at least once.

Here we aim to find bugs originating from logical errors, design errors, and syntax errors.

The following are white box testing techniques:

• Statement coverage, the tester makes sure that all statements are executed at least once. In a language, a statement can be a loop, a conditional, a variable declaration, etc.

• Branch coverage, if-else statements have two conditions, first to verify a branch is true otherwise it is false. In branch coverage, we make sure that all branches are executed at least once.

• Path coverage, when it comes to complex systems or applications, we implement this technique as it is more effective than branch coverage. Here the tester ensures that all paths of the program are traversed at least once.

Commonly used tools for white box testing are:

• Veracode
• RCUnit
• Googletest
• Emma
• Eclemma
• CppUnit

Advantages of black box testing

• Advanced testing since we get into every detail of the software through coverages.
• It can be automated using the tools discussed above.
• We optimize source code while testing, therefore, killing two birds with one stone.
• The above point leads to less time needed for the development cycle. In these cases, a developer finds and solves bugs almost immediately.
• We also have a chance to reason on the implementation of the software since we understand every detail about it.

Disadvantages of black box testing

• All this takes work and therefore white box testing is very expensive as compared to black-box testing.
• Since the code changes a lot, automated tests are no longer useful. This is because as we change the code we need to also change what we are testing for otherwise, the test will break.
• In black-box testing we test as per the business document and therefore all functionalities must work for the test to pass, white box testing, on the other hand, cannot recognize missing functionalities as described in the requirements document.

Comparison: Black box vs White box testing

Black box testing	White box testing
No prior knowledge of the system is needed	Knowledge of the code and structure of the application is required
Done by professional testers	Done by professionals (programmers/ testers) with programming knowledge
Focuses on the application's functionality.	Focuses on the underlying code and its syntax.
Higher level testing.	Low-level testing i.e unit testing, integration testing.
A requirements specification document is needed.	Design documents, data flow diagrams, and flowcharts are needed.
We cannot find defects related to the code.	Here we can optimize the code and minimize errors.
Identification of errors after the code is developed.	Instant feedback.
Low test coverage since the tester is limited to the overall functionality of the application.	High test coverage since developers/ testers are familiar with the source code.
Less time-consuming.	Time-consuming
A lot of possibilities for test data therefore hard to identify the correct data.	We test a single part of the application and get the correct test data.
Only done from a graphical user interface	Testing is done early in development.

Summary.

Black box testing involves interacting with a system as a mystery box, meaning we don't have any knowledge of how it is built but know how it is expected to work.
In white-box testing we first understand the system then we can derive test cases based on that knowledge.

In white-box testing, branch coverage is more effective compared to statement coverage. Also, path coverage is more effective than branch coverage.