What is boundary value analysis?

What is boundary value analysis?

Boundary value analysis (BVA) is a black-box test design technique that places test inputs on each limit of a range and one step past it. For a cart that allows 1 to 20 items, it tests 0, 1, 20, and 21. One of those tests fails when a limit in the code is off by one, e.g. < where <= belongs.

The International Software Testing Qualifications Board (ISTQB) defines boundary value analysis as a black-box test technique whose test conditions are boundary values. A boundary value is the minimum or maximum of an ordered equivalence partition, a group of values that the software should treat the same way. So 21 is a boundary value too, as the lowest value of the invalid partition of 21 or more items. Only the boundary values inside the allowed range, here 1 and 20, are edge cases.

The ISTQB Foundation syllabus teaches BVA with equivalence partitioning, decision table testing, and state transition testing. It says developers are more likely to make errors at boundary values. In software testing, a few tests at the limits of a range can find bugs that many tests with typical values miss.

How does boundary value analysis work?

A tester applies boundary value analysis to one input in five steps:

1. The tester picks an input whose values have an order, e.g. the number of items in a cart.
2. The tester splits the values into equivalence partitions. A simple range has one valid partition and one invalid partition on each side. A rule with tiers, e.g. shipping prices by weight, has one valid partition per tier.
3. The tester marks the boundary values, the lowest and highest values of each partition, where one partition meets the next.
4. The tester picks the test values. In 2-value BVA, they are each boundary value and its closest neighbor in the adjacent partition. In 3-value BVA, they are each boundary value and both of its neighbors.
5. The tester writes the expected result for each value from the requirement, and the test compares it with the actual result.

The syllabus says the typical defects that BVA finds are limits that the code puts above or below their intended place, or leaves out. It calls 3-value BVA more rigorous, because it can catch defects that the 2-value tests miss. At the cart's lower limit, 3-value BVA tests 0, 1, and 2.

Some textbooks and courses teach another set, often called normal BVA. For the cart, it tests 1, 2, 10, 19, and 20, which are the limits, their inner neighbors, and a typical value. Robustness testing adds 0 and 21, and worst case testing combines these values across several inputs.

What is an example of boundary value analysis?

Here is an illustrative example. Acme Co. sells furniture online, and its checkout accepts a cart of 1 to 20 items. A developer asks a coding agent to "Let customers edit their delivery address during checkout." The agent's change checks the cart size again when an address is saved, and the session code clears a cart that fails the check. The agent writes that check as 1 < count <= 20, and its own test with 10 items passes.

The developer applies BVA to the item count:

1. The developer splits the count into 3 partitions of 0 items, 1 to 20 items, and more than 20 items.
2. The boundary values are 0, 1, 20, and 21, and 2-value BVA tests each of them.
3. Each expected result comes from the rule, not the code, so 1 and 20 are allowed and 0 and 21 are refused.

The developer writes one parametrized unit test with one assertion, and pytest runs it for each value:

import pytest
from checkout import cart_size_ok


@pytest.mark.parametrize("count, allowed", [
    (0, False),   # highest value below the range
    (1, True),    # lowest allowed value
    (20, True),   # highest allowed value
    (21, False),  # lowest value above the range
])
def test_cart_size_boundaries(count, allowed):
    assert cart_size_ok(count) == allowed

The run fails at 1 and passes at the rest:

______________________ test_cart_size_boundaries[1-True] _______________________
>       assert cart_size_ok(count) == allowed
E       assert False == True
E        +  where False = cart_size_ok(1)
========================= 1 failed, 3 passed in 0.18s ==========================

A customer with one item who edits the address loses the cart. The address saved, but the cart emptied. The agent changes 1 < count to 1 <= count, and the 4 tests pass. This example is simplified. A real checkout would also need boundary tests for the delivery address length.

What changes when a coding agent writes the code?

A coding agent writes its tests from the same prompt and code as the feature, and a prompt often leaves limits unstated. Acme's agent tested 10 items, a value from the middle of the range.

An agent asked to add boundary tests after the code exists can take the limits from that code. Because the code says 1 < count, a generated test can put its lower pair at 1 and 2 and expect 1 to be refused. The test sits on the code's limit, not the requirement's, so it passes and records the bug as expected behavior. This is one reason AI-written tests can pass on broken code.

A practical adjustment is to write the boundary values and their expected results from the requirement before the agent starts. Keep those tests in a file the agent does not edit. Mutation testing can then check the rest of the suite. A boundary mutant, e.g. <= changed to <, survives when no test sits on the limit. PIT, a mutation tool for Java, makes this change by default with its conditionals boundary mutator.

What are the limits of boundary value analysis?

Boundary value analysis has four main limits:

• Ordered values only. BVA needs values with an order, e.g. a count. Text length has one, so a field of 1 to 100 characters gets tests at 0, 1, 100, and 101 characters. The kinds of character in an address have no order, so equivalence partitioning covers them instead.
• Limits nobody wrote down. A limit that exists only in the code, e.g. a database column that holds 255 characters, is missing from the requirement, so BVA does not pick it. Property-based testing and fuzzing generate many inputs and can reach such a limit.
• One input at a time. A failure that needs two inputs at their limits at once is a corner case. The 2-value and 3-value versions in the ISTQB syllabus treat each input on its own, so they leave such a case out. Pairwise testing and decision table testing choose combinations of inputs instead.
• Mistyped operators. The 2-value version can miss a wrong comparison. In the syllabus's example, x <= 10 written as x == 10 passes the 2-value tests at 10 and 11. The 3-value test at 9 is likely to catch it.

Passing boundary tests show that the software handles the chosen values, not every value between them.

How is BVA different from equivalence partitioning?

Equivalence partitioning is a black-box test design technique that splits inputs into groups the software should treat the same way and tests one value from each group. The ISTQB glossary lists partition testing as another name for equivalence partitioning, and many courses call it equivalence class partitioning. Boundary value analysis starts from the same partitions and tests where they meet, not one value from inside each.

The two techniques find different bugs. At Acme, equivalence partitioning could test 0, 10, and 25 items, and all three pass on the broken check. Boundary value analysis tests 1, and that test fails. Teams often use both, with equivalence partitioning alone for inputs that have no order.

FAQs

What is pairwise testing?

Pairwise testing is a black-box test technique that picks tests so that every pair of values from any two inputs appears together at least once. With more than two inputs, the set is usually far smaller than every combination. It covers combinations that boundary value analysis can leave out when it treats each input on its own.

What is decision table testing?

Decision table testing is a black-box test technique that lists combinations of conditions in a table, with the action the software should take for each. Each feasible combination becomes a test. The technique suits business rules, e.g. a discount that depends on both the cart total and a membership.

Does boundary value analysis work for text inputs?

Boundary value analysis works for text inputs through an ordered property of the text, e.g. its length. The tests sit at the shortest and longest allowed lengths and one character past each, so a required field also gets a test with the field left empty. Which characters the text holds has no order, so equivalence partitioning covers that.

Can property-based testing replace boundary value analysis?

Property-based testing does not replace boundary value analysis, and the two complement each other. A property test checks a rule across many generated inputs and can reach a limit that nobody wrote down. Boundary value analysis takes each stated limit from the requirement, so fixed tests check those exact values on every run.