Skip to content
Advertisement

Hypothesis library: strategy for the complement of some other strategy

I’m using the Hypothesis library for unit testing. With this library you don’t hand-pick inputs, but you define the complete set of inputs for which you want to test. Hypothesis will then sample from this set to look for inputs that break the function. This is also known as property-based testing. In Hypothesis, these sets are called strategies.

Now I would like to unit test a function that validates some input:

GRIDSIZE_FORMAT = "^[0-9]+x[0-9]+$"
CELL_FORMAT = "^[A-Z]+[0-9]+$"

def _validate(gridsize, walls, entrance):
    if not re.match(GRIDSIZE_FORMAT, gridsize):
        raise ValueError(f"grid size '{gridsize}' does not match format '{GRIDSIZE_FORMAT}'")
    for wall in walls:
        if not re.match(CELL_FORMAT, walls):
            raise ValueError(f"wall '{wall}' does not match format '{CELL_FORMAT}'")
    if not re.match(CELL_FORMAT, entrance):
        raise ValueError(f"entrance '{entrance}' does not match format '{CELL_FORMAT}'")

To properly test this function I want to generate examples of the form “anything but X”, X being the proper input format for this function.

Is there a strategy in the Hypothesis library that can generate inputs like this?

Advertisement

Answer

Unfortunately Hypothesis cannot compute the complement of a strategy, because strategies can be composed with arbitrary user-supplied code (including side-effects!). For example, what should be the complement of hypothesis.extra.django.from_model(User), which inserts the generated instance into the database? (invalid, not in the database, …)

In more specific cases, you can instead take the complement of your specification and then derive a strategy from that complement instead. Your regex trick is a good example – explicit set-complement tends to make this more efficient than the naive “generate something right-ish, and filter out valid instances” approach.

Advertisement