Property-Based Testing in a Project

What we need now is a framework. As opposed to many testing practices that require a tiny bit of scaffolding and a lot of care, property-based testing is a practice that requires heavy tool assistance. Without a framework, we have no way to generate data, and all we’re left with are encoded rules that don’t get validated. If we use a framework that doesn’t generate great data or doesn’t let us express the ideas we need, it will be very difficult to generate tests of the same quality that we could using a functional framework.

As the course title implies, we will use PropEr. It can be used by both Erlang and Elixir projects and integrate with the usual build tools used in both languages. There are other frameworks available, namely QuickCheck framework by Quviq and Triq. These two frameworks and PropEr are similar enough that if our team is using any of them, we’ll be able to follow along with the text without a problem. This remains true if we’re using Elixir. We might have heard about StreamData, a property-based testing framework exclusive to Elixir. The concepts should not be hard to carry over from framework to framework in any case.

PropEr is usable on its own through either manual calls or command line utils. This is fine for some isolated tests and quick demos, but it’s usually nicer to be able to get the framework to fit our everyday development workflow instead.

mix: Elixir build tool

For Elixir, we will use mix as our build tool. By adding the propcheck package to our project configuration, mix will be able to find and execute PropEr properties within the same files as those that contain standard Elixir test cases written in ExUnit, the language’s default test framework.

Fear Not the GPLv3 License:
PropEr is licensed under the GPLv3. People are often worried about having to attach and ship GPL-licensed code with their projects. Placing the tests and dependencies related to PropEr in their directory means that they are only used as development tools in a testing context and never in production.
As such, whenever we build a release with code to ship and deploy, none of the test code nor PropEr itself will be bundled or linked to our program. This tends to put most corporate lawyers at ease without preventing the authors from getting contributions back in case the tool is modified or used at the center of a commercial product.

Setting up a new project

We’ll start by setting up a standard project:

mix new new_proj

This will create a new sample project for Elixir. Move into the project directory using cd new_proj, and we can get started. Try using the commands in the terminal provided below.

Adding the PropEr dependency

Within that project, we again have to add the propcheck dependency. We’ll add it into the mix.exs file in the root directory of the project. Here is how we add the dependency:

Note: This has already been done in the code widget provided below.

defp deps do 
[
    {:propcheck, "~> 1.1", only: [:test, :dev]} 
]
end

This makes PropEr available to the tool, and also allows properties to be perceived like regular ExUnit unit tests. As such, there’s no need to use special commands, just the regular ones we may be used to. To get going we just have to remember to fetch the dependencies using the command:

mix deps.get

Running the project

The command we use to test Elixir projects in a terminal is:

mix test

Let’s call it in the code widget below and check its output.

Note: To execute the code after changing it, click “Run” which will save our code, and then run mix test in the terminal

Note: Don’t be worried if the terminal shows a bunch of unfamiliar code. That is just the platform getting the dependencies to run the code.

We’ve now learned how to set up Erlang projects with PropEr to use property-based testing. We can now move on to running properties.