Have you used ScalaCheck testing library before?
Not only can you do Property-based testing but you can also do Law testing which is even more powerful and can take your programming to the next level. Want to know how?
Check out this article by Scala Developer Daniel Sebban.
'I have been using ScalaCheck testing library for at least 2 years now. It allows you to take your unit tests to the next level.
- You can do Property-based testing by generating a lot of tests with random data and asserting properties on your functions. A simple code example is described below.
- You can do Law testing that is even more powerful and allows you to check mathematical properties on your types.
Property-based testing
Here is our beloved 'User' data type:
case class User(name: String, age: Int)
And a random 'User' generator:
import org.scalacheck.{ Gen, Arbitrary }
import Arbitrary.arbitrary
implicit val randomUser: Arbitrary[User] = Arbitrary(for {
randomName <- Gen.alphaStr
randomAge <- Gen.choose(0,80)
} yield User(randomName, randomAge))
We can now generate a 'User' like this:
scala> randomUser.arbitrary.sample
res0: Option[User] = Some(User(OtwlaaxGbmdhuorlmgvXitbmGfbgetm,22))
Let’s define some functions on the 'User':
def isAdult: User => Boolean = _.age >= 18
def isAllowedToDrink : User => Boolean = _.age >= 21
Let’s claim that:
All adults are allowed to drink.
Can we somehow prove this? Is this correct for all users?
This is where property testing comes to the rescue. It allows us not to write specific unit-tests. Here they would be:
- 18-year-olds are not allowed to drink
- 19-year-olds are not allowed to drink
- 20-year-olds are not allowed to drink
All of these statements can be replaced by a single property check:
import org.scalacheck.Prop.forAll
val allAdultsCanDrink = forAll { u: User =>
if(isAdult(u)) isAllowedToDrink(u) else true }
Let’s run it:
scala> allAdultsCanDrink.check()
! Falsified after 0 passed tests.
> ARG_0: User(,19)
It fails as expected for a 19-year-old.
Property testing is awesome for a few reasons:
- Saves time by writing less specific tests
- Finds new use cases generated by Scala check that you forgot to handle
- Forces you think in a more general way
- Gives you more confidence for refactoring than conventional unit tests
Law testing
It gets better: let’s take it to the next level and define an Ordering between Users:
import scala.math.Ordering
implicit val userOrdering: Ordering[User] = Ordering.by(_.age)
We want to make sure that we didn't forget any edge cases and that we defined our order properly. This property has a name, and it’s called a total order. It needs to holds for the following properties:
- Totality
- Antisymmetry
- Transitivity
Can we somehow prove this? Is this correct for all users?
This is possible without writing a single test!
We use 'cats-laws' library to define the laws we want to test on the ordering we defined:
import cats.kernel.laws.discipline.OrderTests
import cats._
import org.scalatest.FunSuite
import org.typelevel.discipline.scalatest.Discipline
import org.scalacheck.ScalacheckShapeless._
class UserOrderSpec extends FunSuite with Discipline {
//needed boilerplate to satisfy the dependencies of the framework
implicit def eqUser[A: Eq]: Eq[Option[User]] =
Eq.fromUniversalEquals
//convert our standard ordering to a `cats` order
implicit val catsUserOrder: Order[User] = Order.fromOrdering(userOrdering)
//check all mathematical properties on our ordering
checkAll("User", OrderTests[User].order)
}
Let’s run it:
scala> new UserOrderSpec().execute()
UserOrderSpec:
- User.order.antisymmetry *** FAILED ***
GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation.
(Discipline.scala:14)
Falsified after 1 successful property evaluations.
Location: (Discipline.scala:14)
Occurred when passed generated values (
arg0 = User(h,17),
arg1 = User(edsb,17),
arg2 = org.scalacheck.GenArities$$Lambda$2739/1277317528@41d7b4cf
)
Label of failing property:
Expected: true
Received: false
- User.order.compare
- User.order.gt
- User.order.gteqv
- User.order.lt
- User.order.max
- User.order.min
- User.order.partialCompare
- User.order.pmax
- User.order.pmin
- User.order.reflexitivity
- User.order.reflexitivity gt
- User.order.reflexitivity lt
- User.order.symmetry
- User.order.totality
- User.order.transitivity
Sure enough, it fails on the antisymmetry law! Same age and different names are not supposed to be equals. We forgot to use the name in our original 'Ordering', so let's fix it and rerun the laws:
implicit val userOrdering: Ordering[User] = Ordering.by( u => (
u.age, u.name))
scala> new UserOrderSpec().execute()
UserOrderSpec:
- User.order.antisymmetry
- User.order.compare
- User.order.gt
- User.order.gteqv
- User.order.lt
- User.order.max
- User.order.min
- User.order.partialCompare
- User.order.pmax
- User.order.pmin
- User.order.reflexitivity
- User.order.reflexitivity gt
- User.order.reflexitivity lt
- User.order.symmetry
- User.order.totality
- User.order.transitivity
And now it passes
If you are wondering what can you test besides 'Order's, go check out the docs here: https://typelevel.org/cats/typeclasses/lawtesting.html
Summary
- Property tests are more powerful than unit tests. They allow us to define properties on functions and generate a large number of tests using random data generators.
- Law testing takes it to the next level and uses the mathematical properties of structures like 'Order' to generate the properties and the tests.
- Next time you define an ordering and wonder if it’s well-defined, go ahead and run the laws on it!'