Type class derivation with Shapeless: An introduction by Basement Crowd

At Basement Crowd we make use of Shapeless for a couple of problems – the most common use is for the automatic derivation of JSON de/serialisers for our APIs (spray-json-shapeless/circe), but it also comes into play for test data generation (scalacheck shapeless) and its polymorphic functions.

This article will attempt to give an introduction to some aspects of Shapeless and how it can help with automatic type class derivation (a feature that is at the heart of how circe and scalacheck can automatically handle case classes). To discuss this, we will consider the problem of generating test data for unit tests, already solved for us with scalacheck, but this gives us an understandable problem context to discuss the solution.

Generating test data with type-classes

Let’s say we have some case classes in our code, and we need to generate a bunch of random instances of them so we can use them in unit tests. We could approach this by defining a type-class for our generator and have an implementation of the generator for each of our case classes. For example:

1  trait Generator[A] {
2   def gen: A
3  }
4
5  case class Capybara(name: String, age: Int, awesome: Boolean)
6
7  object Generator {
8   import scala.util.Random
9   def generate[A](implicit gen: Generator[A]): A = gen.gen
10  implicit val capybaraGenerator = new Generator[Capybara] {
11    override def gen: Capybara = Capybara(
12      name = Random.nextString(10),
13      age = Random.nextInt(50),
14      awesome = Random.nextBoolean()
15    )
16  }
17 }

As you can see, we define our parameterised trait Generator[A] with a single generate method, and then in our companion object we have a helpful entrypoint method “generate” that will look for implicit type-class implementations available in-scope. We have also defined our Capybara implementation of the type-class, this means if we call Generator.generate[Capybara] we get a randomly generated Capybara instance!

The type-class approach provides a pretty good pattern for generating test data, which is all good and well, but it’s pretty verbose and seems like it will be a lot of boilerplate once we add all our case classes.

An obvious and easy first step to reduce the boilerplate is to pull out the simple type generation into their own Generator implementations, so they can be common, and can be re-used across case classes. For example:

1  case class Capybara(name: String, age: Int, awesome: Boolean)
2  case class Dog(name: String, age: Int, awesome: Boolean)
3
4  object Generator {
5    import scala.util.Random
6    def generate[A](implicit gen: Generator[A]): A = gen.gen
7
8    implicit val intGenerator = new Generator[Int]{
9      override def gen: Int = Random.nextInt(50)
10   }
11   implicit val stringGenerator = new Generator[String]{
12     override def gen: String = Random.nextString(10)
13   }
14   implicit val booleanGenerator = new Generator[Boolean]{
15     override def gen: Boolean = Random.nextBoolean
16   }
17   implicit val capybaraGenerator = new Generator[Capybara] {
18     override def gen: Capybara = Capybara(generate[String], generate[Int], generate[Boolean])
19   }
20   implicit val dogGenerator = new Generator[Dog] {
21     override def gen: Dog = Dog(generate[String], generate[Int], generate[Boolean])
22   }
23 }

Now we have a generator defined for our simple types, we can easily define a generator for our case classes Capybara and Dog, but thats still boilerplate that is going to grow linearly with the number of case classes we have.

Enter Shapeless

To avoid this increase in boiler plate, we would ideally have some clever method that lets us pass in any case class, and as long as we have implicit Generators for all the member types in scope, it generates our class, without the need for the explicit boilerplate linking them up.

A case class represents a product (in the functional programming, algebraic data type sense), that is, case class Capybara(name: String, age: Int, awesome: Boolean) is the product of name AND age AND awesome – that is, every time you have an instance of that case class you will always have an instance of name AND age AND awesome. Of course, as well as being a product of its members, a case class also carries semantic meaning in the type itself – that is, for this instance as well as having these three attributes we also know an additional piece of information that this is a Capybara. This is of course super helpful, and central to the idea of a type system, but maybe sometimes (as in this case), we want to be able to just generically operate on that case class without being tied to the specific type – and that’s what Shapeless provides – It provides a structure called a HList (a Heterogeneous List) that allows you to define a type as a list of different types, but in a compile time checked way.

Rather that a standard List in Scala, where we define the type that the list will contain, with a HList we can define the list as a sequence of specific types, for example:

1 val capybara: String :: Int :: Boolean :: HNil

The above example allows us to define out HList with specific types, which provides our compile time safety – the :: notation is some syntactic sugar provided by Shapeless to mirror normal Scala list behaviour, and HNil is the type for an empty HList.

Hopefully, it might already be coming clear as to how this can help – if we can convert any case class into a common generic format that represents the member types, then it could be useful for our boilerplate.

Fortunately, Shapeless provides a class called Generic[A]. Shapeless generates instances of this class through compile time macros for all case classes (and sealed traits), so with the addition of the correct import you can bring into scope a Generic[A] for any arbitrary case class you have, for example:

1 import shapeless._
2 val genericCapybara = Generic[Capybara]

From there, we can use this class to convert any instance of our case class to a HList and back again:

1 val hlistCapy = genericCapybara.to(Generator.generate[Capybara])

Our generic type-class

1 implicit def caseClassGenerator[T, L <: HList](implicit generic: Generic.Aux[T, L], lGen: Generator[L]): Generator[T] =
2  new Generator[T] {
3    override def generate = generic.from(lGen.generate)
4  }

1 implicit def hnilGenerator = new Generator[HNil] {
2  override def generate = HNil
3 }
4
5 implicit def hlistGenerator[H, T <: HList](implicit headGen: Generator[H], tailGen: Generator[T]) =
6  new Generator[H :: T] {
7    override def generate = headGen.gen :: tailGen.gen
8  }