Check out this article from Michał Lebida on Updatable Scala config which is an enhanced version of pure config with the ability to dynamically update configuration from multiple backends. Happy learning!
'Scala-updatable-config library is an enhanced version of pure config with ability to dynamically update configuration from multiple backends. Best part of our tool is that there is no need to redeploy application if you want to change some basic configuration.
Great help with writing this library was an awesome book about shapeless library “The Type Astronaut’s Guide to Shapeless”.
Project is open source and you can find it here.
But why?
Some of you may ask why we tried to reinvent the wheel, and write some ‘yet another configuration library’. Reason is quite simple, we didn’t want go through redeploy process just because there was a need to change something in configuration. Additionally, from security reason, we didn’t want to use environment variables.
We achieve that by using distributed key-value stores, Consul and Vault.
Consul is service discovery component where we register services and keep simple configuration for them. Vault is tool for storing all kind of secrets, from simple key-value to managing credentials to database.
How to use it?
Best way of describing how something works is by showing it.
Let’s start with defining ‘application.conf’ file:
azimo {
cfg {
use-only-file = false
}
consul {
host = "consul.host"
host = ${?CONSUL_IP4_ADDR}
port = 8500
port = ${?CONSUL_PORT}
}
vault {
host = "vault.host"
host = ${?VAULT_IP4_ADDR}
port = 8200
port = ${?VAULT_PORT}
service-name: "SERVICE-NAME"
}
externalservice {
address: "localhost"
endpoint: "/endpoint"
port: 8000
}
externalservice2 {
address: "localhost"
endpoint: "/endpoint"
port: 8000
}
}
What is worth noticing is the following line
azimo.cfg.use-only-file
It’s helpful for testing purposes, with that set on true library will look just into file to get values.
ADT for above config file:
import com.azimolabs.config.definitions.{ConsulConfiguration, VaultConfiguration}
case class ApplicationConfig(
cfg: ConfigUsage,
consul: ConsulConfiguration,
vault: VaultConfiguration,
externalservice: ExternalServiceConfiguration,
externalservice2: ExternalServiceConfiguration
)
case class ConfigUsage(useOnlyFile: Boolean = false)
case class ExternalServiceConfiguration(
address: String,
endpoint: String,
port: Int
)
Now we are ready to define object with configuration.
import akka.actor.ActorSystem
import com.azimolabs.config.definitions.AzimoApplicationConfig._
import com.azimolabs.config.definitions.{ConsulConfiguration, VaultConfiguration}
import pureconfig._
import eu.timepit.refined._
import eu.timepit.refined.pureconfig._
import eu.timepit.refined.api.Refined
import eu.timepit.refined.numeric._
import eu.timepit.refined.string.{MatchesRegex, Uri}
import cats.instances.all._
import com.azimolabs.config.{ConsulAdapter, UpdatableConfiguration, VaultAdapter}
import scala.concurrent.ExecutionContext
import scala.util.Success
object ApplicationConfig {
implicit val actorSystem: ActorSystem = ActorSystem.create()
implicit val executionContext: ExecutionContext = actorSystem.dispatcher
private val updatableConfig = UpdatableConfiguration[ApplicationConfig]
private val fileConfiguration: ApplicationConfig = loadConfig[ApplicationConfig]("azimo") match {
case Left(err) => throw new RuntimeException(err.toString)
case Right(conf) => conf
}
private val vaultConfigGetter: VaultAdapter = VaultAdapter.defaultVaultAdapter(fileConfiguration.vault)
private def vaultConfiguration = updatableConfig.checkForUpdates(List(""), vaultConfigGetter, _ => ())(fileConfiguration)
def consul = new ConsulAdapter(fileConfiguration.consul)
val configuration: ApplicationConfig = if (fileConfiguration.cfg.useOnlyFile) {
fileConfiguration
} else {
updatableConfig.checkForUpdates(List("example-project-group", "service-name"), consul, _ => ())(fileConfiguration)
}
// If you want to get e.g. credentials for PostgreSQL:
private val postgresCredentials = vaultConfigGetter.getCredentials(UpdatableConfiguration.ValuePath(path = List("credentials_path_here")))
val postgresUsername: String = postgresCredentials.map(_.username).getOrElse(configuration.db.default.user)
val postgresPassword: String = postgresCredentials.map(_.password).getOrElse(configuration.db.default.password)
// If you want to get information about external service from consul
def haproxy: Option[(String, Int)] = {
consul.serviceAddress("haproxy").map(c => c.getServiceAddress -> c.getServicePort)
}
def externalService(): ExternalServiceConfiguration = {
haproxy.map(item => configuration.externalservice.copy(
address = item._1,
port = item._2,
)).getOrElse(configuration.externalservice)
}
}
Most of library magic is hidden behind method 'checkForUpdates' where we defined:
- Path to ‘key‘’.
- ValueGetter which try to return value under path.
- Callback function which is called when value in path changed.
- Current value used when there is no change or we can’t find path.
And that’s all, with that configuration every time you reload you application or run `checkForUpdates`on ApplicationConfig you will get new values from backend.
There is room for improvement!
At this point we support only two backends, but with our approach we can easily add more like Apache Zookeeper or Etcd. In addition, we can certainly improve support for coproducts.
What is important to us is better utilization of Vault backend. Currently we support only token for predefined AppRole but there is much more we can use.
Final words.
With our approach to dynamic application configuration you can simplify way of configuring application. It became standard in our Scala applications and I encourage you to try it out.
We are looking forward to any feedback, pull requests, issue requests.
This article was written by Michał Lebida and posted originally on medium.com