The sun was shining but we were keeping cool at Scala in the City!
The ice cream, cold beer and Scala talks were flowing last night at Vrbo and we are thrilled to have held another awesome meet-up. Thank you again to Vrbo for hosting us, part of the Expedia Group they are the world-leading company within their industry and it was really interesting to hear how they stay ahead with technology being at the forefront of everything they do. Jeff Martin, Staff Scala Engineer at Vrbo gave us an insight into how they use Scala Stewards to manage technical debt and maintain development velocity, stay posted to watch his talk!
We're so pleased to have had Gerard Maas, Lightbend Principal Engineer come all the way from Belgium to present for us and we hope you all enjoyed his talk. Attendees were also lucky enough to gain some free resources which Gerard brought along from Lightbend as well as one lucky winner receiving a copy of his book!
Thank you again to everyone who came along, despite the warm weather we all had a great evening.
If you didn't get the chance to come out then not to worry as we have Gerard Maas's talk ready for you to watch as well as his slides.
Stay posted to see where Scala in the City will be next month, you can subscribe to our Meetup page here.
Scala in the City Overview
Kubernetes as a Streaming Data Platform with Kafka, Spark, and Scala
Abstract: Kubernetes has become the de-facto platform for running containerized workloads on a cluster. Its native abstractions let us deploy and manage individual distributed applications and services.
In contrast, streaming applications require the coordinated deployment of different components such as topics, storage, and application runtimes that need to work in unison.
In this talk, we are going to discuss how to use Kubernetes Operators to manage this complexity. In particular, we will see:
- How to use the Kafka Operator to manage topics
- How to use the Spark Operator to deploy and manage Spark Structured Streaming applications
- How a custom operator can harness Kubernetes resources and other operators to implement the operational requirements of our applications.
We will share our experience building an operator for streaming data pipelines using Scala and show how this all works in real life.