Data Engineer – Scala – Spark – Kafka – AWS

Job Description

Senior Data Engineer – Scala – Spark – Kafka – AWS

This role provides the opportunity to work on the following tech stack: Scala, Spark, Kafka, AWS & Docker/Kubernetes.

The company:

This company is evolving alongside personal mobility as more vehicles are fleet-owned and Insurance must adapt to the changing risk profile. Consumer decisions & preferences are forcing underwriting models to be transformed. Increasing usage of technology has created a torrent of new data which enhances the predictive power of modern AI-driven applications. Their core product utilizes streaming telematics data and amends insurance pricing in real-time responding to the drivers' needs, environment, behaviour and usage.

The role:

They have built their working environment around the people and not the skills required (although they are important). They believe heavily in open source software, idea sharing & open collaboration throughout all development teams. They need people who share the same values and know how important they are to the success of the project.

Benefits:

• Competitive salaries
• Options scheme and will benefit from being an early joiner
• Work from the office, from home or anywhere!
• Very flexible and generous holiday policy
• Opportunity to learn and develop skills within a modern working environment.

Key Skills:

• Spark/Flink/Kafka automated testing experience.
• Scala, Java. Functional programming experience. Python experience a plus.
• AWS frameworks.
• Kubernetes/Docker – Microservices.
• Jenkins, Maven or Gradle, Git, Ansible.
• Prometheus, Grafana.
• Experience developing or testing microservices, especially data-centric ones.
• Interest in systems architecture and building distributed systems at scale.
• Strong automation mindset and a passion for root cause analysis.
• Expertise in performance tuning and service monitoring.

Nice to haves:

• Event sourcing / CQRS patterns
• Timeseries data stores (Influx DC, CrateDB)
• NoSQL databases (Cassandra or HBase)
• Modern data warehousing (Hive, Kylin or Presto)
• Indexing and search engines (Elasticsearch)