As a start-up, there are so many learnings along the way but this is what makes a business great! At Scala in the City, Head of Engineering, Jendrik Poloczek shared the hard requirements the engineering team of TokenAnalyst have to meet and how they achieve this.
If Blockchain is your biggest interest then check this out!
We're a London based startup and we track everything that happens on public blockchains such as Bitcoin and Ethereum - specifically around "who" is transacting - in order to bring more transparency into what really moves crypto-asset markets. This involves challenges such as how to identify exchange wallets, providing semantic meaning to identified on-chain value flow behaviour and putting it into the bigger context of the fast-paced crypto environment.
On the engineering side, we face hard requirements such as resilient exactly-once data pipelines for data lakes, on-demand queries, on-line model training and application for address labels, and minimizing the amount of blockchain-specific code and developing abstract pipelines that can be used across blockchains. The goal of the talk is to share our architecture learnings, as well as our 5 use-cases of Scala and general recommendations when using Scala (in the Kafka Streams ecosystem).
The 5 use cases are:
- FP microservices in a Kafka Streams, Avro + Schema Registry architecture
- Extracting data from blockchains, such as Bitcoin
- Model application to tag exchange addresses in a UTXO flows
- Lake ingestion, de-normalization and Parquet
- WebSocket API, with Akka Streams
This talk was given by Jendrik Poloczek at Scala in the City.