Connecting...

Pexels Photo 1148820

Consensus Algorithms in Distributed Systems by Yifan Xing

Pexels Photo 1148820

With many messaging systems in the industry using centralized distributed systems services to achieve reliability and consensus between servers, not all who use these services understand the details of the protocols. 

At Scale by the Bay Software Developer, Yifan Xing, brings the principles used in academia to the industry by introducing the common distributed systems protocols implemented underneath the popular services. However, this is not all, Yifan will take a look at what we can do to avoid the vulnerabilities when applying the academic theories in the industry. 

 

 
Consensus Algorithms in Distributed Systems at Scale by the Bay 2018
Many messaging systems that are widely used in the industry, e.g., Kafka, use centralized distributed systems services to achieve reliability and consensus between servers. Companies in the industry use the services; however, only a few of them understand the details of the protocols. This talk brings the principles used in academia to the industry by introducing the common distributed systems protocols implemented underneath the popular services. In addition, this talk will compare the differences between how the protocols are used in both academia and the industry. It provides details of how the protocols, specifically Paxos and Raft, work, including how they elect leaders among servers, how they achieve consensus between machines, and how they reliably process and execute client commands. Therefore, it shows how the systems and services, which use the protocols, are enabled to have fault-tolerance, and to achieve confidentiality, integrity, authenticity, availability, etc. From the reliability and security point of view, the talk discusses how the protocols deal with machine failures, including leader failures and replicas failures. It shows the vulnerabilities and potential security issues exist in the protocols. Last but not least, we'll take a look at what we can do to avoid the vulnerabilities when applying the academic theories in the industry.
 
 
This talk was given by Yifan Xing at Scale by the Bay 2018.