Messaging at Scale with Open Source

The open source community offers a plethora of technologies for building highly scalable and fault tolerant systems. By coupling the right tools with the right transport technologies, developers can create applications that can scale to thousands of nodes. I will cover these tools and techniques.

In this talk we will discuss a few of the most popular open source tools for messaging at scale:

  • ActiveMQ Artemis
  • Qpid Dispatch
  • Messaging Patterns with ZeroMQ
  • Message Streaming with Kafka
  • Efficient transport protocols such as protobufs and Apache thrift, the same protocols in use at Google and Facebook

This talk will also briefly discuss how to structure application development to take advantage of scalable architectures and some of the challenges around scalable system engineering.