Building Realtime Streaming Architectures

Realtime is not only about client interactions. We have been using XMPP & MQTT since a long time to connect people and things together.

However, there is another use case for realtime that is a little less known: realtime streaming architectures. This is a design pattern that you can use to make the core of your applications or your information system realtime.

The principle is simple: you decouple components into isolated services that talk to each other through a data bus, very often Kafka, using the publish & subscribe pattern.

This approach is very powerful, as you can tolerate a component being down for a short maintenance, with the confidence that the system will catch up. The pattern is also useful to tune the computing capacity at each stage of the processing pipeline. And finally, the pattern is also popular to decouple services and make them able to evolve independently.

This is just the technical benefits, but from a high level business perspective, it simply means that you get a more resilient system, that is able to process events in realtime, with a system serving as a basis of new innovative offerings for customers. This leads to a significant business edge.

I had already presented back in 2017 at DotGo Conference what we we had been doing, at a technical level, to build realtime streaming architectures:

Today, we are able to share more by publishing a case study, showing the type of architecture we have been building for Colissimo, a leading postal service in France. Go ahead and read Colissimo case study.

Do not hesitate to contact us, if you need guidance on how to build, improve, troubleshoot or rework such an architecture.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.