Pubsub upcoming improvements in ejabberd 2.1

Posted by Mickaël Rémond on July 20, 2009

During the past months we have greatly enhanced pubsub and PEP support in ejabberd.

Lots of improvements

For several projects we have been working on, we have mostly rewritten the pubsub and PEP support in ejabberd. The code is still extensible by plugins and thus is as flexible as ever but now offer a huge performance enhancement.

Among the improvements in version 2.1, you will get:

  • Relational database storage: in ejabberd 2.1, pubsub is able to store a relational database (MySQL or PostreSQL), as a replacement of Mnesia if you wish. This can be usefull either if you plan to store really large volume of data or would like to have all your data stored in a relational database for central management and backup.
  • Performance and scalability: From our tests, the latest pubsub version can be used even for the most demanding deployments supporting large number of nodes and/or subscribers.
  • Better capabilities handling for PEP: One of the main bottleneck for PEP is the management of the capabilities and the cache of those capabilities. It is particularly challenging if you want to make it right with good s2s support. Our new implementation is much more robust at handling the caps caching and should perform much better on large deployments.
  • Update to the latest pubsub specification and we also now have a broader feature coverage.
  • Change in default policy: The default is now to use the flat hierarchy of node, but you can switch back to the tree hierarchy of nodes.

Some features of this new pubsub implementation are already available in ejabberd development SVN (trunk), you can give them a try.

Performance and adoption for online realtime services

We have been able to support 100 000 subscribers spread over 10 pubsub nodes, having an average events every 2 seconds on a cluster of two ejabberd nodes. We have more optimisation in the making. It makes ejabberd the XMPP server of choice if you plan to heavily use pubsub.

Wordpress, Collecta and Buddymob for example are using ejabberd pubsub to support their distribution system and more projects are currently being deployed. If you have XMPP pubsub use case to share, we would love to hear it.



Categories: Jabber / XMPP  ejabberd  
Share article:   Tweet this   Make delicious   Stumble upon  

Comments

anonymous avatar

Mickaël -

This is excellent news indeed!

When you say ‘cluster of 2 ejabberd nodes’, can you give the class of machine (CPU, memory, disk space) that you’re referring to?

Thanks again!

Cheers,

- Bill

Posted by William J. Edney on 20 Jul 2009 at 02:50



 
Mickaël Rémond's avatar

Hello,

It was standard servers in term of CPU (4 cores). Disk space was not a problem as it was a routing test. Memory on each node was 16Gb.
However, we stopped the test because the injection machines were overloaded (We stopped at 120 000 subscribers exactly). We would have very likely gone further with more injection machines.

Posted by Mickaël Rémond on 20 Jul 2009 at 15:47



 
anonymous avatar

I was wondering what operating system were you using on the ejabberd cluster? Were the servers used in the cluster also handling the presence/other xmpp traffic for the 120,000 simultaneous users?

Thanks in advance!

Sylvain

Posted by Sylvain Niles on 30 Nov 2009 at 21:38



 
Mickaël Rémond's avatar

Operating system: Linux.
It was not a chat test, but an event distribution test, so we did not simulate presence and chatting.

Posted by Mickaël Rémond on 30 Nov 2009 at 22:01



 


Add comment

Name:

Email:

URL:

Smileys

Remember my personal information

Notify me of follow-up comments?