The article has been written by Jérôme Sautret and presents basic ejabberd architecture and concepts.
]]>This is a very clever move from AOL. They decided that the value lies in their users base and in the network they provide. They thus want to grow the community usage of their platform. Revenue is based on advertising. You can filter ads in your client but you can also decide to build an ads supported software and benefit from AOL advertising network. You will thus be able to share advertissement revenues with AOL. I bet that MSN and Yahoo! will now think about the large set of opportunity that this model opens.
What does this move mean for the Instant Messaging community ? Simply, it means that building a client for AIM is now officially possible. Building an Instant Messaging aggregator is possible regarding the AIM / ICQ protocol aggregation.
For ProcessOne, it also means that AOL and ICQ gateway software can be published officially. Maybe it is time to dream about the most scalable AIM / ICQ gateway ever published (hint, hint !).
AOL has now opened the AIM / ICQ platform and protocol. Now, there is no major reason to not fully adopt the XMPP protocol. See my previous blog post on that topic for more information: The AOL XMPP scalability challenge. The missing part is still allowing to chat with a contact without requiring an AIM or ICQ account. In the XMPP world, this is called federation. I am not convinced that they will do this move if they managed to extend advertisements and revenue sharing to this larger community.
For now, let's celebrate !
]]>Experience in Erlang and/or XMPP protocol is a big plus, but is not mandatory.
The jobs are preferably based in France, but our company being international and our team distributed across the world, remote work is possible.
If you are interested, please drop me a mail (mremond at this website domain) with a summary of your technical achievements and a description of why you would love working at Process-One.
]]>I think this is an interesting read.
]]>
The article has been written by Jérôme Sautret and presents basic ejabberd architecture and concepts.
]]>Proof is given with the following sequence:
Memphis-2:~ mremond$ telnet chat.live.yahoo.com 5222 Trying 68.180.220.125... Connected to chat.live.yahoo10.akadns.net. Escape character is '^]'. <?xml version='1.0' encoding='UTF-8'?> <stream:stream xmlns:stream='http://etherx.jabber.org/streams' version='1.0' xmlns='jabber:client' to='chat.live.yahoo.com' > <?xml version='1.0' encoding='UTF-8'?><stream:stream xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client" from="chat.live.yahoo.com" id="5d46000b" xml:lang="en" version="1.0"><stream:features> <mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl"><mechanism>PLAIN</mechanism> <mechanism>JIVE-SHAREDSECRET</mechanism></mechanisms><compression xmlns="http://jabber.org/features/compress"><method>zlib</method></compression> <auth xmlns="http://jabber.org/features/iq-auth"/></stream:features>
This is a bit technical, but what what this command shows, is that Yahoo! server is replying on the standard XMPP port. When sending the stream opening XML, it replies with a server stream, meaning that there is this is really an XMPP server.
Note that we have no reason to believe yet that those experimentations will ever come to Yahoo! Instant Messaging service. However, as they gather XMPP knowledge, they are more and more likely to progressively add XMPP support to their main instant messaging service.
Update 1: I have checked and server-to-server connections are not open: It means you cannot join a Yahoo! Live chat room with your usual XMPP account and client.
Since I designed the first version of the pluggable pubsub module for ejabberd in early 2007, I had in mind to turn it into a powerful application server. I have already blogged about the power of the new API of ejabberd 2.0 pubsub engine. However, this single article does not do any justice to this idea and how it can change the face of the web.
Customizable services based on pubsub is XMPP (eXtensible Messaging and Presence Protocol) at its full speed. The XMPP protocol has been designed since the beginning to be a near real time routing / distribution engine. A piece in the puzzle was missing however. This is where I think our new plugin based pubsub API fills the gap and turn ejabberd into the first XMPP application server.
As a proof of concept and as a way to robustify our API, we have written the Personal Eventing via Pubsub (PEP) as a plugin of our pubsub engine. It is a big specification and should clearly show the potential of the framework.
The PEP implementation was still not enough to demonstrate that ejabberd is a fault-tolerant, highly scalable application development platform for the Web 2.0 era. I decided to start a serie of articles demonstrating how to build a distributed Twitter-like microblogging platform based on ejabberd 2.0.
The goals of the first development step are the following:
The architecture is distributed across as many user base (domain) as you want: A user can get events from users having their account in a large number of different servers.
I wanted to build a special service hierarchy so that the application integrates nicely in XMPP service discovery. I wanted a dynamic discovery tree whose content was depending on who was doing the query. This would make the application more user friendly and the example much more impressive.
To write a custom node hierarchy with our API, you simply have to write a nodetree plugin. I have written the nodetree_twitter module to handle our special hierachy looking like the following:
Friends are taken from the instant messaging contacts list: I defined this friends list as "the people that already share their presence with me". This way users do not have to manage the cumbersome subscription process specially for this application: Their friends from their contact list are cleverly reused.
Note that the node hierarchy is not defined in the Pubsub specification (XEP-0060), but is totally needed to make the application flexible. This pubsub node hierarchy does not violate the Pubsub specification, so we are fine with it.
This is the result as shown by Psi's service discovery::
The contact Blaine (thank you for your comments, by the way) is coming directly from my roster.
Posting an event is simply sending a small piece of XML code in ejabberd. As Psi does not support publishing events to our special virtual nodes (yet 
), we will send it from Psi XML console.
For example, I can send the following piece of code in the XML console. I publish the post on my special pubsub node MyStream:
<iq type='set'
to='pubsub.localhost'
id='publish1'>
<pubsub xmlns='http://jabber.org/protocol/pubsub'>
<publish node='/twitter/MyStream'>
<item>Demoing my Twitter-like code</item>
</publish>
</pubsub>
</iq>
The message is received instantly by all the contacts that have subscribed to my presence. No polling mechanism is needed. This is the beauty of using ejabberd XMPP App Server.
The received message is the following:
<message from="pubsub.localhost" to="test1@localhost" >
<event xmlns="http://jabber.org/protocol/pubsub#event">
<items node="/twitter/FriendsStream" >
<item id="4B2C8277256F" >Demoing my Twitter-like code</item>
</items>
</event>
</message>
Note that the dynamic hierarchy is very powerful: We publish on our special node. The message seems to come from another special node on the other end, for the sake of consistency.
I did nothing special for persistence of the published items. It is handled as a default by the ejabberd pubsub framework.
We can browse our new service to check that the blog post has been actually published:
I did not yet write the part of the code to browse friends posts. This will be part of a second article.
The end result is the basis of a Twitter-like application. It is fully distributed at the web scale. You can run this application on different servers and get events from people with account on different servers. It is compliant with ejabberd clustering and high scalability features. Most of all, thanks to our Pubsub API, I wrote this first code in 3 hours. It tooks something like 200 to 300 lines of code to write this extremely powerful infrastructure. Yes, that's amazing, I agree, as I was surprised myself by how easy and fast it was to write !
After a bit of clean-up, the code will be added to ProcessOne contributions Subversion repository. If you are impatient to play with the code, please drop me a mail 
In a next article, I will present the second stage of this application. I will add better and complete browsing of the items and publish directly for the service browser thank to adhoc commands supported in Psi.
In the meantime, I hope to get this experiment running with client developers: It would be very nice see the messages received properly displayed in OneTeam and in Psi. Stay tuned !
Let's try some guess work imagining what it could mean in the Instant Messaging space. To keep the reasoning, we will concentrate on the biggest IM services.
I see three possible options for the Microsoft and Yahoo! public Instant Messaging and Presence service:
Merging the two Instant Messaging networks is not enough for MSN to become the biggest Instant Messaging network.
The best figures I found are the following: in 2006, according to Neilsen/Netratings, the market share in the public IM space was as follow:
The figures are a bit outdated, but the hierarchy is probably still the same.
As said earlier, we can expect that in a distant future MSN and Yahoo services will be at least fully and totally interoperable, if not completly merged.
Recently, it has been found that AOL was experimenting XMPP access to their service. XMPP stands for eXtensible Messaging and Presence Protocol and is becoming very hot. It is the same protocol that Google is using. This protocol is also used by a large federation of IM servers around the world.
It means that the battle lines are shaping up, splitting the public IM world in two or three parts, depending on these hypothesis:
AOL needs absolutely to join the XMPP front if they want to keep their leadership in Public Instant Messaging service.
I bet that the future of XMPP and XMPP companies is even brighter today.
]]>It is still early days, but this news proves that interest in XMPP is growing at a tremendous rate. Big players, like IBM, Google, Sun, Apple, Adobe and now possibly AOL, are embracing the protocol, and it seems that the battle between XMPP and SIMPLE (Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions) is drawing to an end. The market has already chosen XMPP because it is closer to Internet technology. In comparison, SIMPLE feels like an overly complicated Telco protocol. It covers only a fraction of the scope of XMPP, and XMPP supports a very large number of extensions.
The other interesting thing is that, if XMPP wins the battle over SIMPLE, it can also win over other proprietary protocols. Google chose it for Google Talk, and now AOL is considering it. Certainly, the Instant Messaging market is moving, and 2008 will probably be a very interesting year for XMPP.
For us, XMPP protocol designers and XMPP server developers, this year might present some of our biggest challenges yet. We have lots of customer deployments that prove that our XMPP server can scale to large numbers of users. Google Talk is also proving that XMPP can work on a large scale. However, the next challenge is XMPP federation on a large scale.
Federation in the XMPP world means sharing presence information, so that users can exchange messages with others users, no matter which server (called domain) they have an account with. In practice, different servers need to be able to put users in touch, connect together and route messages.
Google Talk is federated with quite a good number of other XMPP servers, as is Jabber.org (powered by our XMPP implementation, ejabberd, and supported by ProcessOne). Google Talk and Jabber.org probably have around the same number of connections (approximately 3000 server connections each is a good guess).
With the rapid spread of XMPP, two challenges need to be addressed:
News of the AOL experience will probably boost the number of XMPP servers in production. Even though Jabber.org has 3000 connections to servers, this number is still very small. What will happen when this number grows?
Netcraft counted 70 million active domains in December 2007. This is a possible target for XMPP when all domains have web, mail and Instant Messaging.
I believe that the new version of ejabberd, ejabberd 2.0, is a step in the right direction. ejabberd 2.0 embeds a lot of improvements in the management of server-to-server connections. It can be set up in clusters and relies on special technical features to handle tens of thousands of connections on a single node. Connections between servers in ejabberd are now multiplexed to cope with large traffic between servers. We have also added protection mechanisms to detect when another server stalls and to disconnect it gracefully.
As a final note on this topic, my personal view is that, at some point in the future of XMPP, we will need to add meta routing nodes to avoid the possibility of up to 70 million servers connecting to 70 million other servers. I will share more thought on this point at a later time.
The possible future developments highlighted above will lead to another potential scale limit.
What will happen when two servers (for example AIM and Google) disconnect, and one of the servers needs to restart? If those two servers have a large overlap in their user base (for example if many users from one server are linked to many users in the other), those two servers would need to synchronise presence for millions of users at once.
Again, we believe that we are leading the way in ejabberd with experimental code. We are developing priority mechanisms to synchronise presence in an incremental way when servers need to get in touch for the first time. We are also testing protocol improvements that will limit the number of presence synchronisation packets that need to be exchanged between two servers. If this works well, we will propose this optimisation as an extension to the XMPP protocol.
Fault-tolerance and high-availability are very important features to consider when you select an XMPP server. This is not only because you cannot schedule downtime when your users are spread all over the world, but also because, each time you need to restart your server, you then need to undertake a possibly costly presence resynchronisation. Our Instant Messaging solution provides features to upgrade the code in a running system. It means less maintenance downtime and thus a smaller synchronisation burden over the whole XMPP network when you restart one server.
Please note that the rumoured AOL experiment is only a gateway that just targets XMPP client use. At this stage, we do not know if AOL will allow communication with other XMPP servers in the future. This question is only purely theoretical for the moment, but we need to make sure we are ready.
The big public server providers are considering switching to XMPP. This is a great opportunity for the XMPP protocol and brings new challenges. As a developer of XMPP servers, we are ready to face these challenges. Let's hope 2008 will give us even more opportunity to prove it.
On the scalability topic, you can read a previous blog post: Web 2.0: Shifting from "Get Fast" to "Get Massive"
]]>You can get more details from the official ejabberd 2.0.0 rc1 announcement.
]]>If you have already downloaded it, you can retry with the latest version if you had installation problems.
Thank you for your feedback !
]]>However, I get bored having to write and maintained Emakefile manually. I decided to write a tool to do the job for me and this is what Erlang automake does. It try to does it "smartly" by finding the location of your needed include files and taking this information into account in your Emakefile.
It does much more however. You can pass several project directories and Erlang automake will build an Erlang Emakefile compiling all you project to a single directory. This is useful during development as it ease path setting, and simplify dynamic code reload. This is particularly usefull if you are developing code base on several different repository (example: code coming from ejabberd main repository, from ejabberd-modules repository and/or some other private projects.
More features are already there or are planned but I will not take about them yet.
The code is developed in Process-one contribs Subversion repository and is tracked in Process-one Erlang Forge. You can also download the archive: automake-1.0.tgz
Your ideas, comments, patchs, feature requests are of course welcome.
Install the Erlang Automake tool somewhere in your path. You should also install the eautomake script as it acts as a useful wrapper.
Once your done, you can experiment with the following commands:
"erl -make" command is used to compile your project once the Eautomake file has been generated.
]]>