ejabberd 23.10

A new ejabberd release, ejabberd 23.10, is now published with more than 150 commits since the previous 23.04. It includes many new features and improvements, and also many more bugfixes.

• Support for XEP-0402: PEP Native Bookmarks
• Support for XEP-0421: Occupant Id
• Many new options and features
  
  

A more detailed explanation of improvements and features:

Added support for XEP-0402: PEP Native Bookmarks

XEP-0402: PEP Native Bookmarks describes how to keep a list of chatroom bookmarks as PEP nodes on the PubSub service. That’s an improvement over XEP-0048: Bookmark Storage which described how to store in a single Private XML Storage or a single PEP node.

mod_private now supports the bookmark conversion described in XEP-0402:
ejabberd synchronizes XEP-0402 bookmarks, private storage bookmarks and XEP-0048 bookmarks.

In this sense, the bookmarks_to_pep command performs an initial synchronization of bookmarks, getting bookmarks from Private XML Storage and stores them in PEP nodes as described both in XEP-0048 and XEP-0402.

New mod_muc_occupantid module with support for XEP-0421: Occupant Id

XEP-0421: Anonymous unique occupant identifiers for MUCs is useful in anonymous MUC rooms, message correction and message retractions. Right now the only client found to support XEP-0421 is Dino, since version 0.4.

ejabberd now implements XEP-0421 0.1.0 in mod_muc_occupantid. The module is quite simple and has no configurable options: just enabled it in the modules section in your ejabberd.yml configuration file and restart ejabberd or reload_config.

New option auth_external_user_exists_check

The new option auth_external_user_exists_check makes user_check hook work better with authentication methods that don’t have a way to determine if user exists. This happens, for example, in the case of jwt and cert based authentication. As result, enabling this option improves mod_offline and mod_mam handling of offline messages to those users. This reuses information stored by mod_last for this purpose.

Improved offline messages handling when using authentication methods without users lists

Authentication methods that manage users list outside of ejabberd, like for example JWT token or tls certificate authentication, had issue with processing of offline messages. Those methods didn’t have a way to tell if given user existed when user was not logged in, and that did block processing of offline messages, which were only performed for users that we know did exists. This release adds code that also consults data stored by mod_last for that purpose, and it should fix offline messages for users that were logged at least once before.

Changes in get_roster command

There are some changes in the result output of the get_roster command defined in mod_admin_extra:

• ask is renamed to pending
• group is renamed to groups
• the new groups is a list with all the group names
• a contact that is in several groups is now listed only once, and the groups are properly listed.

For example, let’s say that admin@localhost has two contacts: a contact is present in two groups (group1 and group2), the other contact is only present in a group (group3).

The old get_roster command in ejabberd 23.04 and previous versions was like:

$ ejabberdctl get_roster admin localhost
jan@localhost jan   none    subscribe       group1
jan@localhost jan   none    subscribe       group2
tom@localhost tom   none    subscribe       group3

The new get_roster command in ejabberd 23.XX and newer versions returns as result:

$ ejabberdctl get_roster admin localhost
jan@localhost jan   none    subscribe       group1;group2
tom@localhost tom   none    subscribe       group3

Notice that the ejabberdctl command-line tool since now will represent list elements in results separated with ;

New halt command

Until now there were two API commands to stop ejabberd:

• stop stops ejabberd gracefully, calling to stop each of its components (client sessions, modules, listeners, …)
• stop_kindly first of all sends messages to all the online users and all the online MUC rooms, waits a few seconds, and then stops ejabberd gracefully.

Those comands are useful when there’s an ejabberd running for many time, with many users connected, and you want to stop it.

A new command is now added: halt, which abruptly stops the ejabberd node, without taking care to close gracefully any of its components. It also returns error code 1. This command is useful if some problem is detected while ejabberd is starting.

For example, it is now used in the ecs and the ejabberd container images when CTL_ON_CREATE or CTL_ON_START were provided and failed to execute correctly. See docker-ejabberd#97 for details.

MySQL driver improvements

MySQL driver will now use prepared statements whenever possible, this should improve database load. This feature can be disabled with sql_prepared_statement: false.

We also added alternative implementation of upsert that doesn’t use replace .. or insert ... on conflict update, as in some versions of MySQL this can lead to excessive deadlocks. We switch between implementations based on version but it’s possible to override version check by having:

sql_flags:
  - mysql_alternative_upsert

inside config file.

New unix_socket listener option

When defining a listener, the port option can be a port number or a string in form "unix:/path/to/socket" to create and listen on a unix domain socket /path/to/socket.

The new unix_socket listener option allows to customize some options of that unix socket file.

The configurable options are:

• mode: which should be an octal
• owner: which should be an integer
• group: which should be an integer

Those values have no default: only when they are set, they are changed.

Example configuration:

listen:
  -
    port: "unix://tmp/asd/socket"
    unix_socket:
      mode: '0775'
      owner: 117
      group: 135

New install_contrib_modules top-level option

The new install_contrib_modules top-level option lets you declare a list of modules from ejabberd-contrib that will be installed automatically by ejabberd when it is being started. This option is read during ejabberd start or configuration reload.

This option is equivalent to installing the module manually with the command ejabberdctl module_install whatever. It is useful when deploying ejabberd automatically with a configuration file that mentions a contrib module.

For example, let’s enable and configure some modules from ejabberd-contrib, and use the new option to ensure they get installed, all of this the very first time ejabberd runs. Extract from ejabberd.yml:

...

install_contrib_modules:
  - mod_statsdx
  - mod_webadmin_config

modules:
  mod_statsdx:
    hooks: true
  mod_webadmin_config: {}
  ...

The ejabberd.log file will show something like:

2023-09-25 15:32:40.282446+02:00 [info] Loading configuration from _build/relive/conf/ejabberd.yml
Module mod_statsdx has been installed and started.
The mod_statsdx configuration in your ejabberd.yml is used.
Module mod_webadmin_config has been installed and started.
The mod_webadmin_config configuration in your ejabberd.yml is used.
2023-09-25 15:32:42.201199+02:00 [info] Configuration loaded successfully

...
2023-09-25 15:32:43.163099+02:00 [info] ejabberd 23.04.115 is started in the node ejabberd@localhost in 3.15s
2023-09-25 15:32:47.069875+02:00 [info] Reloading configuration from _build/relive/conf/ejabberd.yml
2023-09-25 15:32:47.100917+02:00 [info] Configuration reloaded successfully

New notify_on option in mod_push

mod_push has a new option: notify_on, which possible values:

• all: generate a notification on any kind of XMPP stanzas. This is the default value.
• messages: notifications are only triggered for actual chat messages with a body text (or some encrypted payload).

Add support to register nick in a room

A nick can be registered in the MUC service since ejabberd 13.06, this prevents anybody else to use that nick in any room of that MUC service.

Now ejabberd gets support to register a nick in a room, as described in XEP-0045 section 7.10 Registering with a Room

Registering a nick in the MUC service or in a room is mutually exclusive:
– A nick that is registered in the service cannot be registered in any room, not even the original owner can register it.
– Similarly, a nick registered in any room cannot be registered in the service.

MUC room option allow_private_messages converted to allowpm

Until ejabberd 23.04, MUC rooms had a configurable option called allow_private_messages with possible values true or false.

Since ejabberd 23.10, that option is converted into allowpm, with possible values:

• anyone: equivalent to allow_private_messages=true
• none: equivalent to allow_private_messages=false
• participants
• moderators

gen_mod API to simplify hooks and IQ handlers registration

If you wrote some ejabberd module, you may want to update your module to the simplified gen_mod API. This is not mandatory, because the old way to do this is supported.

Until now, erlang modules that implemented ejabberd’s gen_mod behaviour called ejabberd_hooks:add and gen_iq_handler:add_iq_handler in ther start functions. Similarly, in their stop function they called ejabberd_hooks:delete and gen_iq_hanlder:remove_iq_handler.

Since ejabberd 23.10, there is an alternative way to do this: let your start function return {ok, List}, where List is a list of iq handlers and hooks that you want your module to register to. No need to unregister them in your stop function!

How to change your module to the new API? See the changes done in mod_adhoc.erl in commit 60002fc.

MS SQL requirements

To use the Microsoft SQL Server database, the libtdsodbc library is required, as explained in the corresponding section of the ejabberd Docs: Configuration > Databases > Microsoft SQL Server

Since this release, the ejabberd container image includes this library.

Please notice if you install ejabberd using the binary installers and want to use MS SQL: you must install the libtdsodbc libraries on your machine. It cannot be included in the ejabberd installer due to the nature of the odbc drivers being dynamic depending on the respective odbc backend in use.

Erlang/OTP 20.0 or higher required

This ejabberd release requires Erlang/OTP 20.0 or newer to compile and run, support for Erlang/OTP 19.3 is deprecated. If you are still using Erlang/OTP 19.3, please update to a more recent Erlang version. For example, the ejabberd binary installers and container images are using Erlang/OTP 26.1. That requirement increase was announced almost a year ago, check more details in the ejabberd 22.10 release announcement.

If you are still using Erlang/OTP 19.3 and cannot update it right now, there’s still a possibility to compile ejabberd 23.10 with Erlang/OTP 19.3, but please notice: there is no guarantee or support that it will compile or run correctly. If interested, revert the changed line in the file configure.ac done in commit d299b97 and recompile.

Acknowledgments

We would like to thank the contributions to the source code, documentation, and translation provided for this release by:

• Holger Weiß improvements in the installers and several modules
• Saarko, improvements in the containers
• EISST International Ltd for sponsoring work on mod_push new notify_on option
• Mr. EddX, new Bulgarian translation
• Wellington Uemura, updated the Portuguese (Brazil) translation
• Олександр Кревський, updated the Ukrainian translation
• Nautilusx, updated the German translation

And also to all the people contributing in the ejabberd chatroom, issue tracker…

Improvements in ejabberd Business Edition

Customers of the ejabberd Business Edition, in addition to all those improvements and bugfixes, also get:

• Push:
  • Add support for Webpush
  • Various APNS & GCM fixes and optimizations
  • async calls to push backends
  • Improved error messages
  • Improve error detection and reconnection strategy
  • New mod_push_logger module to log push related events
• Matrix:
  • Add support for Matrix v10 rooms
  • Add SRV support in mod_matrix_gw_s2s
• Misc:
  • Add max_concurrent_connections option to webhook
  • Add module for logging chat & jingle events in a separate file
  • Add retraction handling in MAM for p1db & dynamodb databases

ChangeLog

This is a more detailed list of changes in this ejabberd release:

Compilation

• Erlang/OTP: Raise the requirement to Erlang/OTP 20.0 as a minimum
• CI: Update tests to Erlang/OTP 26 and recent Elixir
• Move Xref and Dialyzer options from workflows to rebar.config
• Add sections to rebar.config to organize its content
• Dialyzer dirty workarounds because re:mp() is not an exported type
• When installing module already configured, keep config as example
• Elixir 1.15 removed support for --app
• Elixir: Improve support to stop external modules written in Elixir
• Elixir: Update syntax of function calls as recommended by Elixir compiler
• Elixir: When building OTP release with mix, keep ERLANG_NODE=ejabberd@localhost
• ejabberdctl: Pass ERLANG_OPTS when calling erl to parse the INET_DIST_INTERFACE (#4066

Commands

• create_room_with_opts: Fix typo and move examples to args_example (#4080)
• etop: Let ejabberdctl etop work in a release (if observer application is available)
• get_roster: Command now returns groups in a list instead of newlines (#4088)
• halt: New command to halt ejabberd abruptly with an error status code
• ejabberdctl: Fix calling ejabberdctl command with wrong number of arguments with Erlang 26
• ejabberdctl: Improve printing lists in results
• ejabberdctl: Support policy=user in the help and return proper arguments
• ejabberdctl: Document how to stop a debug shell: control+g
• ejabberdctl: Support policy=user in the help and return proper arguments
• ejabberdctl: Improve printing lists in results

Container

• Dockerfile: Add missing dependency for mssql databases
• Dockerfile: Reorder stages and steps for consistency
• Dockerfile: Use Alpine as base for METHOD=package
• Dockerfile: Rename packages to improve compatibility
• Dockerfile: Provide specific OTP and elixir vsn for direct compilation
• Halt ejabberd if a command in CTL_ON_ fails during ejabberd startup

Core

• auth_external_user_exists_check: New option (#3377)
• gen_mod: Extend gen_mod API to simplify hooks and IQ handlers registration
• gen_mod: Add shorter forms for gen_mod hook/iq_handler API
• gen_mod: Update modules to the new gen_mod API
• install_contrib_modules: New option to define contrib modules to install automatically
• unix_socket: New listener option, useful when setting unix socket files (#4059)
• ejabberd_systemd: Add a few debug messages
• ejabberd_systemd: Avoid using gen_server timeout (#4054)(#4058)
• ejabberd_listener: Increase default listen queue backlog value to 128, which is the default value on both Linux and FreeBSD (#4025)
• OAuth: Handle badpass error message
• When sending message on behalf of user, trigger user_send_packet (#3990)
• Web Admin: In roster page move the AddJID textbox to top (#4067)
• Web Admin: Show a warning when visiting webadmin with non-privileged account (#4089)

Docs

• Example configuration: clarify 5223 tls options; specify s2s shaper
• Make sure that policy=user commands have host instead of server arg in docs
• Improve syntax of many command descriptions for the Docs site
• Move example Perl extauth script from ejabberd git to Docs site
• Remove obsolete example files, and add link in Docs to the archived copies

Installers (make-binaries)

• Bump Erlang/OTP version to 26.1.1, and other dependencies
• Remove outdated workaround
• Don’t build Linux-PAM examples
• Fix check for current Expat version
• Apply minor simplifications
• Don’t duplicate config entries
• Don’t hard-code musl version
• Omit unnecessary glibc setting
• Set kernel version for all builds
• Let curl fail on HTTP errors

Modules

• mod_muc_log: Add trailing backslash to URLs shown in disco info
• mod_muc_occupantid: New module with support for XEP-0421 Occupant Id (#3397)
• mod_muc_rtbl: Better error handling in (#4050)
• mod_private: Add support for XEP-0402 PEP Native Bookmarks
• mod_privilege: Don’t fail to edit roster (#3942)
• mod_pubsub: Fix usage of plugins option, which produced default_node_config ignore (#4070)
• mod_pubsub: Add pubsub_delete_item hook
• mod_pubsub: Report support of config-node-max in pep
• mod_pubsub: Relay pubsub iq queries to muc members without using bare jid (#4093)
• mod_pubsub: Allow pubsub node owner to overwrite items published by other persons
• mod_push_keepalive: Delay wake_on_start
• mod_push_keepalive: Don’t let hook crash
• mod_push: Add notify_on option
• mod_push: Set last-message-sender to bare JID
• mod_register_web: Make redirect to page that end with / (#3177)
• mod_shared_roster_ldap: Don’t crash in get_member_jid on empty output (#3614)

MUC

• Add support to register nick in a room (#3455)
• Convert allow_private_message MUC room option to allowpm (#3736)
• Update xmpp version to send roomconfig_changesubject in disco#info (#4085)
• Fix crash when loading room from DB older than ffa07c6, 23.04
• Fix support to retract a MUC room message
• Don’t always store messages passed through muc_filter_message (#4083)
• Pass also MUC room retract messages over the muc_filter_message (#3397)
• Pass MUC room private messages over the muc_filter_message too (#3397)
• Store the subject author JID, and run muc_filter_message when sending subject (#3397)
• Remove existing role information for users that are kicked from room (#4035)
• Expand rule “mucsub subscribers are members in members only rooms” to more places

SQL

• Add ability to force alternative upsert implementation in mysql
• Properly parse mysql version even if it doesn’t have type tag
• Use prepared statement with mysql
• Add alternate version of mysql upsert
• ejabberd_auth_sql: Reset scram fields when setting plain password
• mod_privacy_sql: Fix return values from calculate_diff
• mod_privacy_sql: Optimize set_list
• mod_privacy_sql: Use more efficient way to calculate changes in set_privacy_list

Full Changelog

https://github.com/processone/ejabberd/compare/23.04…23.10

ejabberd 23.10 download & feedback

As usual, the release is tagged in the Git source code repository on GitHub.

The source package and installers are available in ejabberd Downloads page. To check the *.asc signature files, see How to verify ProcessOne downloads integrity.

For convenience, there are alternative download locations like the ejabberd DEB/RPM Packages Repository and the GitHub Release / Tags.

The ecs container image is available in docker.io/ejabberd/ecs and ghcr.io/processone/ecs. The alternative ejabberd container image is available in ghcr.io/processone/ejabberd.

If you consider that you’ve found a bug, please search or fill a bug report on GitHub Issues.