Chat standard (XMPP)

- Reading time: 20 minutes / whole rubric: 108 minutes -

Vendor-independent chat based on international standardized protocol is a recommendation in the Quick Overview!

In contrast to other messengers/systems, chat based on the XMPP protocol can and may(!) be used legally wherever email is used!

The information sheet (PDF) summarizes all the essentials.

Preface

If someone knows the chat standard under the name “Jabber”, it should be noted that the system has undergone major changes in recent years and has adapted to the current requirements of modern “instant messaging”. However, the name is still often used today - which is interesting for smartphone users: In the normal address book, there is often a field “chat” for each contact in which you can store the chat address of your contacts. For standardized chat simply choose “Jabber” as system.

Significant enhancements:

  • Users do not have to be online at the same time
  • End-to-end encryption with current encryption methods
  • Audio/video telephony
  • Voice messaging
  • File exchange independent of format

Table of contents:

Advantages & disadvantages

  • positive: No dependence on a central office
  • positive: Interoperability through the use of standards
  • positive: Freedom in the choice of software
  • pos&neg: Different user interfaces due to variety of clients; no cross-operating system client (except browser)
  • negative: You have to rely on recommendations for clients and on recommendations for server bed drivers (there is not one provider and not one app)
  • negative: Different apps/programs with different functionalities
  • negative: Android clients have a clear lead over iOS clients in terms of user-friendliness and functionality
  • negative: Apart from the recommended apps/programs, there are also those that have not implemented OMEMO encryption, for example (these are then still usable for public chat rooms)
  • negative: Federation generates metadata due to the system; not designed to avoid metadata, but for interoperability
  • negative: Verified OMEMO encryption is less recommendable when using multiple devices/clients, as with “total” security messages cannot be read on new devices and this is often not wanted in daily practice. Remedy only through “blind trust before verification” (BTBV) - or through other encryption (PGP)
  • negative: Server side of ejabberd software: members lose connection when server restarts; server forgets member list when server restarts; server to server connections are not robust and are lost when server IP changes
  • negative: Subjective info: In the past, message loss was reported from time to time when devices were offline
  • negative: Not caused by the protocol but a consequence of the “federation”: Publicly available servers rarely guarantee a certain level of reliability. Especially with privately operated servers, longer outages can always be the result. Sometimes such servers are no longer actively maintained or the service is completely discontinued. You have to look for reliable servers yourself, rely on recommendations or become active in hosting yourself.
  • negative: There are some hosters with many server operators running their services - if one fails, many different instances would be affected. See the following overview (external).

Special case “public chat rooms”:

  • positive: Possibility for exchange, which WhatsApp, for example, does not offer
  • Negative:** If a user leaves a public chat room (which anyone can join with any alias), another user can join under the same nickname and continue writing without others noticing the change.
  • negative: If messages that have already been sent are “corrected” (subsequently sent again in a modified version), they may also be displayed correspondingly often, depending on the client or setting used

Sources that also list critical points are curius.de / May 2021 (external), Privacy Manual (external), Kuketz blog / 18.02.2018 - test period Oct. 2016 to Nov. 2017 (external) and Signal / 10.05.2016 (external)

Basics

To be able to chat, a chat account must exist or be created at a server. The message exchange is then done with an XMPP compatible program of free choice.

There are no exact user numbers - however these lie in the multi-digit million range:

Normal case: Internet use

Servers

XMPP is a federated, decentralized system, just as is the case with e-mail. In theory, this means that each user operates their own server. However, if you want to do this yourself, a certain amount of technical understanding and commitment is required. This is why - as with e-mail - there are dozens of different public providers whose servers are then used to manage accounts, address books and chat histories for several devices.

Graphical representation of federation and server networking based on a few servers:

Preview
Normal view (WebGL) / Source (external)
Preview
Compressed view (WebGL) / Source (external)
Preview
Normal view / Source (external)
Preview
Compressed view / Source (external)

The whole thing is also available in a 3D version (WebGL): https://xmppnetwork.goodbytes.im/3d.html (external)

So there are an extremely large number of providers and XMPP servers and it seems difficult for a newcomer to make a choice.

Tip about the xmppnetwork service:
It is possible to display only the connections from a single server. To do this, the query simply needs to be extended with ?focus=server.tld.

Which servers are listed for this service and how they can be added can be found in the FAQ (external)

Clients

Thanks to the pioneering role of “Conversations”, it is possible to “text” on 80% of smartphones with a modern and intuitive user interface. Beyond that, however, there are other inteoperable free messengers for a wide variety of operating systems and for browsers:

* The multi-messenger Pidgin can only be recommended with restrictions (as of 2021). It is intuitively usable and supports OMEMO - not however the XMPP extensions “MAM” (XEP-0313) and “Message Carbons” (XEP-0280): These are required so that chat history and messages are reliably distributed even on multiple owned devices.

Equally restricted is the desktop client “coy.im”, which also does not support the extensions XEP-0313 and XEP-0280. This client has committed itself to OTR as encryption method. coy.im according to its own statement not suitable for the exchange of sensitive content: https://github.com/coyim/coyim#security-warning (external)

It is even possible to write an OMEMO-enabled Jabber client yourself using Java, which does not require 200 lines of source code:

A good overview of different clients can be found at xmpp24.de (external), linkmauve.fr (external; English) and at xmppchat.eu (external).

However, no matter which messenger is chosen - through the common basis one can communicate without having to have the same messenger installed. As with other programs (e.g. browsers, file managers, image editing, music programs, …), the user can choose the one he likes best! The “system” works independently of this.

Client-XEP-Overview

The outstanding feature (for some a flaw) of standardized chat is that it can be adapted to changing requirements and kept modern by extensions/“XEPs”. However, not every client needs every extension and if extensions are to be supported, appropriate programming work is required. The following overview provides an overview of this:

PDF file (approx. 0,2 MB): >> here <<
Kindly provided by: ‘kikuchiyo’ / CC-BY-SA-4.0

Encryption

In addition to the generally used transport encryption, there are several encryption types that can be used:

  • OMEMO
    = Encryption per terminal (client); messages can only be read on the devices for which they were encrypted.
    Apps/programs that support this current encryption can be found at: https://omemo.top (external)
    Background information: https://conversations.im/omemo (external)
  • OpenPGP
    = Encryption per user account; messages can also be read on other devices.
  • Cannot be used in groups: ‘OTR’ (only if both are online at the time of sending)
  • Cannot be used in groups: ‘PGP’ (old)

Overview table: https://wiki.xmpp.org/web/XMPP_E2E_Security#Comparative_Overview (external)

Thoughts on end-to-end encryption

Chat rooms

Besides classic chats with normal contacts (2s chats / “1:1”), there are also private chat rooms (“groups”) and even public chat rooms. For and in each chatroom there are therefore different …

… settings:

  • private (by invitation only) or public (everyone who knows the address can enter)
  • moderated (yes/no)
  • visibility of the chat addresses for the participants: (partly) anonymous or visible for everyone
  • visible in public directories (yes/no)
  • permanent (persistent) or ephemeral (the chat room continues to exist when the last participant leaves it)
  • default language
  • blocked/banned chat addresses
  • …

Not every server offers its chatroom administrators the same settings options; there are sometimes differences here.

Roles and rights of people present in the chat:

  • visitors (no write permission when “moderated”)
  • participant (write permission also when “moderated”)
  • moderator

Note:
In unmoderated public chatrooms there is no visible difference for visitors and participants - only if e.g. due to garbage messages (“spam”) the chat is switched to “moderated”, writing rights are possible or not.

Affiliation:

  • owner
  • administrator
  • member
  • no (special) affiliation
  • excluded

More information about roles, rights and affiliations: Wikipedia (external)
Source: XMPP.ORG (external)

Bridges

Bridges can be used to exchange messages from/to other systems. One kind of universal bridge is Matterbridge (external). An example usage is described here (external).

However, bridges should always be used with caution and are not always legal - it is better to use standardized interfaces.

Cross-reference: Are bridges and their use legal?

Use in LAN: “Bonjour”.

XMPP can also be used within an own network without internet access. The corresponding technology in the background is called “Bonjour”. A useful application for this are larger user groups in a closed network (LAN/WAN) such as work networks, lecture halls, dormitories. But also as a fallback level for internal company communication, if there is no internet connection due to outages/maintenance work at telecommunication providers.

Recommendations

… The development as well as the use and operation of an own messenger service in church and diaconia based on established and freely accessible protocols on federated servers would be the best solution in the view of the Data Protection Officer of the EKD and is therefore recommended.

Source: https://datenschutz.ekd.de/wp-content/uploads/2018/10/Ergänzende-Stellungnahm-Messgr-Dienste.pdf vom 24.10.2018

Problems/FAQ

AV Telephony

Question: Why do calls not work in some constellations?

Problem description:
If I am on the same WLAN with my contact, we can make calls via the app - if I go online via mobile, it no longer works. The other person rings, but no conversation is established. Chatting works smoothly in any combination.

Answer:

Since security settings (of firewalls or NAT) often take effect for external data traffic or there is appropriate hardware (e.g. Fritz!Box) between the two conversation partners, another service (STUN-/TURN-Server) is needed for direct connection establishment, which mediates the actual IP addresses required for direct transmission (without the respective chat servers). Otherwise calls outside the same WLANS will work only in some (rare) cases.

Whether the own server offers the help of STUN-/TURN can be seen here: https://compliance.conversations.im/test/turn (external)
Or directly e.g. in Conversations in the account settings over the menu the point “server details” activate. Here “XEP 0215 External Service Discovery” must be set to “yes”.

Other causes could be:
TOR is used or a throttled data speed. So 32/64kbit are theoretically sufficient after used up data volume - but are often throttled pulsed. In other words, you only get decent bandwidth for data transfer every few seconds, but then you have long dropouts, which means A/V doesn’t work.

Interesting additional knowledge

History and demarcation

Difference “Jabber” / “Cisco Jabber “

Even though both have the same origin, “Cisco Jabber” is not identical to the free “Jabber (XMPP)”. “Cisco Jabber” is an implementation of Cisco’s XMP protocol for enterprise communications using Cisco devices. It is not open source and has received its own extensions from the Cisco company, which are not available to the general public.

Nevertheless, it is technically possible to exchange messages based on the “free” protocol with “Cisco Jabber”. However, this federation (open collaboration) only works if the appropriate settings/shares are set up on the respective company’s own Cisco server.

Originally XMPP was developed by Jabber, Inc. Then handed over to XSF, and Jabber Inc was sold to Cisco.

Dossier on XMPP

The State of XMPP in 2019 (external) with information to:

  • Open Source’ Community
  • Commercial Usage
  • Clients

The State of Mobile XMPP in 2016 (external) with information to:

  • Reliability
  • Images and multiple devices
  • Mobile ready encryption
  • An Excurse on Push

Hidden user groups

It is often not at all obvious that XMPP is used as the basis in the background. For example, there are really large applications with many millions of users, such as online games or messengers, which use a sometimes individually modified XMPP internally - but do not open it to the outside world. These include, among others:

  • Messenger
    • WhatsApp: ~ 2 billion users
    • Kik Messenger: ~300 million users
    • Zoom: ~200 million users
    • Jitsi: ~20 million users
    • Moya App: ~6.5 million users
    • Grindr: ~4 million users
    • Mailfence: ~350,000 users
  • IT systems & telecommunications provider
    • Apple Push Notifications: ~500 million users
    • Facebook: precise number of users unknown
    • Firebase Cloud Messaging : precise number of users unknown
    • GitHub: precise number of users unknown
    • GMX: precise number of users unknown
    • Google Push Notifications: ~1.5 billion users
    • Google Cloud Print: precise number of users unknown
    • Logitech Harmony Hub: precise number of users unknown
    • Orange: precise number of users unknown
  • Games
    • Fortnite (external): ~250 million users
    • Nintendo Switch (external): ~34 million users
    • League of Legends (external) (~27 million users)
    • EA Origin: ~40 million users
    • Neverwinter: ~16 million users
    • EVE (external) (~1 million users)
    • Star Trek Online: ~900,000 users
    • Champions Online: ~345,000 users
  • Economy
    • Trans.eu (logistic)

Sources:

Military use

Isode (external) is a company that develops commercial off-the-shelf products (client and server software) for secure messaging and directory systems for government, military and aerospace customers in over 150 countries. For mission-critical solutions, emphasis is placed on the use of international standards and open protocols (IMAP, XMPP, LDAP):

Graphic representation of military use

Example graphic of military use

Source for graphic: https://www.isode.com/solutions/military-xmpp.html (external) - will be forwarded to actual site (external; 10.12.2022)

Military use (NATO)

Excerpt from the NATO paper “MAJIIC - Multisensor Aerospace-ground Joint ISR Interoperability Coalition”:
”… In areas where no STANAG is available, such as Instant Messaging tools for distributed operator collaboration, the project will assess widely used commercial standards for potential use in coalition operations, such as the XMPP standard used in the Jabber chat tool …”

Source: https://www.nato.int/docu/update/2007/pdf/majic.pdf (external)

Military use (USA)

Source: https://www.defencetalk.com/us-dod-selects-jabber-to-deliver-net-centric-collaboration-services-12613/ (external)

Secret Service NSA

The US intelligence service also uses the standard protocol.

Source: https://www.vice.com/en/article/8qxdez/the-nsa-uses-the-same-chat-protocol-as-hackers-and-activists (external)

Other

Open Source is free of charge - however, companies can be commissioned with development and the result is then available to the general public. There is also e.g. a job exchange around XMPP: https://xmpp.work/ (external)

Links to external information:

Detailed external opinions

The “XMPP” system has been around for about 20 years. Nevertheless, or rather because of this, there are various opinions/viewpoints on this:

International info pages

French

Conclusion

Cooperation with Matrix through functioning bridges would be ideal. This would allow companies/agencies to enjoy the benefits of failsafe chat rooms and team chat capabilities internally - while still allowing for standardized message exchange to/from the outside world.

For system comparison of standardized chat (XMPP) and Matrix: >> here <<

XMPP Logo