stranger Posted 8 hours ago Posted 8 hours ago (edited) Connecting your world to Instant Messengers with XMPP Ever dreamed about chatting with your DW friends from the comfort of your sofa or while traveling somewhere - because you can connect to them from your phone? Then this bot could be something for you. This relay bot is based on the previously published bot for chatting, reusing the same basic Qt intefrace. It's function is different though. The difference is that it will ask account details for an additional account - XMPP (more on that below). The bot listens to messages in the world (with caretaker privileges) or local chat (without) and relays them to XMPP users and back. Essentially by relaying messages to everyone it creates a single interconnected chat space. To make this interaction more orderly, the bot does not reveal credentials of users connecting to it from mobile devices to users in DW, and instead asks these users to pick a nickname. The idea being that they would pick their DW nickname - if they wish to do so of course. To prevent abuse, a system of verification is in place, where messages of unverified users appear as "tourists" in DW. Bot owner or any assigned admin (among XMPP users) then controls verification of users, as well as other essential controls, including being able to silence ("ban") them. Hopefully that last one will never have be used though Ok, what's XMPP? (What a weird name!) And why is it better than Discord? The name is not great, I agree. It used to be named 'Jabber' and a few folks may even remember it from the time just after the peak popularity of ICQ. Well, unlike ICQ, Jabber is still alive and well (but due to legal reasons had to change name to XMPP). Moreover it did evolve quite a bit sunce then, becoming a modern, sleek and secure instant messenger protocol with all usual features such as end-to-end encryption, file exchange, audio-video calls and more. But there is oneâvery cool and importantâadvantage over most other modern instant chat and social media apps. XMPP is an open protocol. What does it mean? It means that unlike Discord, WhatsApp, Telegram and many others, you have full control how to use it. We've got so used to big corporations assuming full control of our communication that it feels so natural, but it is actually pretty weird. Think about it. Why does WhatsApp need access to all your contacts? Is it somehow required for it to function? Not at all, they just want to sell your data to advertisers. Can you opt out? No, you can't. The only option is not to use it, but all your friends are there. So you are left with no choice. XMPP, on the other hand, gives you complete freedom over your communication. XMPP itself is a Foundation which only specifies standards of the protocol - basically a shared language that apps and servers should speak. The large and diverse open source community then uses these standards to create open source servers and clients. That's why there's ample choice with XMPP. Don't like your current app? Just get another one, and use the same account where all your friends can reach you. Same goes for your server operator. You can change it any time because XMPP is federated, so no matter which server you use, you can talk to others. Up to and including hosting your own server (which many people in the community do!). Ah and the best parts of those XMPP accounts? You only need to come up with some username that is still free on the server and a password. No phone number required! Not even email is required, although some servers ask for it to be able to restore your password. And - specifically for Discord users - of course no requirement to send your goverment ID anywhere! Being open source has some other important advantages than just giving you more freedom of choice. As already mentioned, since there's no commercial interest associated with you using it, no one is collecting your data. How can you be sure? Well, the magic of open source is that anyone can look at it and I assure you that a lot of people do. Any attempt to sneak in something like that would be noticed and flagged. Moreover it is just better for security: bugs introducing vulnerabilities are quickly found too because so many pairs of eyes are looking through the code. A caveat here is that these points are true for more established/popular clients and servers, of course not any client created by John Doe of Evil Corp who has 10 users; more on which clients/servers can be trusted below. Speaking of caveats: so where's the catch? Why not everyone using XMPP already if it's so awesome? Well the biggest reason is that most of this ecosystem is maintained by enthusiasts with very little commercial gain. They don't have venture capital to pay massive sums for marketing. But they also don't have to recuperate these costs in profits later, at your expense! Ever noted how all chat/social media apps start out nice and then become increasingly filled with commercial slop and ads? Well that's the simple reason. Investors want their money back, with profits. But there are actual caveats, and it would be dishonest not to mention them. However I think they are temporary and are a consequence of available freedoms, rather than real issues. XMPP is a modular protocol, and due to so many different servers/clients without a single controlling authority you may face certain issues. Such as: video calls work with your friend on the same server and app, but they don't work with friends on other apps, or another server. Stuff like this can be a bit frustrating. But consider that there is another app and server to begin with. Something about which I was a bit disappointed lately, is that there's still apparently no single Windows app which supports video calls. (There are some work-in-progress projects so it may change soon.) The situation is much better on mobile actually. But worry not. The point of this guide is to help you navigate these strange open source waters - so let's dive in! I'm convinced to give it a try! How to start using XMPP? To start using XMPP is easier than ever. Thanks to this collection of recipies, you can easily follow one depending on your needs and circumstances. You may note that some options involve your phone number. But this is just for those to whom it's the most convenient way! It is certainly not a must, just see the options with "username". In any case, in the end you will have something that is called "JID" in XMPP-parlance, a User ID in the form of [email protected]. This is how others can find you on XMPP network. Note although it looks much like an email address, it has nothing to do with email besides superficial similarity. Trying to send emails to XMPP addresses will not lead you anywhere Ok, cool, you can send messages (and files and do calls and all). There are public channels, but not a ton of them... (you can find many of them here by the way). But how does any of this replace Discord?? To have a richer, social-media-like experience, take a look at Movim. Movim is a sort of social media infrastructure built on top of the XMPP bedrock, where users can interact with other Movim users and also "regular" XMPP users. Note that Movim is also federated and open source, so you don't have to join their server and can run your own Movim, just like with XMPP itself. This is especially relevant to leaders of communities on Discord. Why not move the whole community to the server that you fully control - and only you set the rules, and/or make sure that there are no surprise changes in "Terms of Service" that you have to immediately comply with - such as suddenly needing to send a government ID to keep using it. Think about it Is this post about XMPP or a DW bot? What's about that bot? Yes, yes, sorry, we deviated quite a bit Now back on topic. The bot will greet you with a welcome screen where you need to enter your citizenship details like with any other bot, as well as XMPP details. XMPP address takes the form [email protected], see above how to set up an XMPP account. You'll need two accounts to fully profit from your bot. One will belong to the bot itself, it can be anything, but let's say it's [email protected]. XMPP users should write to this address and the bot will greet them with the instructions suggesting to set up a name and start chatting. This is done with commands - text messages that start with a "/" symbol. See below for available commands. On this screen you can also specify your own XMPP address, e.g. [email protected]. By specifying it here, you will be automatically added to bot admins. The bot respects privacy of users by never sharing their XMPP address to DW or among each other. A separate username is used that the users have to set up themselves. (Bot owner and admins have access to XMPP address; be very careful whom you add as admin. Perhaps it's enough to only add yourself to start with.) An admin can "verify" username, otherwise any texts will appear in quotes or gray color (in caretaker mode), suggestive of a "tourist" account. Users can select any username, whether they are verified or not, but only text sent with the verified name will appear as "citizens" (normal text, no quotes). Once you start the bot and typed in your data, the main screen will appear. It will show you XMPP status, nearby users and usual bot controls such as teleport, avatar change and displaying how many users are in the world. There's an additional window that can be accessed from IM Users -> Manage IM Users menu. This shows you currently "registered" users from XMPP, and they will appear in (a combination of) following states: Inactive (gray color): either user did not set up a username yet, stopped message transmission, or reached the limit of inactivity (which you can configure below; default: 24 hours). In any case, for these users, no messages are being transmitted. Active (default text color, depends on color theme): active users (will also have "Yes" in "Active?" column). They receive messages and can send them any time. Banned (red): that user has been banned, they don't receive messages and can't send messages to anyone. Admin (bold text): users with admin rights. Can do basically anything you can do from this interface by sending commands via chat. If you right-click any row you will see a set of actions that are available to you. They are the same as admin commands for XMPP users listed below. There's also a clarification text that appears before any command is executed. Here's a list of command available to XMPP users: User commands: /setname username: the first time a user interacts with the bot, they will be prompted to set a username for themselves. Messages in DW and to other XMPP users will appear under this name, XMPP address of the user is not revealed (but is visible to admins and the bot owner). /start: this command starts transmitting messages to DW and back. /stop: this command stops transmitting messages from DW. /settings: this command is just a placeholder which points to currently the only setting available - setting a username. /help: this command explains the purpose of this bot and lists available commands. Note for admins: it will list the admin commands as well. /status: this command shows users connected via both XMPP and DW (in the world - if bot is running with caretaker privileges, or locally - without). Note for admins: this will show you XMPP IDs of the users as well, which you will have to use to ban of forget a user (see below). Admin commands: /ban [email protected]: ban a user (stops message transmission to and from them). Note that XMPP ID is used here instead of user's self-chosen username to prevent mistakes and abuse by sneaky users. As an admin, you can see all XMPP IDs by running /status command. Also note that banning an admin does not revoke their rights, including to restore themselves. For that you should use "/forget" command below. /restore [email protected]: restore a user. /verify [email protected] username: verify username for XMPP user. The text of this user will appear in normal color and without quotes (as if "citizen" in DW). If the username you specify here different from user's current name, they will have to change it with /setname; writing under any other name they will not be verified. For that reason double check for spelling mistakes here. /forget [email protected]: forget the specified XMPP user, meaning delete their names and (if any) admin rights from the system and stop message transmission. Note that users can set a new name and start transmitting again, see "/ban" for banning them instead. This command is mostly to force drop inactive users even if inactivity timeout is not yet ran out or if it is not set. A combo of "/forget" + "/ban" on the second connection is the only way to ban a misbehaving admin, which hopefully does not happen too often. /settimeout hours: set inactivity timeout in hours, after which message transmission to users stops automatically. They can always restart it by sending "/start". Setting a reasonable inactivity timeout (e.g. 24 hours) is highly recommended to prevent spamming users with all DW chatter for prolonged periods of time. It is especially relevant if users are going to be offline for a while, in which case messages are still transmitted to them if they are still "active" (note "activity" here has nothing to do with "online" status), and then they will be greeted by a ton of messages when they go back online. /addadmin [email protected]: add XMPP user as admin. Can be useful if you run the bot but there's someone else who is around more often and can keep an eye on users. Because of a big potential for abuse, only add very trusted users as admins, perhaps best only yourself to begin with. Note if you specify "XMPP Bot Owner's JID" before you start the bot, your XMPP user will be added as admin automatically. Finally as the bot owner you can always use GUI interface to manage users (see above). Finally, when you write to bot from XMPP keep in mind that it will not "understand" encrypted messages, so switch off encryption in your client in your chat with the bot. It could be technically possible to implement but does not make sense here, because it transmits to DW which is not encrypted anyway. Attached to this post is bot for windows along with its source code. Version for Linux coming in the next post. Your bot is cool but it's not enough for me, I need more! This bot is of course limited in scope because it was created for a specific task, but there's a potential for much cooler things. For this reason, the code is packaged with the bot (source.zip) and contains of XMPP project and source files (see below specific versions of everything). A couple of things that you might want and that are totally possible are following: Bot interacting with a public channel (called MUC in XMPP-parlance), instead of one-by-one with each user. Currently it's not supported, mostly because spamming to a public channel is not nice (and this bot may generate a lot of messages). With one-on one interaction it is much easier to opt out and in for each user. But this is not relevant if you run your own XMPP server, and create a dedicated public channel for communicating with DW, for example. In this case you can do even cooler things such as having this channel accessible from the web too (with converse.js) and therefore connect DW-to-XMPP-to-Web all in one go, creating a truly fully interconnected space accessible from anywhere! QXMPP library already supports joining MUCs, so you can implement it yourself but feel free to ask me for help if that's something you want. I may consider making a special version of the bot. Similarly, if you are running servers more seriously, you may want to do away with graphical interface and have something to run via command-line only. I have some code for that already, it needs a bit of brush up, but again please let me know if you are interested. I even have something to package it as a Windows Service, while on Linux you can already run anything as a daemon. As mentioned above, end-to-end encryption is currently not implemented. But in source code, you can easily enable it as it is supported by the QXMPP library (OMEMO encryption), you will only need to install a few extra dependencies. What about connecting to other chat apps? Discord?? Yes, technically should be possible. But I'm not interested in this  However you don't have to ask me. You have the code, and it is already made modular such that the class managing users (UserManager) is completely agnostic to specific implementations for IM communication (XMPPBot class). So it should be doable. Source code and dependencies: Code for DW bot and DW depencencies are in source.zip archive. To compile source code you will need Qt>=5.15 (current version compiled with Qt 5.15.18) and QXMPP library installed in your system. Both dependencies need to be compiled for 32 bit, because DW library is 32-bit... Both of the above requirements are open source as well but it may be tricky to get the source and compile it yourself (in particular Qt always tries to trick you into getting commercial version... you don't need it!). Ask me if you have issues, I managed to compile everything from sources both on Windows (10) and Linux (Kubuntu 24.04). im_chat_relay_v1.0_win.zip Edited 8 hours ago by stranger clarify dependencies Quote
stranger Posted 6 hours ago Author Posted 6 hours ago ... and here comes the Linux version (compiled/tested on Ubuntu 24.04). Launch with: LD_LIBRARY_PATH=. ./im_chat_relay im_chat_relay_v1.0_linux.zip Quote
Xandrah Posted 4 hours ago Posted 4 hours ago Ohh this is amazing, maybe we should start hosting our own XMPP server? Quote
stranger Posted 44 minutes ago Author Posted 44 minutes ago You can certainly do it! There are basically two established, modern options: Prosody and Ejabberd. Both are very powerful but I personally got the impression that Prosody is more suitable for smaller installations (although it still easily supports hundreds of users!) and Ejabberd for large-scale enterprise ones. Prosody is a bit easier to set up (I think!) and it is very easy on resources. That's why I chose it to run my XMPP server on a Raspberry PI (arm64 with a Debian-based OS). Therefore I do have experience with Prosody but not with Ejabberd so perhaps I'm a bit biased. That said my experience with Prosody had been very smooth. A basic setup is very easy and then you can add building blocks - modules - as needed. For that reason if something does not work in Prosody, it is probably just missing the right module. For instance, for file exchange you'll need an http server (it has a built-in one in corresponding module, or you can use your own) and for calls a turn server. But you can bring all this online gradually, starting with just text messages. For the full support of public channels (things like message history properly delivered to users and other perks) you'd also likely need to enable certain modules. For any questions about Prosody, devs are very eager to help you in the Prosody support room ([email protected] - you need an XMPP account to access it, how to get it - see in my post above). Usually someone answers any question within minutes at any time of day. Finally once you have your XMPP server set up, for a more rich social media-like experience for your users, you can also set up your own Movim server. I personally don't have experience with it, but others say it can be used to replicate some functionality of the likes of Discord. Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.