XMPP doesn't "work", but on the other hand, the alternatives probably worse:
1. Don't run your own server? A co-worker once operated a fairly popular "public" XMPP server with a decent number of typically active users, a couple hundred. Some eastern European "darkweb" drug sellers had an account on the server, so their competitors thought it worthwhile to threaten the XMPP operator with very directed violence. If you don't host your own, who is running your XMPP server? What will they do when the government/yourThreatActor threatens them?
2. If you do (or don't!) run your own server, how do the end users find a halfway decent application on their fancy iPhone 18 Pro++ that supports push notifications? Their Windows 10 desktop? Ubuntu Linux?
3. If you run your own server, what do you do in the unlikely event of spam?
4. Crypto? OTR sucks for multi-device, OMEMO isn't extensively supported.
XMPP obviously doesn't work, but I can't suggest anything that's better.
I run my own ejabberd instance for a group of friends and it obviously works.
I do manual account creation and know people there (or they are vetted by my friends). The server federates with any other server that supports s2s encryption so they can talk to whoever they want.
On windows probably the most mature client is Gajim. You can use it on Linux too, but there are more alternatives (such as the more modern Dino.im) On Android there is the excellent Conversations, with a few more to choose from. Or you can go with a platform-independent web-based client and use Movim. All support OMEMO. I am not invested in the Apple ecosystem so I don't know about that (Beagle IM? Monal?).
Typically the XMPP naysaying is a mix of bad experiences in the past and NIH syndrome. Things like server-side message archiving, message carbons, message delivery receipts or MUCs are long solved. You want to make a stand for open, federated internet you need to put your money where your mouth is.
About the only sore points are cross-platform VoIP (Windows lacking here) and direct file transfers (SOCKS5 bytestreams are wonky which leaves http uploads) although arguably neither have to be a part of a chat application.
Good reply, thanks. I am very pro #XMPP. Which is also why I yearn for a couple real areas of improvement & focus:
* MIX. there's a new standard for group-chats/rooms that is far less ad-hoc/special-case, that uses other XMPP extensions (XEPs) very well & enables features people expect in chat these days. it's a good standard. MIX (XEP-0369) support is still alpha level or non-existent. this is needed for a quality experience, with reactions, chat history, & as a technical advancement out of a bad shifty early legacy solution.
* Video. There are not a lot of video supporting clients. I think the spec is mostly ok for 1:1 video. But "Muji" multi-user video got demoed a decade ago & works but is more or less unimplemented. This is not cool. I think Muji also should probably be rebuilt anyways atop MIX.
* web integration. i'd like to be able to use my XMPP account online in a well-defined way. xmpp should start to define ways to make user accounts also OpenID Connect (OIDC) providers, & begin to define interoperation for webrtc & XMPP & the web to work together harmoniously.
This is the gap Snikket[1] is aiming to fill. A preconfigured XMPP server that you can safely and easily run yourself (it's invite-based registration), with "blessed" apps provided for each platform that are guaranteed to be maintained to a common standard.
It's early days and there's still some way to go, but the current release (server + Android client) is receiving excellent feedback from those using it.
The next primary focus is iOS. There are several XMPP clients available on iOS, but none yet fit the bill. That's where most of the work ahead lies for the coming months.
So you took Conversations, renamed them, and call it a day, focusing next on iOS? Then I've got rather bad news for you. You'll need to be focusing for a long, long time.
It took us 'only' 2.5 years to build a good XMPP client for iOS (which we plan to release this month, finally), and while building it we had to break down XMPP down almost to a core and replace most of its components with better thought through solutions that account for real life scenarios.
We had to throw away MUC garbage, do push notifications differently, make different video calling, etc. - and that requires modifications on the server-side. Now it works acceptably, just some rather little quirks/bugs remain. It is no coincidence that not one released XMPP client today can be called 'working'.
However, to make it work _really_ great we'll have to do 'break' XMPP even further and rethink how presences work. Currently, they are a major pain.
Hi Andrew, I'm familiar with your team's work, and don't doubt your determination to build a quality product. From what I've seen, that appears to be your priority above and beyond XMPP, self-hosting, and interoperability with others in the ecosystem. There is nothing wrong with that choosing that as a primary goal, but it's a different goal to Snikket.
Snikket is building on and improving (pushing upstream whenever possible) existing projects in the ecosystem, not starting from scratch. The smooth invite-based onboarding flow for example originated from Snikket work, and is now upstream in Conversations and other clients are adopting it too.
I've been working with and on XMPP for 15 years, and have no illusions about the challenges we (or any fully open decentralized messaging system) face in today's world. However we can never give up on free, open and interoperable technologies.
Quick scan of the website & the github. Did not find direct mentions of XMPP. Does this interop with other XMPP systems or is this based-on-XMPP but standalone? I would not adopt or recommend this, framed as it is as it's own thing.
If your primary criteria is "must support XMPP", you are very likely not amongst the target audience of Snikket, or you are at least in a minority of it. There are many XMPP servers and clients already out there easily discoverable by someone who knows that's what they want.
Snikket is aimed at people who primarily just want an out-of-the-box self-hosted secure messaging service. "XMPP" is not a feature for this crowd, but the benefits of XMPP are (federation, free choice of client software, etc.). For the same reason Mozilla doesn't advertise Firefox as a "HTTP client", the Snikket marketing is focused on features and not how the underlying tech is implemented - what we do, not how we do it.
The fact that Snikket is built on XMPP is not something we aim to hide however, it's discussed in [1] for example. It's just not what we lead with when introducing the project to general users.
Many good open-source projects fall into a trap of their developers marketing their project in a way that they would want to see it marketed to themselves. The truth is that nobody outside certain internet communities cares about open standards. They should, of course, but they don't realise it. We're trying to reach these people with Snikket.
When I used it professionally it was always a pain. XMPP was very chatty for mobile. Multi user chat and chat history was a pain to get working. Clustering was also dark magic. This was Ejabberd which was considered the best option at the time.
At the time, it felt like every feature was stuck in beta/RFC mode with very poor cross server compatibility. How is a federated protocol supposed to work like that?
I think XMPP just failed to cater to any audience. The Googles and Facebooks could roll their own and for everyone else it was too cumbersome. It's not as "easy" as running an email server and no one wants to do that either. It wasn't even agile enough to woo small communities with whiz-bang features.
> Everyone in the late 90's, early 2000's was on it.
Outside the United States, very few people were on it. When I signed up to play some video games with some US friends from a forum, I remember having to Google a valid US address and it's matching ZIP code as it wouldn't even let me sign up without one.
MSN was more popular in western Europe and as I understand it ICQ (eventually federated and shared some tech, but not the same network) was the top early IM in eastern Europe. Can't comment on the rest of the world at that time but I doubt the sign-up form was any more accepting of Korean addresses for example.
afaik AIM was a derivative of ICQ, but yea those were the good times. today i know more ppl that won't touch whatsapp then there were w/o icq numbers back than, probably the closest we've got to a ubiquitous instant messanging. i still remember my icq# althou i havent used it in like 15y and have burned throu like five phone numbers since.
the MUC is not good. this is true. there is MIX, XEP-0369, that has a way better under-the-hood approach that re-uses a lot of the existing XEPs (whereas MUC was early & kind of it's own thing).
alas MIX adoption is very low. ejabberd got MIX support in 2016 but it's stayed at 0.1.0. and, more pressingly, i don't think any clients support it. this has to change for XMPP to pass MVP muster. https://www.reddit.com/r/xmpp/comments/arpruf/what_clients_s...
My use cases were a large social app with chat and a AAA game with team chat. This was almost a decade ago but in both cases a single server was deemed insufficient by our ops people. I suppose you're right, though. I've never had to maintain a mail system of similar scale.
1. there are a bunch of public servers that have been running for a long time. Including one by the CCC (Chaos Computer Club, German privacy and security advocacy NGO).
2. IMO the biggest hindrance with XMPP. So which XEPs do I need to support on the server? Okay, got what I wanted. How do I find out exactly which are supported on which client?
3. Manual account creation, thanks to federation I don’t need external users to be on my server.
4. OMEMO works with every halfway decent client. Of course, closed ecosystems like iOS might be different, but that’s what you get for locking yourself in.
> 2. If you do (or don't!) run your own server, how do the end users find a halfway decent application on their fancy iPhone 18 Pro++ that supports push notifications?
That's a problem with the iphone, not xmpp. Apple restricts what you can do with your hardware and offers no other options than their push notification approach. You cannot blame existing protocols for apple's distributed-software-hostile whims of the day. Blame apple for only offering an anemic subset of the internet.
As an iPhone user, I'm very happy that apps can't implement their own push notification systems, or have persistent background connections open, as it's one of the main reasons I get good battery life.
Riot uses the platform’s push server unless you’re using the F-Droid version on Android. That version will indeed destroy my battery if I leave it running with my (rather large) account.
For example, Apple could allow an app to register as a receiver for a class of push notifications, and then require all the pushes to go through the Apple-maintained background process.
League of Legends chat, WhatsApp and early versions of Facebook chat were build with XMPP and Ejabberd and they were able to handle massive loads with it.
Obviously for an internal company chat app this is not the best option as it will require some maintenance, and the app support is not great, though there are some good options now.
But for heavy chat load it is a great choice that will save you a ton of money, time and resources.
1) Host your own server, don't make it public.
2) The fact that there were so many non-mandatory and sometimes competing XEPs was a major setback indeed.
3) There were measures for that, such as that unknown users should answer "how much is pi*0" kind of questions
Matrix seems to solve items 2 and 4, lets see how much traction it gets.
I use an android phone, so this may be out of date, but every time I check, the iOS XMPP clients are terrible in at least one important way, and often in several.
I built Siashable with ExtJS and xmpp4js in 2008 against Openfire. I’ve run it at various times against Tigase and Prosody and it works, and even ran the XMPP lib I made for it in iOS JS Core recently, and it has its own everything down to DOM with namespace support in JS so works back to early IE versions. I’d say XMPP with BOSH works pretty good.
What was it about XMPP that made people move to facebook, slack, hangout or other closed systems?
In 2011, at my job everyone used their own favorite account to chat with each other. I'd logged on Empathy with my yahoo mail. My coworkers were on gmail, aol, yahoo, hotmail, facebook, and some I can't even remember. Chatting wasn't a problem. How did it go from being a convenient thing for everyone to everyone should use slack?
Probably around 2011 I was trying to push for XMPP at a .edu for internal chat (they was no current system at all).
Around that time the iPad was showing up everywhere and I found was that there were zero good iOS XMPP clients. Every single one we tried, I'd notice it was proxying the credentials through a 3rd party server to deal with the lack of background connection support and to handle the push notifications. This was a major security issue, and also meant it would have been unusable during an internet outage.
This sort of thing, combined with the lack of server side history was probably what killed it.
Yeah, Android in 2011 could just run a connection in the background and actually work (assuming the client pings when idle / and adapts the ping interval to avoid stateful firewalls with low idle timeouts).
But iOS doesn't let you do background sockets, so you need push messages, and that's hard to coordinate between unrelated XMPP servers and clients. Of course, modern Android doesn't let you have background sockets either.
I have an android app that uses background connections; it works fine on modern Android. The connections are broken when people walk in and out of WLAN coverage, but as long as the app uses minimal CPU and network traffic Android doesn't intercede much.
Not so I've really noticed. The user starting apps that require very much memory probably does too, but that's also a minor effect.
If you don't use scheduled jobs to reopen your connections, WLAN/mobile switching is a pain and there are some smaller similar pains. If you solve that big pain and make sure to use very little battery power, the smaller pains go away.
facebook and google at least simply isolated their users from the rest of xmpp at some point. oops, if only we had been warned.
I will never forget how a few years back, a friend (and a techie at that, albeit more scientific) explained to me their amazing idea of a messenger that you could use between different chat providers, and messages would just go from one to the other.
"Imagine you could send a message on facebook and I would read it on google!"
I could see something die in their eyes when I explained that not only was there a solution to this, but also, facebook and google had both already decided to kill that solution.
> Email seems like such an anachronism. All the big providers manage to get along with each other.
Think of the value of this! Anyone, anywhere, using whatever client they choose, can send a basic message to anyone and expect it to arrive and be taken seriously.
Imagine if IM had worked the same way! I know XMPP is a shitstorm, but one can dream.
Not until I left academia and started working for MegaCorp that mandates outlook web as the only email client did I truly internalize the value of this.
Email is not a synchronous channel, and shouldn't be treated as such. Plenty of mailservers will happily delay your messages for a few minutes for spam and virus scanning (because who cares if an email arrives a minute later), and you have all sorts of asynchronous things built into the major servers and protocols (queues, delays/greylisting, 4-day(!!) backoff-retries, etc).
Most of the IM of today isn't synchronous as well. That's the exact reason OTR was deemed not enough, because it requires synchronization, and the Open Whisper guys came up with a novel encryption scheme that works with asynchronicity in mind.
Now of course that doesn't mean you should never be synchronous, and it is true email might not be the best vector here. But from my little experience it's good enough for simple messaging.
Even though it's widely recognized as impossible for individuals to self-host, at least there are many available providers, and any medium+ size organization can self-host. Things could be better but they could be worse too.
I host my family & friends email server for about a half dozen domains for the last 20 years using postfix with courier imap. Once every 5 years, Gmail doesn’t like something and starts sending my emails to spam but it resolves by itself in 2 weeks ;) Otherwise, the only maintenance is OS upgrades every 6 months or so.
It's not reliable. Large providers will black hole you and you don't know until someone says they never got an email. Even following all the rules you get hit and with Google, Microsoft, &c it's impossible to get to anyone to resolve the issue once you know about it. Even for small companies this is a nightmare.
Big providers are not reliable either. It's a symetrical problem. If they don't deliver messages, they are not reliable from both sender and recipient PoV.
And you don't have any control over what they blackhole or reject without even delivering to your inbox/spam folder, if you're their user. You don't get to see logs either.
And big benefit of self-hosting is that you can actually make your server receive everything with no rejection or blackholing. So in that sense, self-hosting can actually be made extremely reliable for reception, and that is much more important to me than reliable sending. So I'd say self-hosting is superior when it comes to reliability, because at least you can make one direction "100%" reliable. With big freemail you can't make any direction comparably reliable.
As soon as you get some IP address reputation, sending gets fairly reliable too. And it will also work fine without any modern cruft on top. You really don't need spf, dkim, dmarc or any of that, for your personal e-mails to be delivered.
Kinda unfortunate that others sometimes block you when you're starting out just based on your IP address [range], so it's discouraging, but it mostly works after a while.
> Big providers are not reliable either. It's a symetrical problem. If they don't deliver messages, they are not reliable from both sender and recipient PoV.
It's symmetric in the sense that if Alice can't communicate with Bob, then Bob can't communicate with Alice. But network effects break the symmetry. If you have a big provider you can reliably communicate with 99.99% of people. If you have a small provider you can reliably communicate with 95% of people.
My use case is also to receive e-mails from non gmail/outlook users. In fact if I check my logs 1.5% of messages came from google's smtp service and 1 from outlook, out of last 15000 emails.
So if big provider drops 95% of incomming emails from non-big providers into black void, their service is less than useless to me, and their quality would be at par with small providers that can deliver 95% emails successfully.
XMPP was only the protocol. Google and others provided federation with it then took it away later. This decimated the social network size. When the vast majority connects to others primarily via Google Talk and its descendants or Facebook messenger, XMPP, as a protocol, didn’t stand a chance.
Bridges were shut down to push proprietary chat spinoffs of social media websites. Ex. Facebook had an XMPP bridge for a couple years but shut it down for this reason.
My tinfoil hat theory (that others may have had, or it may be something I read somewhere...) is that the XMPP protocol was part of what made PRISM successful, and the timing of these orgs adding XMPP is kind of aligned with when they were “online” for PRISM.
I’m pretty sure the larger orgs that were targeted could tell that the only way this information leaked easily, or was justified, was because of XMPP metadata, or some other large scale flaw.
"How did it go from being a convenient thing for everyone to everyone should use slack?"
Correction. Everyone went to Facebook. Because Facebook was convenient. Now we are all heading to slack (& discord) because XMPP was no longer convenient & the right-wingnuts are the top 10 shares every single day every day on Faceboook so we no longer wanna hang out on it.
"right-wingnuts are the top 10 shares every single day every day on Faceboook so we no longer wanna hang out on it"
I'm not sure which Facebook you're using, but I certainly don't see anything shared by 'right-wingnuts' going very far. In fact, it's usually silenced and removed almost immediately.
I actually had to stop using Facebook over the past couple of months because of all the left-wing nut job ideas that seem to have become mainstream like the hundreds of thousands of protesters around the US that spread Covid by not wearing masks and are now trying to blame business owners and Republicans. Science and facts seem to now be a thing of the past.
It's also very difficult to have any actual opinions about any current events, especially of you are not African American and disagree with any of the current narratives the mainstream media is attempting to shove down our throats.
Sure you can have an opinion, but if the wrong person sees it, they will attempt to get you fired from your job and shame people into de-friending you. It's the 2020 Salem witch trials.
Ben Shapiro & BlueLives Matter were 6 of the 10 most reshared pieces of content yesterday & none of the other top 10 are anything but polarizing trashy antagonistic regressives either.
I would be willing to admit, this kind of content does not feature a lot in my personal experience when I did use Facebook. But in general, I'm not inclined to log in & stay logged in, as it doesn't add continuous value. And I don't want to give advertising-dollars to a platform that does, day after day after day, reliably, serve to brainwash America with these reactionary attention-hungry wolves.
I'd point out that the Salem witch trials you raise judged people based on fiction (there are no witches), & killed them. Losing some friends & your job for being a demonstratively terrible awful person does not seem like a fair comparison: there is real guilt, & the consequences seem survivable. I'm not sure that the ability of any one incident to become a lightning rod of attention is ideal, but it felt like being an absolutely terrible person for too long had no consequences: I would like to see more safety nets, more forgiveness, more healing, but there being consequences for actions is an enormous improvement over where we were, and frankly the consequences seem reasonably inline & reasonable in a vast majority of the situations at hand.
I'm sorry that this seems so overwhelming for you. I hope you can find some peace.
A while ago I tried getting the latest version of Gajim (XMPP client) to run with OMEMO (encryption plugin) on Trisquel. I worked through lots and lots of compilation issues for dependency libraries of Gnome and other stuff that was new to me, but after trying for something like 3 or 4 days, I lost motivation. I also did not want to use the version number-wise much older version of Gajim to be able to use it. Perhaps that was silly of me.
I did run the older version to try Gajim out and to see how one connects to servers and which servers to use and whatever. That worked actually pretty well. I only had to find an OMEMO supporting server in at least superficially trustworthy hands. I really likes the instant feeling of sending messages and being able to log into 2 accounts on different servers and sending myself messages.
I'll have to try it again at some point. I would really like to know a reliable long term server, run by people with a privacy mindset though.
As a board member of the XSF for this term and for the first time, I don't really find myself in disagreement. It seems to me the culture of change is almost nonexistent and it's hard or close to impossible to get the XSF to provoke it.
Can anyone point me to other RSS-able newsletters I should follow?
Between This Week In Matrix, Sergey Tihon's F# Weekly, Pine64s updates on the Pinephone, and now this XMPP newsletter (which admittedly I thought was a Johnny-come-lately approach to showing that XMPP was cool like Matrix, see, we've got the newsletter stuff too! - but A: it's clear they have been doing this for a bit and B: I digress), I'm enjoying reading tech 'periodicals' and keeping up with what's new in my favorite technologies.
1. Don't run your own server? A co-worker once operated a fairly popular "public" XMPP server with a decent number of typically active users, a couple hundred. Some eastern European "darkweb" drug sellers had an account on the server, so their competitors thought it worthwhile to threaten the XMPP operator with very directed violence. If you don't host your own, who is running your XMPP server? What will they do when the government/yourThreatActor threatens them?
2. If you do (or don't!) run your own server, how do the end users find a halfway decent application on their fancy iPhone 18 Pro++ that supports push notifications? Their Windows 10 desktop? Ubuntu Linux?
3. If you run your own server, what do you do in the unlikely event of spam?
4. Crypto? OTR sucks for multi-device, OMEMO isn't extensively supported.
XMPP obviously doesn't work, but I can't suggest anything that's better.