"in the spirit of IRC" is a bit of a stretch. This is all a massive blob of web technologies which contrasts IRC's minimalism and it's lightweight clients written in C. It doesn't appear to require any specific service though so at least it has that.
"Web technologies" makes it sound like it requires WebRTC or something. It's a set of HTTPS endpoints that you shove JSON-encoded messages at. Certainly a bit "fluffier" than IRC (no prefix-length binary protocols here, sadly), but much more interoperable in the modern world (i.e. you can write a browser extension that can speak directly to a Matrix server using the built-in JSON+AJAX APIs; Matrix servers won't need specific egress exceptions in enterprise firewalls; etc.)
HTTP2 (if used) already makes the metadata parts of the protocol pretty concise and low-overhead; it's just the messages themselves that are large. So it'd be nice if—at least for the message bodies—there was an optional binary layer-6 encoding (e.g. CBOR, BERT-RPC) that clients and servers could try to negotiate with their Accept headers, with an expectation that many servers would implement it, and an allowance for constrained clients that implement that encoding but not JSON.
I'm not sure it's fair to call it a massive blob of web technologies (although I'm biased, given I work on it). To compare sending a message on IRC versus sending one on Matrix:
$ telnet irc.freenode.net 6667
...
NICK Ara4n
USER Ara4n Ara4n irc.freenode.net :Matthew Hodgson
...
JOIN #matrix
PRIVMSG #matrix :test
versus:
curl "https://matrix.org/_matrix/client/r0/rooms/!cURbafjkfsMDVwdRDQ%3Amatrix.org/send/m.room.message/uuid?access_token=$access_token" -X PUT --data '{"msgtype":"m.text","body":"test"}'
(okay, this assume you've grabbed a valid access_token from an existing session, and already know that #matrix:matrix.org's internal ID is !cURbafjkfsMDVwdRDQ:matrix.org, but that's easy to find by curling https://matrix.org/_matrix/client/r0/directory/room/%23matri...).
In the end, if you don't like HTTP or web technologies, there is NOTHING stopping people from contributing alternative transports - COAP, MQTT, capnproto, QUIC, XMPP, WebSockets or whatever floats their boat. But it seems that the simple HTTP+JSON API works well enough that nobody has bothered yet (other than https://github.com/matrix-org/matrix-doc/blob/master/drafts/...).
Call me crazy, but I think your example illustrates one reason why IRC is great: I can read it. And I could probably get it right even typing it from memory.
The IRC protocol is surprisingly difficult to implement correctly. I'm not an expert on the issues involved, but a few of the high points include:
* No standard maximum message length. If you input a line which is more than 510 characters after being prefixed with the channel name, server name, and other metadata, it may be truncated for some clients. If you're on a network which includes multiple servers, some clients may actually see your message truncated in different locations, based on how long their local server's name is.
* No standard text encoding. Some clients assume ISO8859-1, some assume UTF-8, and some will mix and match (ew).
* No way to determine when certain commands have succeeded. In particular, commands which can generate multiple lines of output (like WHOIS) don't have a standardized termination. Command-line clients sidestep the issue by just showing all the output in one place; higher-level clients have to guess.
* No standard identity services. NickServ is a common bolt-on, but it's a total hack. (It works by pretending it's another IRC server!)
* Bold, colors, and emotes (/me) are all client-side hacks.
* DCC file transfer is a complete and utter mess. It's like TFTP, but poorly implemented, over a TCP transport.
Agree about DCC but lot of the other issues you list are due to curmudgeonly (IMHO) users insisting on using non-modern clients and networks.
If they would tolerate a little breakage of backwards compatibility, the big networks could easily move to utf8 with standardized identity and colours, etc.
Unfortunately there is a small but very vocal faction of IRC users that are highly entrenched and resistant to change.
At the same time, that existing base of legacy IRC clients and servers is the only reason it could be worth hanging on to the old protocol. If we're going to break backward compatibility, we might as well take the opportunity to fix all of the other long-standing issues with the protocol.
I still have hope for the IRCv3 project, it appears to be fixing a lot of the issues mentioned. I'm on Matrix, but until #debian moves, I'll be found on Freenode and OFTC.
That was true a few years ago, but is long invalid, due to many recent changes. Nowadays it looks like this:
> * No standard identity services. NickServ is a common bolt-on, but it's a total hack. (It works by pretending it's another IRC server!)
SASL and account-notify.
> * No way to determine when certain commands have succeeded. In particular, commands which can generate multiple lines of output (like WHOIS) don't have a standardized termination. Command-line clients sidestep the issue by just showing all the output in one place; higher-level clients have to guess.
Batch messages.
You can find more at http://ircv3.net/, there’s a lot of specs on modern IRC features. IRC has changed a lot, you might want to take a look at it again ;)
> * No standard maximum message length. If you input a line which is more than 510 characters after being prefixed with the channel name, server name, and other metadata, it may be truncated for some clients. If you're on a network which includes multiple servers, some clients may actually see your message truncated in different locations, based on how long their local server's name is.
Well, doesn't that mean the limit is 510 characters including the command? I never as an user had issue with this. This is also implementation issue, if you use one server such as you do with slack, then you know its capabilities.
> * No standard text encoding. Some clients assume ISO8859-1, some assume UTF-8, and some will mix and match (ew).
Well, there's a CHARSET value in 005 numeric from the server which declares the official encoding.
> No way to determine when certain commands have succeeded. In particular, commands which can generate multiple lines of output (like WHOIS) don't have a standardized termination. Command-line clients sidestep the issue by just showing all the output in one place; higher-level clients have to guess.
> No standard identity services. NickServ is a common bolt-on, but it's a total hack. (It works by pretending it's another IRC server!)
You mean it works by pretending that it is another user. The reason for it is an optional component. Some networks (especially the largest one) are running successfully without such services. I kind of like this approach because it leaves the server protocol simple (networks often provide multiple services NickServ, ChanServ, MemoServ etc) and makes the component optional.
> Bold, colors, and emotes (/me) are all client-side hacks.
If you think about it, does it have to be server side? Server is just to relay messages, what they mean is up to a client. In fact pretty much any chat network that allows multiple clients has different looking emoticons. So making it part of the network doesn't help at all with unification.
> DCC file transfer is a complete and utter mess. It's like TFTP, but poorly implemented, over a TCP transport.
HTTP is also over TCP so not sure what you are trying to say here (BTW: TFTP works over UDP).
But ignoring the TCP part, yes DCC feels like it has problem, but that's mainly because of NAT becoming popular. Hopefully as IPv6 is adopted we get back to previous times where ach user had own IP address.
Anyway, comparing DCC to slack etc. Is a bit unfair, because their service is nothing like DCC (Direct-Client-to-Client i.e. P2P) when you upload a file to slack you upload it to their server, but with DCC you're sending it directly to the user without any 3rd party involvement.
Looking at the issues you listed, it feels to me that the problem is not as much with IRC servers, but with IRC clients, in particularly lack of standardization. You could solve most of the issue by defining client protocol on top of IRC, for example:
- handling encoding
- abstracting communication with services
- colors/emoticons
- file transfer (instead of using DCC which sucks with NAT) a client could upload file to a server and provide link to others, emulating behavior of other chat networks
The nice thing about it is that most of this could be implemented in a way that standard clients would still be usable, but if a standard would be created and other IRC clients would respect it, IRC could easily compete with existing solutions.
> Well, doesn't that mean the limit is 510 characters including the command?
No. The limit is on what the server sends to other clients when a message is delivered. For instance, if you send:
PRIVMSG #channel :some long message
the server will proceed to send everyone else in that channel something along the lines of:
:your@hostmask PRIVMSG #channel :some long message
which will be limited to 510 characters (512 minus CRLF). Unless you're perfectly aware of how the server will render your hostmask, there's no way to know where the limit will end up.
(I incorrectly said earlier that the prefix is based on the server -- it's actually your client hostmask.)
> HTTP is also over TCP so not sure what you are trying to say here (BTW: TFTP works over UDP).
DCC file transfers work on a chunk-by-chunk basis. The sending client sends one block of data at a time, then waits for an acknowledgement before sending the next one. (This is why I describe it as being like TFTP.)
Re: message length, this does still sound like a limit on the implementation, and you could easily make IRC servers that have either longer messages (ircv3 has 1024) or no limit at all (although perhaps that's not a great idea)
> DCC file transfers work on a chunk-by-chunk basis. The sending client sends one block of data at a time, then waits for an acknowledgement before sending the next one. (This is why I describe it as being like TFTP.)
You're right, I was not aware of the acknowledgments. But this in previous response I said that a lot of issues could be fixed on a client side. DCC is also completely implemented with clients and similarly can be fixed this way. There apparently is already DCC XMIT standard which fixes latency caused by the ACKs (assuming our client wouldn't do same thing as Slack does which is upload files to S3 and share the URL)
Agreed that the IRC line protocol is superficially easier on the eyes, but it's far from intuitive (especially the weird USER command). Whereas remembering the URL shape for Matrix is... just another URL (plus you can always cheat and just 'copy as curl' from a browser's network inspector). But each to their own :D
(that said, I think the point of Dave Cheney's article is more that sync realtime chat is bad for FOSS collaboration relative to mailing lists and forums. So until Matrix finally lands threading & forum semantics and smtp/nntp bridges, I suspect it's part of the problem he's complaining about :D)
SMTP/IMAP is very serious - we would love to bridge email convos into Matrix eventually. I suspect we're a little late to rescue NNTP though but perhaps someone will write a bridge for sentimental reasons :)
Well we are moving to a web first world, and passing the terminal days of IRC. Trying to avoid web technologies isn't going to do anyone good. Best we can do is try to make them better are secure. Light weight clients can be written in C if you want. There's nothing that needs a web browser
Here's the spec if you're interested[1].
There's also a weechat script if you want a command line client[2]
Too bad much of the web stinks, with terrible responsiveness and usability and impose those results through bloated protocols and spaghetti architecture on anything that tries to interface with them.
Multiple times on caltrain I've had people ask me how I'm on the internet as I have meetings on IRC without even noticing the connectivity has gone poor -- while their webchat stuff with multiple megabytes of ajaxy whatever won't even load.
Yeh in this vain, I'll add that I have very experienced colleagues who are so fed up with the current internet that they don't even fight net neutrality, and would rather another protocol rise in its place.
There's nothing inherently 'web' or 'slow' or 'ajaxy' about Gzipped-JSON-over-HTTP2.
I agree that the web is slow and crap a lot of the time, but chat using gzipped JSON over HTTP/2 would be just as fast as IRC as long as it's only sending the same level of info as IRC.
I don't agree at all. HTTP/1.1 and HTTP/2 are not especially simple protocols, but they're not that complicated. They aren't unnecessarily complicated, that's for sure. They have a set of features that necessitates a level of complexity, but it's better to do it in HTTP/2 once and only once than to repeat the same failures and reworks and added features and upgrades etc. etc. etc. to get the same level of features in a million different application protocols.
Because otherwise you'll get to the same place as we were with HTTP/1.1: your nice 'simple' protocol will need to be replaced with one that can support multiple streams, etc.
Programming sucked before HTTP+JSON became the lingua franca of machine communication. Being able to use any programming language with modern HTTP-based protocols is a real luxury.
Before, you could only use a programming language if had a production-ready implementation of the protocol you wanted to speak. Otherwise you would spend the next weeks or months tediously writing one.
This was so crippling that XML actually seemed like a good idea. Just imagine!
> Being able to use any programming language with modern HTTP-based protocols is a real luxury.
No, it's not, it's a huge pile of crap, because HTTP is so complicated that you essentially have to distrust any implementation of it that you haven't audited, and even then, it's questionable (plus, most implementations turn out to be buggy if you look closely, and serious vulnerabilities aren't exactly uncommon), and it doesn't even fit many common scenarios without additional hacks like long polling.
Also, "any programming language" is a huge exaggeration. Ever used HTTP on a microcontroller?
> Before, you could only use a programming language if had a production-ready implementation of the protocol you wanted to speak. Otherwise you would spend the next weeks or months tediously writing one.
Now, that's just bullshit. If you have an HTTP implementation, you necessarily also have a TCP implementation. You don't need to write anything in order to use TCP, you just open a socket and use it and forget about HTTP. HTTP serves no purpose in most cases.
Oh, and maybe more importantly: Ever used HTTP for peer-to-peer communication? No, you need a server! Because it's so much more convenient to add a server "in the cloud" than to simply use a TCP connection between two devices that need to communicate. (And no, of course, you technically don't need a server, but in practice that's how people deal with the pointless complexity of HTTP.)
I've just finished my first project using websockets. Coming from the embedded world and bare UARTs plus a little bit of C sockets, I found it hilarious that we've got so many layers to essentially do the same thing that I could do on an 8-bit microcontroller.
The critical difference? Speed of development. Websockets was so blindingly quick to get working. Haven't hit the edge cases yet though. If I were doing it with raw sockets I'd have to reimplement many of those layers and write a mobile app as opposed to a mobile webpage with only 300 lines of JS.
> Ever used HTTP for peer-to-peer communication?
Sort of. They're peers, but one always acts as the server, I just haven't needed to make them true peers, but I could if I needed to. One of the nodes connects to a public server and has other private nodes connect to it. (300 lines of Python/Twisted)
> The critical difference? Speed of development. Websockets was so blindingly quick to get working.
Now, is that websockets or is that "not C"?
I mean, speed of development certainly is valuable, but I would think that same speed of development only needs little more than an 8 bit µC (or, for that matter, is even possible on an 8 bit µC if the memory size is at the upper end of the range).
> Sort of. They're peers, but one always acts as the server, I just haven't needed to make them true peers, but I could if I needed to.
But that's the thing: You could if you needed to. Instead of it just being the natural thing to do. Though I guess NAT also is part of the problem here, which is another of those idiocies of the "modern internet".
> Now, is that websockets or is that "not C"?
> Though I guess NAT also is part of the problem here, which is another of those idiocies of the "modern internet".
The second point answers the first. It is websockets rather than C, because the client is a random user on a mobile web browser.
Now if I could control the other endpoint and were it not for those modern idiocies, P2P could easily be done in C. The problem then becomes discoverability (if you're after true distributed P2P) rather than the actual communications.
(I did port the lightweight lwIP TCP/IP stack to a 16-bit microcontroller and custom RTOS many years ago...)
It's not exactly super hard and complicated to get things working on a microcontroller though. In fact, the first time I did it, I felt it was so much simpler than all this complicated web stuff.
It absolutely is! I love micro-controller development. It's you and the bare metal, not dozens of layers of abstraction that often feel like they're fighting you at every turn.
On the other hand, it's hard to impress with blinkenlights and basic motor driving.
I'm currently going through a phase of thinking on how I could apply functional programming theory and techniques to the next iteration of this embedded system...
I spent months implementing an undocumented, custom, binary protocol to speak with an IRC bouncer.
And I still prefer doing that over writing a matrix client with HTTP and JSON.
Specifically because my solution works via throttled internet with dialup speeds, while matrix doesn’t.
I’m not saying Matrix is bad, Matrix has some awesome properties (especially regarding encryption). And maybe someday it can become awesome, once it gains a proper binary transport system, which is more efficient and keeps a single socket open. And maybe then, we can finally replace IRC, XMPP, etc. But until then, I’ll keep working on improving IRC.
For the record, there are no bad feelings on the Matrix side towards IRCv3 at all. The protocols are very different, and the design & architecture decisions in IRCv3 look fundamentally different to Matrix. In the end, Matrix is trying to be an openly federated object database with pubsub semantics - a standards-based decentralised encrypted Firebase, if you will. I really don't think that collides with IRCv3's aspirations, and we're looking forward to updating https://github.com/matrix-org/matrix-appservice-irc with IRCv3 features as they land. We're also looking forward to IRCv3 stuff to add to https://github.com/matrix-org/matrix-ircd, which is a Rust server which exposes an IRC listener for the entirety of Matrix. The open communications space has enough threat from silos like Slack and WhatsApp without in-fighting, and I'd hope the communities can get along and support each other rather than the opposite.
In terms of the line protocol - we all agree that HTTP+JSON is an inefficient baseline for Matrix. It's just a matter of time for someone to propose a proper binary transport (although HTTP2+gzipped JSON really isn't that bad, and works fine on dialup speeds). Luckily this thread has prompted a few people to offer to contribute one though, so perhaps we'll see some progress there at last :)
