Two different models. The metaphor I like to use is that RabbitMQ is a postal system, while NATS is a switchboard.
RabbitMQ is a "classical" message broker. It routes messages between queues. Messages are treated like little letters that fly everywhere. They're filed in different places, and consumers come by and pick them up.
Core NATS isn't really a message broker, but more of a network transport. There are no queues as such, but rather topologies of routes where messages are matched from producers and consumers through a "subject". You don't "create a queue"; you announce interest in a subject (which is a kind of path that can contain wildcards, e.g. "ORDERS.us.nike"), and NATS routes stuff according to the interests. So there's nothing on disk; and if a consumer isn't there to receive a message, the message is gone. Thus you can send messages back and forth, both point-to-point or one-to-many. NATS itself isn't reliable, but you can build reliable systems on NATS.
A common example of the lightweight, ephemeral nature of NATS is the request-reply pattern. You send out a message and you tag it with a unique reply address, the "inbox subject". The subject is just a random string (it may be called "INBOX.8pi87kjwi"). The recipient replies by sending its reply to that inbox. The inbox isn't something that exists; it's just a subject temporarily being routed on. So the sender sends a message and waits for the reply. NATS encourages you to use these ephemeral subjects as much as possible, and there can be millions of them. You can do RPC between apps, and that's a popular use of NATS.
JetStream is a subsystem built on core NATS, and is what you get when the designer of NATS thinks he can outsmart the designers of Kafka. JetStream is basically a database. Each stream is a persistent sequential array of messages, similar to Kafka topics or a RabbitMQ queue. A stream can be replicated as well as mirrored; one stream can route into another, so you can have networks of streams feeding into bigger rivers. Unlike core NATS, but similar to RabbitMQ, streams and their consumers have to be created and destroyed, as they are persistent, replicated objects that survive restarts.
Similar to Kafka, streams are just indexed arrays; you can use it for ephemeral events, or you can store long histories of stuff. Consumers can go back in time and "seek" through the stream. Streams are indexed by subject, so you can mix lots of types of data in a single stream (as opposed to multiple streams) and simply filter by subject; NATS is very efficient at using the index to filter. Like RabbitMQ but unlike Kafka, streams don't need to be consumed in order; you can nack (!) messages, or set an ack timeout, causing redelivery if acks aren't sent in time. In other words, JetStream can work like Kafka (where you always read by position) or like RabbitMQ (where messages are skipped once acked, but retried once nacked). JetStream has deduplication and idempotency, which allows you to build "exactly once" delivery, which is awesome.
Similar to how someone built a database on top Kafka (KSQL), the NATS team has built a key-value store on JetStream, as well as a blob store. They work the same way, through message ID deduplication. A stream is basically a bunch of database rows, with the message ID acting as primary key. So the stream acts as a primitive to build new, novel things on top of.
I think it's fair to say that RabbitMQ gives you opinionated tools to do certain things, whereas NATS and JetStream are a hybrid "multi model" system that can be used for more purposes. For example, you can embed NATS in your app and use it as a really lightweight RPC mechanism. You can use JetStream as a classic "work queue" where each worker gets a single copy of each message and has to ack/nack so the queue moves forward. You can use JetStream as a log of all actions taken in a system, with retention going back years. (NATS/JS is actually awesome for logging.) And so on.
We use NATS for different use cases at my company. In one use case, clients connect to an API to follow live, low-latency change events. For each such client connection, we register a NATS subject; then tons of processes will see this subject (and its settings, such as filters) and will all start send changes to that one subject. There's no single "controller"; it's all based on point-to-point and one-to-many communication.
(Full disclosure: I'm not familiar with newer RabbitMQ versions or the streaming stuff they've added, so it's possible that RabbitMQ has caught up here in some ways.)
Sorry, I'm not familiar with this tech so looked up Jetstream and it seems to be archived https://github.com/nats-io/jetstream. Not sure that would be a good endorsement to try to use something that is no longer maintained or am I looking at the wrong one?
JetStream was originally designed/built separately, but is now built into NATS. So that repo is obsolete. NATS with JetStream is here: https://github.com/nats-io/nats-server.
RabbitMQ shines when you need complex queue routing based on keys or headers. Instead of baking the logic to the app you can offload the routing logic to rabbitMQ. Same is true for NATS and Kafka.
I must say that rmq in k8s, is possible but hard to admin. It’s not a “toy”. But has great documentation. Will take several iterations to key in the right configuration for the use case.
NATS and Kafka can handle higher volumes on the same resources but IMO the use cases are different or you have to write lots of code app side to implement what rmq does with these tools.
NATS, especially in its most elementary form, is braindead simple, stateless, and ootb functional. If I only want a message broker/pubsub, I pick that. If I know on day 1 that I need queues or persistence, I would probably pick Rabbit over NATS’ offering (Jetstream).
Would you use NATS over Redis PubSub or Postgres Notify/Listen? Postgres's option I'm wary of for reasons I've discussed here before but Redis PubSub seems fairly simple to use and likewise not as uber-scalable as Kafka and other more heavyweight message broker systems.
Not the parent, but I would. We've been using NATS for about five years at my company, and we recently adopted JetStream, and have been really impressed with it.
NATS, especially with JetStream now, is a Swiss Army knife of messaging. It can do RPC, Kafka-type batch streaming, low-latency realtime notifications, large-scale network transfer, offline sync, work queues, mirroring, weighted routing, partitioning... it's incredibly flexible. The simple but powerful primitive you have around subject routing/mapping/transforms, stream placement, replication, offline lead nodes etc. are just fantastic. It's super reliable. It scales from tiny apps to huge ones. The CLI and API tooling around streams and consumers is also fantastic.
Everything just feels well-designed and sensible. The new key/value store and blob store (both built on top of JetStream as client-side abstractions) are also super neat, and we've started using this functionality.
Yes, and no issues at all. We run superclusters with a bunch of nodes, and the Raft-based leader election system has worked flawlessly so far (knock on wood!).
Keep in mind that NATS does not yet support value operations other than setting the whole value. Optimistic locking is supported, but NATS does not have inc/decrement, append, set members, etc. I believe such support is on the horizon, however.
I would probably use NATS just because I’m more familiar. I love redis but I’m fairly skeptical when my database starts wanting to be my message broker
