I think people don't use it more because people don't really know what it is. From their website:
> RabbitMQ is a reliable and mature messaging and streaming broker, which is easy to deploy on cloud environments, on-premises, and on your local machine.
What does that mean? "Messaging and streaming broker"? I understand the need for worker queues to process videos, images, emails and such but I can't easily tell if that's what this is.
Also, what are the benefits of this over just processing incomplete records straight out of my database? i.e. using MySQL as a queue.
> what are the benefits of this over just processing incomplete records straight out of my database? i.e. using MySQL as a queue
Mainly throughput and latency. I haven’t used MySQL recently so some of this may apply more to Postgres.
Postgres has LISTEN/NOTIFY which helps with latency. I don’t think MySQL has LISTEN/NOTIFY, which means you’d have to resort to polling.
You have to use the `SELECT … FOR UPDATE SKIP LOCKED LIMIT 1` features to grab a message from the queue table, so multiple consumers don’t pull the same message.
The biggest issue, if you’re trying to achieve decent throughput, is dealing with bloat (empty pages still sitting on the disk that haven’t been cleaned up yet). You can run vacuum but an online vacuum will only mark the pages as available for reuse (doesn’t free up the disk space). And if you run a full vacuum (which will free the disk space) it locks the entire database while it runs the vacuum. This can compound if you’re using indexes.
One way of dealing with this is setting up partitioning by message timestamp, so that as old messages roll out, you can just drop those partitions and not have to deal with vacuum.
It can work if your queue needs are low throughput or can tolerate higher latency, but there are some things to manage, and realistically setting up a Redis instance is probably less complex than trying to work around the database-specific quirks, unless you’re already very familiar with your database’s inner workings.
> RabbitMQ is a reliable and mature messaging and streaming broker, which is easy to deploy on cloud environments, on-premises, and on your local machine.
What does that mean? "Messaging and streaming broker"? I understand the need for worker queues to process videos, images, emails and such but I can't easily tell if that's what this is.
Also, what are the benefits of this over just processing incomplete records straight out of my database? i.e. using MySQL as a queue.