You hit the nail on the head. If you don't need strong consistency guarantees any sort of message queue will do. That is a solved problem.
There are two key quotes from the article:
1. "... if our queue begins dropping messages, we run the risk of silently corrupting data"
The two statements that they are executing "app.update" (modifying the database) and "enqueue" (sending to message queue") are not atomic. They can make it atomic using something like 2 phase commit but implementing that will be more trouble than it's worth. Additionally, there will be performance (latency) implications since it is a synchronous operation.
2. "Notice how both of our writes are inside of a local database transaction"
I am actually working on something very similar to this at my day job. The "buffer processor" that the article is describing, we implemented using a scala process. Perhaps what's most interesting about the entire system as mentioned at the end of the article is maintaining the large amount of state for durability (we use hbase) and deliverability (we wrote a custom scala server).
There are two key quotes from the article:
1. "... if our queue begins dropping messages, we run the risk of silently corrupting data"
The two statements that they are executing "app.update" (modifying the database) and "enqueue" (sending to message queue") are not atomic. They can make it atomic using something like 2 phase commit but implementing that will be more trouble than it's worth. Additionally, there will be performance (latency) implications since it is a synchronous operation.
2. "Notice how both of our writes are inside of a local database transaction"
Now they have atomicity very easily!
I agree with the grandparent that using SQL as a message queue is generally a bad idea. However the pitfalls are well known and can be engineered around. See this article: http://www.engineyard.com/blog/2011/5-subtle-ways-youre-usin...
I am actually working on something very similar to this at my day job. The "buffer processor" that the article is describing, we implemented using a scala process. Perhaps what's most interesting about the entire system as mentioned at the end of the article is maintaining the large amount of state for durability (we use hbase) and deliverability (we wrote a custom scala server).