Having recently used MQTT for a project, I can't say that I'm a huge fan. There are a lot of options in the protocol where it's not immediately clear what they do and why they are important, or in what combination they need to be used to make sure things work as intended, and the documentation is often not great at explaining. Part of this may also be on the Eclipse Mosquitto Python client that I had been using. I took me several days to figure out that the client was running into a race condition on a slower system that resulted in subscriptions to topics silently being ignored which caused the associated callbacks to malfunction despite following their documentation 100%. Overall, this has been one the messiest experienced I have ever had with a protocol, certainly one that isn't that old.
My experience with the eclipse clients (paho, etc) is similar to yours in both python and C++, - it makes it overly complicated and seems very low level. It also is somewhat buggy because of the architecture.
I believe (perhaps wrongly), it's all basically maintained by one person or close to it (only one person has contributed to the C++ client in the past 6 months, for example, and nobody has contributed in the past 3), so i kind of understand how it's gotten this way over time.
I filed a simple PR (1 word change) to fix an obvious bug and it took 2 years to review it and accept it with no changes. Again not a complaint, just trying to portray how the state feels - a lot of libraries, bugs, and work, and few overworked people helping get it all done.
I moved to other clients and the experience is much much better in python, rust, C#, and C++.
Most of them have a good combination of high and low level API's, so if you just want to send a message on a topic, you don't have to worry about acks, retries, etc.
But if you need control, you can get it.
Honestly, i worry at this point that keeping paho/etc alive in this state is doing more harm than good - if they were officially dead it would at least force the issue.
Right now you end up with users who have an experience like yours, and then either give up on MQTT or assume they are doing it wrong :)
After reading your post, I went looking for good C/C++ implementations suitable for embedded development, and honestly, I couldn’t find many! MQTT is typically straightforward to implement, so most companies develop their own version to better adapt to the specific systems they use for data transmission. That said, maybe this is an opportunity to create a well-designed MQTT client implementation tailored specifically for embedded devices! :)
There is a very nice MQTT client C++ implementation that works great on embedded devices too. As it so happens, it is right now under formal review for inclusion into Boost.
It is called async-mqtt5 and can be found at:
https://github.com/mireo/async-mqtt5/
My experience using MQTT has been through the paho python MQTT library, and, while we definitely have managed to get a lot done, it has been a terrible experience.
Everything about it rubs me the wrong way. The API design, the poor documentation, even the way they seem to not adhere to typical python conventions.
It looks like it's easy to start using it, but then the breadth of protocols and implementations start getting to you. At a point in time, I remember the only reliable way we had of knowing if we had successfully connected to a server was to try to subscribe to a topic twice and catch a specific error code on the on_connect message (which actually was documented as a success code at the time). I know how crazy it sounds and maybe there was a better way to do it, but, if there was, I don't think it was that easy to find.
It's easy to complain, though. I'm very grateful so many people have worked to create this library. If they hadn't, I wouldn't have built what we've built with it. I really admire people who take on these huge projects.
I've used the Paho APIs in Python, C and C++ and moved on to not using any of these APIs. Where possible I use mosquitto_sub and mosquitto_pub to support the protocol and then just read/write standard input/output. It's not a matter of bugs (though I have encountered at least one) but rather it's just easier to use a program that's been written and tested to manage the connection to the broker.
The only place where I was unable to do this effectively was with the last will and testament message (and I might not have tried hard enough.) It also doesn't work with microcontrollers that don't run something like Linux.
While not directly competing with mqtt, https://pipe.pico.sh is a great pubsub tool that leverages ssh for communication, effectively creating an authenticated, networked *nix pipe system.
It tries to be the simplest way to send and receive events.
I read someone somewhere saying "why not just use TCP" and for my IOT project plain TCP was fine. MQTT didn't have infinite queuing which was important for my use case, and if I'm going to have to track message ids to feed into MQTT which tracks its own message ids to send to the broker, why use it at all.
I'm not sure what the sweet spot is, unless it's just interfacing with projects that have already committed to MQTT.
MQTT is pub/sub protocol with a concept of brokers and clients. It's not in anyway comparable to TCP. You could rewrite an equivalent of MQTT on top of TCP in a fairly straightforward way - MQTT is simple by design - but I think you take the question from the wrong end. Why would you write a new protocol when MQTT already exists and does the job?
The effort to vet MQTT providers, integrate them, and configure everything properly, understand and work around MQTT and implementation quirks, and the protocol overhead were greater than doing something directly on top of TCP, and I don't believe I lost any features that I needed.
MQTT does have its nuances, and diving into the specifications is sometimes required to grasp all the options fully. That said, for most cases, simply focusing on the API for the library you're using should suffice. It sounds like the real issue you're facing is more with the library than the protocol itself.
The eclipse paho MQTT Java client is also terrible, lots of open bugs, terrible API, and no release in over 4 years. For my case, I was able to use NATS instead.
