Shortcomings of MQTT: QoS

MQTT has a lot of good uses, but like anything, it also has drawbacks or misapplications. We want to help you make the best choice for your application, so in this series of articles, we'll address some of them.

QoS

Qos is often described as 'at least once' (QoS 1) or 'exactly once' (QoS 2) delivery. But, this gives most people the wrong impression of what actually goes on.

Let's first see what QoS is:

• QoS 1: a PUBLISH is responded to by the server with a PUBACK.
• QoS 2: a PUBLISH is responded to by the server with a PUBREC (publish received) and onward delivery is initiated. Then, the client sends a PUBREL (publish released), after with the server sends PUBCOMP (publish completed).

Especially in QoS 2, there is the extra functionaility if silently dropping publishes with the same ID when the PUBCUMP hasn't happened yet.

What you have to realize, is that the QoS transaction is between client and server. It's not between publisher and subscriber(s).

Imagine a server with thousands of active clients and messages. Then you come along and subscribe to # with QoS 1. You have a slow connection, so that amount of packets cannot be delivered. What is the server supposed to do? It can't keep writing them to the client buffer, because that will eventually cause the server to run out of memory. Or, the server could stop sending PUBACK packets to the publishers to slow them down? Obviously not, because that would be too easily exploitable. In practice, what happens is that packets are dropped, whether they are QoS 0, 1 or 2.

So, this means the publishers will all have gotten their PUBACK, so the QoS 1 transmission is complete, but client(s) who are interested may not have received anything.

Example of problematic reliance of QoS: storing data ingress

If you have many IoT devices in the field that publish sensor values and you want to to store all of that in a database, it's tempting to connect a subscriber to the server to ingest all of the values and save them somewhere. But, taking the above into account, it should now be clear that this is not a good idea.

Imagine your subscriber has problems, like hanging or crashing. Even when you connect with a 'persistent session' that you can pick up again after reconnect, the server is unlikely to have buffered all the packets of interest, and yet the clients will already have received their PUBACK.

The problem is that there is no transactionality between the publisher and you storing the value.

A better solution, transaction safe

What you want to do instead, is hook your storage into the PUBLISH / PUBACK transactions. This can be done by implementing the saving of the PUBLISH actions in a FlashMQ plugin. That way, the PUBACK is only sent back to the client when the plugin function exists, at which time you have committed it to permanent storage (or some kind of async queueing system, potenentially with crash-recovery and everything, but that's another discussion).

So, what can I do with QoS

The way I think of QoS, is simply literally the matter of getting ack packets back. These PUBACK packets aren't only useful for the MQTT layer, but they are also useful for your client code. Getting a PUBACK back may be the trigger for your event-based application to proceed to the next step. But if you really want to know whether an interested subscriber received it, you'll have to define that at the application level, like with MQTT5 response topics and correlation data.