Mosquitto publish: memory management

Imagine the client was connected for some time, regularly publishing messages into its topics. Then the connection got broken, and was not available for quite a time (e.g. because of hardware problem). The client code calling mosquitto library continues publishing messages. What happens in this situation?

• I suspect if I call mosquitto loop, it will be trying to reconnect, and if successful, it will process all backlog publish commands?
• Or, at some point in time, publish routine will start returning 'not connected' error?
• If not above, and even in network unavailable state, mosquitto client will still accept calls mosquitto_publish() to publish messages - then where is the limit?
• How library code is executed after I call mosquitto_publish() in terms of memory - should provided data buffer be intact until client application receives acknowledgement for publish operation, or mosquitto_publish() internally copies messages to its buffers, and application can modify buffer used to call mosquitto_publish() and perform next mosquitto_publish() using the same buffer memory?
• If mosquitto_publish() uses its own buffers, and network is unavailable, I suspect it will be allocating more and more memory for publish messages - how can application track the decreasing free memory to stop calling mosquitto_publish() in time preventing machine to crash because of out of free memory?

Can not find information related to these questions in mosquitto.h and in protocol docs. I also suspect that different MQTT implementations could perform differently.

Also, is there any flow description in terms of publishing messages and managing in situations when library is unable to send data to broker?

By the way, I tried using this way but looking to total free memory is not helpful as background processes run, and in general does not give even an idea on how mosquitto_publish() manages its allocated memory.

Update: (not sure why I have got -1 for this question)

I was able to simulate free memory shrink using the code I pointed to, with connecting to the broker and not executing loop routines. At the end machine must crash, or become almost inoperational.

So what measures do mosquitto-using code has?

• regularly call mosquitto_loop which thankfully may return MOSQ_ERR_NO_CONN error allowing application to stop publishing. Question - how quick MOSQ_ERR_NO_CONN will catch up and where's guarantee that it comes before memory overflows?
• use mosquitto_loop_start and I do not know how to get network status from it or from library - the only way is using callbacks. But publish callback will not happen if library still tries to publish through non-operational connection, and there's no callback for loss of connection to signal to application to stop publishing.

Even if application would be able to assess the memory size occupied by the mosquitto library, or alternatively track size by the number (or cumulative size of) messages it tries to publish, then there's one good question - what to do with accumulating data from sensor, and if application is able to manage messages already sent for publication but not published to delete them from the mosquitto library queue and replace with newer telemetry messages.

Update: I did the further research on the issue. The course of actions are:

1. connect to broker;
2. in loop: publish one message, check free memory size, then call mosquitto_loop(mosq, 0, 1);

I see in flight messages being 20, but this option has nothing to do with the messages in the local queue awaiting to be sent to broker. As long as client is connected to broker, it is regularly able to send messages to broker, thus memory free size shrinks relatively slowly. But as soon as I physically disconnect the upstream connection (by plugging 3G antenna out - simulating hardware failure), this algorithm is unable to send messages to broker any more, and mosquitto library continues collecting publish requests. This way I have got 67 MB of free system RAM down to 1.8 MB really quickly. Good thing is that machine did not crash and free space remains at 1.8 MB for remaining time I run the test.

Notable is that no one call - mosquitto_publish() nor mosquitto_loop() return error indicating no network (I expected MOSQ_ERR_CONN_LOST error), therefore my test application has no idea that library is unable to send messages to the broker, and my application has no way to identify when it must suspend publishing into the mosquitto_publish() and start doing alternative actions (e.g. write messages onto the storage device for publishing later.

from Recent Questions - Stack Overflow https://ift.tt/31tb8ij
https://ift.tt/eA8V8J