Issue with Message Ordering in ZeroMQ PUSH-PULL Using send_multipart

[fti-ravhad]

Issue with Message Ordering in ZeroMQ PUSH-PULL Using send_multipart

Context:
I am currently working on a multi-threaded module where multiple PUSH sockets send messages to a single PULL socket using ZeroMQ. The messages are sent using the send_multipart method in the following format:

send_socket.send_multipart([msg1, msg2])

• msg1: JSON data
• msg2: JPEG byte data

On the receiver side, I am using:
x = receive_socket.recv_multipart() .
I expect x[0] to always contain the JSON data (msg1), and x[1] to contain the JPEG bytes (msg2). However, in some cases, I am receiving the JPEG bytes as x[0], causing message misalignment.
Current Setup:

Messaging Pattern: PUSH-PULL
Multiple PUSH sockets sending data to a single PULL socket

Questions:

• Is there a more suitable messaging pattern in ZeroMQ that can ensure message ordering while handling multiple senders?
• Are there any techniques or configurations I should apply to prevent message parts from arriving in the wrong order?

Any insights or suggestions would be greatly appreciated.

Thank you!

[minrk]

Can you share an example code that reproduces the problem? Libzmq should guarantee that multipart messages are delivered in order in tact, so this should only happen from your description if a recv call is happening but the partial message is not being retrieved.

[fti-ravhad]

@minrk , Below is the code in sender and receiver side .

The sender processes some calculation and sends the response as below :
Sender side :
self._status_sock.send_multipart([message,image_data])

Receiver side :

request_check = dict(poller.poll(30000))
if request_check.get(on_demand_image_recv) == zmq.POLLIN:
        data = on_demand_image_recv.recv_multipart()

[minrk]

A more complete example that can reproduce the problem would definitely help. If there are e.g. any threads or other interactions with the sockets, or error handling, that will be relevant.

You can check socket.rvcmore before calling recv_multipart. If that is true, it means that a message has already been partially received and you are going to start receiving a message in the middle.

Generally this would mean an earlier call to recv, or an interrupted call to recv_multipart.

[JensHuthmann]

I see the same issue with two publishers and one subscriber. Publishers uses c++ api. And subscriber is using python api.