Decode message sent from server when connecting

[imjuzcy]

First of all, I'm very new at this, so I apologize beforehand if this is a dumb question.

I'm trying to build a client on python, which mimics the function of this page (relevant code at lines 1192-1260 of source).

When connecting to the endpoint with:

sio.connect('https://rapidbus-socketio-avl.prasarana.com.my', transports=['websocket'])

I get:

Attempting WebSocket connection to wss://rapidbus-socketio-avl.prasarana.com.my/socket.io/?transport=websocket&EIO=4
WebSocket connection accepted with {'sid': 'e7f13de4-10f6-43f6-afc0-75a1336e90ce', 'upgrades': ['websocket'], 'pingInterval': 25000, 'pingTimeout': 60000}
Engine.IO connection established
Sending packet MESSAGE data 0{}
Received packet MESSAGE data 0
Namespace / is connected
Received packet MESSAGE data 4{},"Invalid namespace"
Exception in thread Thread-4 (_handle_eio_message):
Traceback (most recent call last):
  File "C:\Python310\lib\threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "C:\Python310\lib\threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python310\lib\site-packages\socketio\client.py", line 699, in _handle_eio_message
    pkt = self.packet_class(encoded_packet=data)
  File "C:\Python310\lib\site-packages\socketio\packet.py", line 43, in __init__
    self.attachment_count = self.decode(encoded_packet) or 0
  File "C:\Python310\lib\site-packages\socketio\packet.py", line 114, in decode
    self.data = self.json.loads(ep)
  File "C:\Python310\lib\site-packages\engineio\json.py", line 16, in loads
    return original_loads(*args, **kwargs)
  File "C:\Python310\lib\json\__init__.py", line 359, in loads
    return cls(**kw).decode(s)
  File "C:\Python310\lib\json\decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 3 (char 2)

From the webpage I linked above, I can assume that the server sent something that is compressed with gzip and encoded in base64. I guess that is the reason I'm getting the above error.

Despite the error, it connected successfully, and the main function can work without problems with:

@sio.on('onFts-client')
def onfts_client_handler(data):
    print(gzip.decompress(base64.b64decode(data)))
...
sio.emit('onFts-reload',{"sid":sid,"uid":uid,"provider":"RKL","route":no_route})

So my question is, how do I write the function so that the message sent from the server when I connect be decoded and decompressed with base64 and gzip?

And also, when diconnecting with sio.disconnect():

Sending packet MESSAGE data 1
Sending packet CLOSE data None
Engine.IO connection dropped
Exiting write loop task
Unexpected error decoding packet: "string index out of range", aborting
Waiting for write loop task to end
Exiting read loop task

I suspect unexpected error is also because the message is encoded and compressed.

[miguelgrinberg]

Not sure if this is the only problem, but the page you linked uses an old version of Socket.IO that is incompatible with the current one. You may want to try downgrading your Python client to the latest 4.x release. See version compatibility for details. This server you are connecting to is using the Engine.IO protocol version 3. You can see the EIO=3 in the connection URL that it uses. The Python client is sending EIO=4.

[imjuzcy]

You're right! I installed python-socketio==4.6.1 and python-engineio==3.14.2 and the errors went away.

[joevo2]

Hi @imjuzcy I'm trying to work with the MyRapid Bus Kiosk API as well. Have you manage to get it working with your apps? Was searching through GitHub on if anyone had worked on it before. Would love to connect with you and apologise that this might not be the best place to communicate. You can reach me at [email protected]

[arffsaad]

I managed to get it to connect and retrieve data stream from the server. HMU if you still need help

[joevo2]

I think I’ll wait for this instead. Exciting development 💪 https://developer.data.gov.my/realtime-api/gtfs-realtime

but appreciate the offer. Thanks!