Mojo to Python Messaging

[tedmoore]

I tested a few approaches and found what I think is the most efficient strategy.

I'm still open to changing the names of functions in this (and anything really).

Examples

See ToPythonExample.py, SpectrogramExample.py, and testing_mmm_audio/py_unit_tests/MessengerRoundTripTest.py to get a quick look at the API.

API

From Mojo, a user can use self.messenger.reply_once() to send a value just one time (or infrequently). These replies will go to Python at the end of the current audio block.

Also, a user can use self.messenger.reply_stream() to continuously "stream" values back. These values will be grabbed and sent to Python at the end of every 10th audio block. ("10" is currently hard coded but I think it could be nice to expose the user (in seconds? instead of blocks) as an argument when launching an MMMAudio).

In Python, one can register_callback("listener name", function) where the function receives the value sent from Mojo (the function can be a lambda). Messenger namespaces still apply, so one can send data back from multiple instances of the same struct (e.g. "analysis1.pitch" and "analysis2.pitch").

In BufferedProcess and FFTProcess one can put reply_stream calls in the send_streams function (similar to the get_messages function). reply_once can be used normally.

Testing

This has been tested and is working well. I wouldn't say it's been "stress" tested. The spectrogram example is a bit intense, sending 513 floats back as a numpy array, however I haven't tried it with, say, 100 different callback functions. This is because I'm not sure that's the best pattern of usage for this. Because Mojo lets one write any kinda code you want, it's best to figure out how to make most things happen in Mojo. Sending data back to Python should happen sparingly.

Concerns

The callback thread in MMMAudio.py has a lag of up to 100 ms which is much to long. I think this needs to come down. The CPU might increase, so there's more testing to be done (which is why this is a draft right now).

[spluta]

My long response to this was deleted, so this is less verbose.

This is great.

I wonder if we could simplify the whole thing, though. I just think it is way to heavy now. There are too many connections and I imagine this is going to slow things down.

Here is a fifo communication between python and mojo:
fifo.zip

We could use this kind of system messaging system (maybe tcp via netcat?) to send all the messages as text through one pipe, like:

f,set_freq,440.0
fs,filter_freqs,550.8,347.98
t,trig

There would need to be some parsing, but this is trivial.

what fifo or nc also give us is the abiltity to send messages to floating MMMAudio instances that are not attached to Python.

[spluta]

btw - i can't get unix socket or tcp working yet. maybe it doesn't.

[tedmoore]

what fifo or nc also give us is the abiltity to send messages to floating MMMAudio instances that are not attached to Python.

This would be a nice reason to do fifo, but I'm skeptical it would be faster. Bouncing off disk always scares me, plus I think all that string manipulation (converting types to a string and back again) is not trivial. I agree the implementation here is heavy, but it's heavy in data structures to make it as light as possible in computation. It's the same pattern as the Python to Mojo messaging system. I think this is quite efficient. It costs about 1% CPU. Having any MMMAudio running is about 3 to 4%, I think there's still plenty of CPU overhead for users to use, especially now that spinning up another audio process is trivial. Testing it will be the best way to know.

The performance that I'm concerned about is the Python callback dispatch. What I would really like is to have each callback be able to launch it's own little thread so that whatever a user puts in there can happen without clogging up other parts of the program.

All that being said, the thing I actually care most about is the API. I want to make sure it feels intuitive and obvious how it "should" work.

[spluta]

What I am proposing would not change the API.

I do think we should try out a version of this that is minimal - just floats and strings - to see what the difference is. I know you disagree with that.

From what I understand fifo doesn't actually go to disk. TCP certainly wouldn't.

[tedmoore]

I learned that the FIFO "file" doesn't actually store the data being pass. I always thought it did. So FIFO doesn't go to disk.

Testing FIFO and TCP would be super.

I do think we should try out a version of this that is minimal - just floats and strings - to see what the difference is.

We should test it. I can't see how that would save CPU. It would mean checking a few less Dicts on the Mojo side. Otherwise the CPU is going to be impacted by how many messages there are, not really what the data type is. I think checking a few extra dicts (on the side with the fast language) is a minimal (probably essentially zero) cost for having a more full API. And the "breadth" of the API (allowing different data types) is transparent because the user doesn't even have to specify the data type on either side of the system.

I wonder if the CPU cost here (which again is like 1%) is mostly on the Python side and/or the cost of passing a Python.dict() from mojo ~30+ times per second. I wonder if the Mojo developers weren't thinking this would happen that frequently and therefore its efficiency is not a priority.

[spluta]

This is really slick.

If you are worried about processes blocking, you can always add the process to the scheduler:

sched = Scheduler()

def pitchy(args):
    async def count(args):
        print(f"pitch: {args}")
        await asyncio.sleep(0.1)
        print(f"pitch: {args}")
        await asyncio.sleep(0.1)
        print(f"pitch: {args}")
    sched.sched(count(args))

m.register_callback("shiver.pitch", lambda args: pitchy(args))

fifo or tcp would mostly be to simplify everything and make the MMMAudio Mojo instance autonomous.

[tedmoore]

The whole idea behind reply_stream and reply_once was in order to throttle the "streams" to reduce cpu, but still allow a "reply once" to happen immediately (not have to wait for the next time audio block % 10 == 0). If fifo or tcp are significantly less cpu, perhaps both stream and once won't be needed!

It might also be nice to have an impulse.next_bool trigger when the messages get sent, that way the user can throttle it themself. i didn't implement this because since this implementation has to wait for the end of the audio block anyway, it doesn't really make sense to "trigger" a reply (triggering 2 in an audio block would arrive back to Python as 1, later than either).