How to Expose LLaVA Model via REST API?

[Abhijithm2447]

I have successfully run the following code to serve a model using LLaVA's CLI:

CUDA_VISIBLE_DEVICES=0 python3 -m llava.serve.cli \
  --model-path liuhaotian/LLaVA-Lightning-MPT-7B-preview \
  --image-file "https://llava-vl.github.io/static/images/view.jpg"

Now, I want to expose this model through a REST API so that I can make predictions programmatically. How can I achieve this?

I would appreciate any guidance on setting up a REST API for serving this LLaVA model.

[sreenivasanm6]

Hi Team,

Please let me know how to give prompt with the above python interface? A quick response is appreciated.

[Wims668]

I'm writing to you because I'd like to create a program that can analyze images imported into the same program, so that the latter can draw conclusions.
I'd like to insert LLaVA on Bubble.io on any application creation site.
Is this possible?
If so, how do I go about it?
Could you please help me?
I love LLaVA and this program inspires me enormously.
So thank you!

Have a nice day.

Best regards,

Willie WIMS