Added support for local GPT4all API Server to decai

[kxynos]

Checklist

• Closing issue: Support GPT4all local API Server #198
• Mark this if you consider it ready to merge
• I've added tests (optional)
• I wrote some documentation

Description
Setup/how to:

• I am using a Mac to host GPT4all. So I am using SSH to expose the localhost.

ssh -R 4891:127.0.0.1:4891 -N VM_IP

• Start GPT4all and in Settings mark Enable local API Server (this is listening on localhost (127.0.0.1) on port 4891.

Using the PasswordCheck example in your VM start radare2:

$ r2 PasswordCheck

[0x100001b34]> decai -e lang=swift
[0x100001b34]> decai -e baseurl=http://localhost:4891
[0x100001b34]> decai -e api=gpt4all
[0x100001b34]> decai -d

[trufae]

Isnt gpt4all just following the openapi endpoints? Because i think theres no need to have a specific provider for it

[kxynos]

I am with you. I wanted to avoid creating another option. GPT4all provides a OpenAI-compatible API which is similar to VLLM (r2aiVLLM) interface. Do you want me to enhance that one with options to make it work ? Because the GPT4all API won't work with the current setup. More options are needed.

[trufae]

I had pending in my head to review all the endpoint providers to reduce the duplicated logic because the endpoints are quite standard nowadays and the only that changes is the baseurl which may be tied to the provider. It will be good to make this happen and simplify because imho its easy to have baseurl than host/port. Also better to have api=claude when it means provider=anthropic;baseurl=…;api=openapi, etc

so yeah im all to make this redesign happen and if you want to take it go for it! I dont have time to write and test, but in open to discuss and review if you are in the mood :)

thanks for taking care of it.

also, this approach comes as an inheritance from r2ai-py which also is reflected in r2ai-c, but decai is defintively better for drafting and making changes and when the final design is made we can then update r2ai-c in sync (the python implementation is abandoned)

[trufae]

@kxynos ping?

[kxynos]

I will see what is possible. I have started refactoring the code.

[trufae]

did you started the refactor @kxynos ?