Add a recipe that works with function calling models

[lstocchi]

In podman desktop ai lab we are planning to add function calling models support -> containers/podman-desktop-extension-ai-lab#1116

We would need a recipe + a list of recommended models to work with

[axel7083]

Depends on #601

[rhatdan]

#601 merged, can we close this?

[lstocchi]

@rhatdan no. This issue is related to the need to have something similar to the chatbot/rag recipe. So we can add it to ai-lab extension.
The user downloads the recipe, play with it, check the code how to rewrite a similar app ....

[axel7083]

To create a recipe providing function calling, we have the following requirements:

• Ported back new grammar changes from C++ to Python implementation abetlen/llama-cpp-python#1637
• Integrate Functionary v2.5 + Refactor Functionary Code abetlen/llama-cpp-python#1509

I opened abetlen/llama-cpp-python#1636 which should be fixed abetlen/llama-cpp-python#1637

[axel7083]

I made some experiments locally with the amazing Instructor library and here are some demos I made using llama-cpp-python 0.83 (any version above crashes)

Instructor library

Information extraction

Given an unstructured text, return a python model.

demo-instructor-1.mp4

A basic pydantic model provided, the instructor will translate the llm response to the model provided, with validation etc.

Single text classification

demo-instructor-single-classification.mp4

Given an enum and a text, got the enum value as result. Allowing basic single text classification.

Multi text classification

demo-multi-classification.mp4

We can request multiple values as result, allowing to get a multi-classification.

Function calling

The function calling need a bit of manual change, as we need abetlen/llama-cpp-python#1509 applied, and some adjustment to be able to uses the functionary-v2.5¹ model.

demo-function-calling.mp4

Thoses example are took from the llama-cpp-python repository²

Footnotes

1. https://huggingface.co/meetkai/functionary-small-v2.5-GGUF/tree/main ↩

2. https://github.com/abetlen/llama-cpp-python/blob/main/examples/notebooks/Functions.ipynb ↩