[Feature Request] Web-search / web-browsing

[dzirtusss]

Reference Issues

No response

Summary

First of all, a huge thank you for creating this fantastic, bloat-free app that’s also open source!

Many providers have recently added the ability to use live web search in AI queries, which typically needs client support through functions (some helpful doc links are below). Even if this might be complex to implement, it seems like a really important feature.

Could you share if there are any plans to add this feature? And would you need any assistance with it? I haven’t coded much with Swift, but with 10+ years of coding experience, I’d love to give it a shot and maybe contribute a PR!

https://platform.openai.com/docs/guides/function-calling
https://boltai.com/docs/plugins/web-search
https://boltai.com/docs/plugins/web-browsing

Basic Example

Implement ability to search web and show links

Drawbacks

No response

Unresolved questions

No response

[Renset]

Hey @dzirtusss,
Thank you for your kind words. Indeed, the idea of a web search feature is great! There are a lot of high priority features and changes on the roadmap right now. While I usually have about 2-4 hours per week for macai, the help with this feature will be really appreciated.

But note that implementing new code might be challenging due to the lack of good architecture in macai (still planning to improve it), but even high-level feature design without writing code could be very helpful (e.g. API research, implementation design, etc.).

[dzirtusss]

👍

Checked a bit and basic flow seems simple (but devil will be in details, e.g. 2 queries):

1. you provide the function prop (as json) in the request
2. model decides that it need to call it (or may not), and replies with function call
3. function executes (actually ommited below for simplicity)
4. chat is sent back with:
   4.1. everything in original request
   4.2. copy of model request
   4.3. response from function
5. then model replies having processed function response

Example working script:

#!/bin/sh

OPENAI_API_KEY=$(mypass my.openai-api-key)

curl https://api.openai.com/v1/chat/completions \
  -o response1.json \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "who won election in USA in 2024"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "search_web",
          "description": "useful for when you need to search for information on the web",
          "parameters": {
            "type": "object",
            "properties": {
              "query": {
                "type": "string",
                "description": "the query to search for"
              }
            },
            "required": ["query"]
          }
        }
      }
    ]
  }'
cat response1.json

TOOL_CALL_ID=$(jq -r '.choices[0].message.tool_calls[0].id' response1.json)
echo $TOOL_CALL_ID

MESSAGE=$(jq '.choices[0].message' response1.json)
echo $MESSAGE

QUERY=$(jq '.choices[0].message.tool_calls[0].function.arguments | fromjson | .query' response1.json)
echo $QUERY

REQUEST='{
    "model": "gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "who won election in USA in 2024"
      },
      '"$MESSAGE"',
      {
        "role": "tool",
        "tool_call_id": "'"$TOOL_CALL_ID"'",
        "name": "search_web",
        "content": "Trump won the election in 2024"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "search_web",
          "description": "useful for when you need to search for information on the web",
          "parameters": {
            "type": "object",
            "properties": {
              "query": {
                "type": "string",
                "description": "the query to search for"
              }
            },
            "required": ["query"]
          }
        }
      }
    ]
  }'
echo $REQUEST

curl https://api.openai.com/v1/chat/completions \
  -o response2.json \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d "$REQUEST"
cat response2.json

In this flow however, it will need to have Google SearchAPI key 🙄 (or any other provider, but that's another topic at all) and it is paid:

Custom Search JSON API provides 100 search queries per day for free. If you need more, you may sign up for billing in the API Console. Additional requests cost $5 per 1000 queries, up to 10k queries per day.

MindMac have nice table of other providers here

I'm not sure how others implement this, maybe there are some free providers, need to check. Maybe DuckDuckGo or smb. Anyway this needs to be configurable (even as a wrapper scipt in the begining).

to be continued...

[dzirtusss]

as I have some frankenstine prototype now, moving discussion to the PR #76