-
Notifications
You must be signed in to change notification settings - Fork 3.3k
docs: add model capabilities guide and update provider documentation with tool_use #6901
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Add new model-capabilities.mdx deep dive guide - Update OpenRouter provider documentation - Update Ollama provider documentation - Update reference documentation - Update troubleshooting documentation
- Add model-capabilities.mdx to deep-dives section in docs.json
- Add comprehensive compatibility matrix for popular models - Include OpenAI, Anthropic, Google, Mistral, DeepSeek, and Ollama models - Document feature support for Chat, Apply, Edit, Tool Use, and Image Input - Add GitHub edit link for community contributions
6008d3c
to
2de4a4c
Compare
- Remove migration section reference to non-existent page - Align table headers with left alignment for better readability - Clean up document structure by removing broken cross-reference
9a75f31
to
9c3bcc5
Compare
9c3bcc5
to
5f7b99e
Compare
- Enhanced model capabilities documentation with more detailed explanations - Added model setup information across agent, autocomplete, chat, and edit features - Improved documentation structure and clarity for better user understanding
5f7b99e
to
a23cd28
Compare
Co-authored-by: BekahHW <[email protected]>
Co-authored-by: BekahHW <[email protected]>
Co-authored-by: BekahHW <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 🚀
You can explicitly set the capabilities you want the model to use. This disables autodetection for any other capabilities. | ||
|
||
<Note> | ||
You cannot configure a model to use no capabilities. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to have
no capabilities? Maybe say you cannot override autodetection
|
||
Continue automatically detects capabilities based on your provider and model name. For example: | ||
|
||
- **OpenAI**: GPT-4 models have tool support, GPT-3.5 doesn't |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is interesting, gpt-3.5 turbo models actually support function calling, we may want to change this support but at least this example might confuse users who know that
<Info> | ||
The `tool_use` capability is for native tool/function calling support. The | ||
model must actually support tools for this to work. If your model doesn't | ||
support native tools, Continue will automatically use system message tools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The wording on this is a bit confusing. System message tools are currently experimental and will not be automatically used as a fallback. capabilities only applies to native tools, it does not impact system message tool use whether it is set or not. An exception is openrouter, which we did a hot fix for
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| GPT-4o | ✅ | ✅ | | ||
| GPT-4 Turbo | ✅ | ✅ | | ||
| GPT-4 | ✅ | ❌ | | ||
| GPT-3.5 Turbo | ❌ | ❌ | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See above
|
||
### Notes | ||
|
||
- **Tool Use**: Function calling for Agent mode (required for Agent mode) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The tool_use property is not required for agent mode (tools are), wording is a bit confusing
### Notes | ||
|
||
- **Tool Use**: Function calling for Agent mode (required for Agent mode) | ||
- **Image Input**: Processing screenshots and images |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think "screenshots and images" is confusing because a screenshot is an image and feels oddly specific here
### Recommended Models | ||
|
||
For the best Agent mode experience, we recommend models with strong reasoning and instruction-following capabilities: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I might recommend against using these models with system message tools because they are trained to use native tools and very effective with them. Besides deepseek
docs/troubleshooting.mdx
Outdated
2. Ensure your model actually supports vision (e.g., gpt-4-vision, claude-3) | ||
3. Check that your provider passes through image data | ||
|
||
#### Override capabilities |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe *add capabilities, since there's no override functionality
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
@@ -175,7 +175,11 @@ The `models` section defines the language models used in your configuration. Mod | |||
|
|||
- `roles`: An array specifying the roles this model can fulfill, such as `chat`, `autocomplete`, `embed`, `rerank`, `edit`, `apply`, `summarize`. The default value is `[chat, edit, apply, summarize]`. Note that the `summarize` role is not currently used. | |||
|
|||
- `capabilities`: Array of strings denoting model capabilities, which will overwrite Continue's autodetection based on provider and model. Supported capabilities include `tool_use` and `image_input`. | |||
- `capabilities`: Array of strings denoting model capabilities, which will overwrite Continue's autodetection based on provider and model. Supported capabilities include: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tool_use does not overwrite autodetection, as noted above
Description
[ What changed? Feel free to be brief. ]
Checklist
Screen recording or screenshot
[ When applicable, please include a short screen recording or screenshot - this makes it much easier for us as contributors to review and understand your changes. See this PR as a good example. ]
There is also a compatibility matrix added. I did generate this, but expect to get feedback on actual compatibilities in review as this is not my expertise.
Tests
[ What tests were added or updated to ensure the changes work as expected? ]
Summary by cubic
Added a new guide on model capabilities and updated provider documentation to help users configure tool and image support for custom and proxy models.