The existing google provider targets the consumer Gemini API (generativelanguage.googleapis.com) with API-key auth. Vertex AI is a separate deployment surface — different endpoints, OAuth2/ADC auth, and notably it also hosts third-party models (e.g. Claude via Model Garden).
Before I dig into any of this: is Vertex AI support something you'd find useful for Kilroy? If so, I'm happy to put together an investigation and implementation proposal. I ask because my team uses Vertex AI backing for our company software dev (which is Anthropic based right now). We do this for code source control reasons.
If it does appeal, a couple of early design questions to provide some early "what are impacts" thought exploration:
-
Separate provider or mode? Should this be a new top-level provider key (e.g. vertex_ai) in run.yaml, or a configuration mode on the existing google and anthropic etc providers (e.g. anthropic.api.auth_mode: vertex)?
-
API Shape I believe the Vertex AI offered completion APIs etc are - possibly - different from the standard model-provider API. E.g. I'm not sure that Anthropic's default API is what is exposed by Vertex AI. May be a more "general" LLM API. I feel like I've read such in the past. Needs exploration but this may factor into "where would this be implemented" question.
-
Auth approach? Vertex uses OAuth2 / Application Default Credentials rather than API keys. Any preference on whether Kilroy should shell out to gcloud for tokens, use the Go ADC library directly, or accept a raw access token via env var?
No pressure — just wanted to ask before spending cycles on it. I realize that having open source repos can be a contributor-dumping-overload with Agentic AI support nowadays.
The existing
googleprovider targets the consumer Gemini API (generativelanguage.googleapis.com) with API-key auth. Vertex AI is a separate deployment surface — different endpoints, OAuth2/ADC auth, and notably it also hosts third-party models (e.g. Claude via Model Garden).Before I dig into any of this: is Vertex AI support something you'd find useful for Kilroy? If so, I'm happy to put together an investigation and implementation proposal. I ask because my team uses Vertex AI backing for our company software dev (which is Anthropic based right now). We do this for code source control reasons.
If it does appeal, a couple of early design questions to provide some early "what are impacts" thought exploration:
Separate provider or mode? Should this be a new top-level provider key (e.g.
vertex_ai) inrun.yaml, or a configuration mode on the existinggoogleandanthropicetc providers (e.g.anthropic.api.auth_mode: vertex)?API Shape I believe the Vertex AI offered completion APIs etc are - possibly - different from the standard model-provider API. E.g. I'm not sure that Anthropic's default API is what is exposed by Vertex AI. May be a more "general" LLM API. I feel like I've read such in the past. Needs exploration but this may factor into "where would this be implemented" question.
Auth approach? Vertex uses OAuth2 / Application Default Credentials rather than API keys. Any preference on whether Kilroy should shell out to
gcloudfor tokens, use the Go ADC library directly, or accept a raw access token via env var?No pressure — just wanted to ask before spending cycles on it. I realize that having open source repos can be a contributor-dumping-overload with Agentic AI support nowadays.