Adding custom/non-Workers AI models to Cloudflare OS through AI Gateway

[dscott98]

This project is really cool. I’d actually started building something along the same lines, so when Cloudflare released Cloudflare OS I decided to build on top of it instead.

One of the first things I wanted was the ability to use models that aren’t running on Workers AI, while still routing everything through Cloudflare AI Gateway.

I put together an overlay that adds that support. It lets an administrator add OpenAI-compatible models. OpenRouter, private endpoints, other providers, etc., through AI Gateway and make them available in Cloudflare OS.

The upstream API credentials stay in AI Gateway, so users can use the configured models without needing their own provider API keys.

This is not a fork of Cloudflare OS. It pins an official upstream commit and applies a small, reproducible patch set on top, so it remains clear what is Cloudflare’s code and what I’ve added.

If anyone else was looking for a way to use custom/non-Workers AI models with Cloudflare OS, the code and setup instructions are here:

https://github.com/dscott98/cloudflare-os-overlay

Would be interested to hear how others are approaching this.