Skip to content

feat: add Cloudflare AI API module#191

Open
hderms wants to merge 1 commit into
syumai:mainfrom
hderms:main
Open

feat: add Cloudflare AI API module#191
hderms wants to merge 1 commit into
syumai:mainfrom
hderms:main

Conversation

@hderms

@hderms hderms commented Apr 8, 2026

Copy link
Copy Markdown

What

This works with https://developers.cloudflare.com/workers-ai/ to allow users to run the supported models https://developers.cloudflare.com/workers-ai/models/ expecting to either return a static JSON output or streaming (the models seem to return SSE in some AI-api format but I'm unfamiliar with the specifics)

Motivation

I was using the library and came across the Cloudflare AI functionality and think it would be useful for people trying to write Cloudflare Workers that call AI models. In the spirit of supporting more Cloudflare functionality it seems useful.

Examples

here's using the default method ai.Run which unmarshals the response into a user provided type:

ype WhisperResponse struct {
	Text      string `json:"text"`
	WordCount int    `json:"word_count"`
	Words     []Word `json:"words"`
}

func transcribe(ai ai.AI, audioBuf bytes.Buffer) (string, error) {
	println("Got bytes with size: ", audioBuf.Len())

	var pr WhisperResponse
	//audioBuf.Reset()
	audioBuf.Bytes()

	err := ai.Run("@cf/openai/whisper", WhisperAudioInput{Audio: audioBuf.Bytes()}, &pr)
	fmt.Printf("%+v\n", pr)

	return pr.Text, err
}

here's using a convenience method ai.RunBinary to return binary output from the API

http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
		ttir := &TextToImageRequest{}
		err := json.NewDecoder(req.Body).Decode(ttir)

		if err != nil {
			http.Error(w, err.Error(), http.StatusBadRequest)
			return
		}

		ai, err := ai.NewAI("AI")
		if err != nil {
			panic(err)
		}
		imagePrompt := &PromptRequest{
			Prompt: ttir.Prompt,
		}

		imageBinary, err := ai.RunBytes("@cf/leonardo/lucid-origin", imagePrompt, "image")
		if err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}
		w.WriteHeader(200)
		w.Header().Set("Content-Type", "image/jpg")
		w.Write(imageBinary)
	})

and here's an example of having it produce streaming output:

geminiRequest := GeminiRequest{
		Messages:            createMessages(gAPIRequest.Prompt),
		MaxCompletionTokens: 100,
		Stream:              true,
	}
	stream, errCh, err := s.AI.RunStream("@cf/meta/llama-3.1-8b-instruct-fp8", geminiRequest)
	if err != nil {
		http.Error(w, "AI processing error: "+err.Error(), http.StatusInternalServerError)
		return
	}
	w.Header().Set("Content-Type", "text/event-stream")

	w.WriteHeader(200)

ReadLoop:
	for {
		select {
		case msg, ok := <-stream:
			if ok {
				w.Write([]byte(msg))
			} else {
				break ReadLoop
			}
		case err := <-errCh:
			fmt.Fprintf(w, "error: %s", err.Error())
			break ReadLoop
		}
	}

This works with https://developers.cloudflare.com/workers-ai/ to allow
users to run the supported models https://developers.cloudflare.com/workers-ai/models/
expecting to either return a static JSON output or streaming (the models
seem to return SSE in some AI-api format but I'm unfamiliar with the
specifics)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant