Skip to content

Initial import from huggingface.js #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 2, 2025
Merged

Initial import from huggingface.js #1

merged 1 commit into from
Jul 2, 2025

Conversation

Wauplin
Copy link
Collaborator

@Wauplin Wauplin commented Jul 2, 2025

PR initially opened in huggingface/huggingface.js#1576 before moving it to its own repository.


Original PR description:

Goal is to run a server that implements OpenAI's Responses API on top of Inference Providers.

cc @Vaibhavs10 @julien-c @hanouticelina

Quick Start

Start server

cd packages/responses-server
pnpm install
pnpm dev

Run examples

# Text input
pnpm run example text

# Multi-turn text input
pnpm run example multi_turn

# Text + image input
pnpm run example image

# Streaming
pnpm run example streaming

# Structured output
pnpm run example structured_output
pnpm run example structured_output_streaming

# Function calling
pnpm run example function
pnpm run example function_streaming

TODOs

This is a POC. Most features are currently not implemented. Check in schemas.ts all fields that exist in official API but are not supported yet.

Here is a list of the main features of the API in no particular order. We don't need all of them right now:

Example

import OpenAI from "openai";

const openai = new OpenAI({ baseURL: "http://localhost:3000/v1", apiKey: process.env.HF_TOKEN });

const response = await openai.responses.create({
	model: "Qwen/Qwen2.5-VL-7B-Instruct",
	instructions: "You are a helpful assistant.",
	input: "Tell me a three sentence bedtime story about a unicorn.",
});

console.log(response);
console.log(response.output_text);
pnpm run example text_single

> @huggingface/responses-server@0.1.0 example /home/wauplin/projects/huggingface.js/packages/responses-server
> node scripts/run-example.js text_single

{
  object: 'response',
  id: 'resp_ff152426c1c1ad544392e8ff512ef73dedc3a9cbab60e65a',
  status: 'completed',
  error: null,
  instructions: 'You are a helpful assistant.',
  model: 'Qwen/Qwen2.5-VL-7B-Instruct',
  temperature: 1,
  top_p: 1,
  created_at: 1751359624,
  output: [
    {
      id: 'msg_0d05372ec992169434028304842f1123b195e9bebdf1305f',
      type: 'message',
      role: 'assistant',
      status: 'completed',
      content: [Array]
    }
  ],
  output_text: "Once upon a time, in a faraway realm, lived a young unicorn named Starlight. Every night, Starlight would Sadly come down from a hill and heatpocumentt the Sleeping Forest. One Moonlight night, Starlight accidentally stumbled upon a secluded glade filled with silver stars and glowing pixies. Dazzled by the magical light, Starlight decided to let the pixies light Starlight. With the pixies' help, Starlight found a way to make the Severnvalsea never gonna gonna gonna stop glittering magnificently. As the last stars disappeared into the night, Starlight felt a sense of peace, knowing that the Gadondo's secret was now safe, forever encompassed in the magic of their icy depiction."
}

@Wauplin Wauplin merged commit e59cc6e into main Jul 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant