Skip to content

Commit 8c38fb3

Browse files
author
1bcMax
committed
docs: add streaming section, update intro and FAQ for v1.6.1
1 parent 9f5fc9e commit 8c38fb3

1 file changed

Lines changed: 82 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 82 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# @blockrun/llm (TypeScript SDK)
22

3-
> **@blockrun/llm** is a TypeScript/Node.js SDK for accessing 40+ large language models (GPT-5, Claude, Gemini, Grok, DeepSeek, Kimi, and more) with automatic pay-per-request USDC micropayments via the x402 protocol. No API keys required — your wallet signature is your authentication. Supports Base and Solana chains.
3+
> **@blockrun/llm** is a TypeScript/Node.js SDK for accessing 41+ large language models (GPT-5, Claude, Gemini, Grok, DeepSeek, Kimi, and more) with automatic pay-per-request USDC micropayments via the x402 protocol. No API keys required — your wallet signature is your authentication. Supports **streaming**, smart routing, Base and Solana chains.
44
55
[![npm](https://img.shields.io/npm/v/@blockrun/llm.svg)](https://www.npmjs.com/package/@blockrun/llm)
66
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
@@ -462,6 +462,74 @@ const result = await client.chatCompletion('openai/gpt-4o', messages);
462462
console.log(result.choices[0].message.content);
463463
```
464464

465+
### Streaming
466+
467+
Stream responses token-by-token with automatic x402 payment. Uses a **pre-auth cache** to skip the 402 round-trip on repeat calls to the same model (~200ms saved per request after the first).
468+
469+
#### OpenAI-compatible (recommended)
470+
471+
```typescript
472+
import { OpenAI } from '@blockrun/llm';
473+
474+
const client = new OpenAI({ walletKey: process.env.BASE_CHAIN_WALLET_KEY });
475+
476+
const stream = await client.chat.completions.create({
477+
model: 'openai/gpt-5.4',
478+
messages: [{ role: 'user', content: 'Write a short story about AI agents' }],
479+
stream: true,
480+
});
481+
482+
for await (const chunk of stream) {
483+
process.stdout.write(chunk.choices[0]?.delta?.content || '');
484+
}
485+
```
486+
487+
#### Native client
488+
489+
```typescript
490+
import { LLMClient, type ChatMessage } from '@blockrun/llm';
491+
492+
const client = new LLMClient();
493+
494+
const messages: ChatMessage[] = [
495+
{ role: 'user', content: 'Explain quantum computing in simple terms' },
496+
];
497+
498+
// Returns a raw fetch Response with SSE body
499+
const response = await client.chatCompletionStream('google/gemini-2.5-flash', messages);
500+
501+
const reader = response.body!.getReader();
502+
const decoder = new TextDecoder();
503+
504+
while (true) {
505+
const { done, value } = await reader.read();
506+
if (done) break;
507+
508+
const chunk = decoder.decode(value, { stream: true });
509+
for (const line of chunk.split('\n')) {
510+
if (!line.startsWith('data: ') || line === 'data: [DONE]') continue;
511+
const data = JSON.parse(line.slice(6));
512+
process.stdout.write(data.choices?.[0]?.delta?.content || '');
513+
}
514+
}
515+
```
516+
517+
#### Payment + streaming flow
518+
519+
```
520+
First call (cache miss):
521+
1. Send request → 402 response (BlockRun returns price)
522+
2. Sign USDC payment locally (key never leaves machine)
523+
3. Retry with PAYMENT-SIGNATURE header + stream: true
524+
4. Cache payment requirements for this model (1h TTL)
525+
5. Stream tokens as they arrive
526+
527+
Subsequent calls (cache hit):
528+
1. Pre-sign payment from cache — skip 402 round-trip
529+
2. Send request with PAYMENT-SIGNATURE upfront
530+
3. Stream tokens immediately (~200ms faster)
531+
```
532+
465533
### List Available Models
466534

467535
```typescript
@@ -724,10 +792,12 @@ Full TypeScript support with exported types:
724792
```typescript
725793
import {
726794
LLMClient,
795+
OpenAI,
727796
testnetClient,
728797
type ChatMessage,
729798
type ChatResponse,
730799
type ChatOptions,
800+
type ChatCompletionOptions,
731801
type Model,
732802
// Smart routing types
733803
type SmartChatOptions,
@@ -738,6 +808,14 @@ import {
738808
APIError,
739809
PaymentError,
740810
} from '@blockrun/llm';
811+
812+
// chatCompletionStream returns a standard fetch Response with SSE body
813+
const streamResponse: Response = await client.chatCompletionStream(model, messages, options);
814+
815+
// OpenAI-compat stream returns AsyncIterable
816+
const stream: AsyncIterable<OpenAIChatCompletionChunk> = await openaiClient.chat.completions.create({
817+
model, messages, stream: true
818+
});
741819
```
742820

743821
## Agent Wallet Setup
@@ -862,6 +940,9 @@ When you make an API call, the SDK automatically handles x402 payment. It signs
862940
### What is smart routing / ClawRouter?
863941
ClawRouter is a built-in smart routing engine that analyzes your request across 14 dimensions and automatically picks the cheapest model capable of handling it. Routing happens locally in under 1ms. It can save up to 78% on LLM costs compared to using premium models for every request.
864942

943+
### Does it support streaming?
944+
Yes — as of v1.6.1. Use `client.chatCompletionStream()` for native streaming or `stream: true` in the OpenAI-compatible client. Payment is handled automatically: the SDK signs USDC payment before streaming begins, and caches payment requirements per model so subsequent calls skip the 402 round-trip (~200ms faster).
945+
865946
### How much does it cost?
866947
Pay only for what you use. Prices start at $0.0002 per request (GPT-5 Nano). There are no minimums, subscriptions, or monthly fees. $5 in USDC gets you thousands of requests.
867948

0 commit comments

Comments
 (0)