Skip to content

Commit 1da1db5

Browse files
Add: JSON HTTP model loader for dynamic configuration from HTTP endpoints
Features: - Fetches model configuration from any HTTP(S) endpoint - Supports polling for auto-reload with hash-based change detection - Graceful degradation for invalid model entries - Configurable timeout and polling interval - Detailed error messages for network and HTTP failures Implementation: - Create JsonHttpModelLoader class with load(), watch(), stopWatching() - Register in ModelLoaderFactory loader registry - Add comprehensive unit tests (35 tests, 100% coverage) - Update MODELLOADER.md documentation Advanced Use Case: - Combine with n8n workflows for dynamic model selection - Support role-based, tenant-based, and resource-aware filtering - Apply custom business logic to determine available models - Unlimited filtering criteria vs tag-based N8nApiModelLoader Configuration: - MODEL_LOADER_TYPE=json-http - JSON_HTTP_ENDPOINT=https://api.example.com/models - JSON_HTTP_POLL_INTERVAL=300 (optional, default 300s) - JSON_HTTP_TIMEOUT=10000 (optional, default 10000ms) All 127 loader tests passing with 98.88% coverage. Code quality: ESLint and Prettier checks passing.
1 parent 9dce046 commit 1da1db5

File tree

8 files changed

+1279
-8
lines changed

8 files changed

+1279
-8
lines changed

CLAUDE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
@AGENTS.md

docs/MODELLOADER.md

Lines changed: 211 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,207 @@ AUTO_DISCOVERY_POLL_INTERVAL=300
9595
- Never commit token to git
9696
- Webhook URLs are public (use `N8N_WEBHOOK_BEARER_TOKEN` for webhook auth)
9797

98+
### JsonHttpModelLoader (Type: `json-http`)
99+
100+
Fetches models from any HTTP(S) endpoint that returns JSON. No authentication required (basic support, future enhancements possible).
101+
102+
**Configuration:**
103+
```bash
104+
MODEL_LOADER_TYPE=json-http
105+
JSON_HTTP_ENDPOINT=https://api.example.com/models
106+
JSON_HTTP_POLL_INTERVAL=300
107+
JSON_HTTP_TIMEOUT=10000
108+
```
109+
110+
**Environment Variables:**
111+
112+
| Variable | Required | Default | Description |
113+
|----------|----------|---------|-------------|
114+
| `JSON_HTTP_ENDPOINT` | Yes | - | HTTP(S) endpoint URL that returns JSON models |
115+
| `JSON_HTTP_POLL_INTERVAL` | No | `300` | Polling interval in seconds (0=disabled, 60-600 range) |
116+
| `JSON_HTTP_TIMEOUT` | No | `10000` | HTTP request timeout in milliseconds (min: 1000) |
117+
118+
**Expected Response Format:**
119+
```json
120+
{
121+
"model-id-1": "https://webhook.example.com/path1",
122+
"model-id-2": "https://webhook.example.com/path2"
123+
}
124+
```
125+
126+
**How It Works:**
127+
1. Fetches JSON from configured HTTP endpoint
128+
2. Validates response is a JSON object (not array or primitive)
129+
3. Validates each model entry (graceful degradation)
130+
4. Optional: Polls for changes at configured interval
131+
5. Hash-based change detection (only fires callbacks on actual changes)
132+
133+
**Use Cases:**
134+
- Centralized model configuration service
135+
- Dynamic model provisioning
136+
- Third-party model registries
137+
- Multi-tenant model discovery
138+
139+
**Polling:**
140+
- Runs at startup, then at configured interval
141+
- Hash comparison: Only fires callbacks when models actually change
142+
- On failure: Logs error, keeps existing models, retries later
143+
- Disabled when `JSON_HTTP_POLL_INTERVAL=0`
144+
145+
**Setup Steps:**
146+
1. Create HTTP endpoint that returns JSON object with model mappings
147+
2. Configure bridge with endpoint URL
148+
3. Set polling interval (optional)
149+
4. Set timeout if endpoint is slow (optional)
150+
5. Restart bridge
151+
152+
**Error Handling:**
153+
- HTTP errors (401, 403, 404, 5xx): Detailed error messages
154+
- Network errors: Clear failure messages with endpoint URL
155+
- Invalid JSON format: Throws during startup
156+
- Invalid model entries: Filtered with warnings (graceful degradation)
157+
158+
**Security Considerations:**
159+
- Endpoint should be HTTPS in production
160+
- Future versions will support authentication (bearer tokens, custom headers)
161+
- Webhook URLs in response should be HTTPS
162+
- Consider using VPN/private network for internal endpoints
163+
164+
**Example Endpoint Response:**
165+
```json
166+
{
167+
"gpt-4": "https://n8n.company.com/webhook/gpt4-chat/chat",
168+
"claude-3": "https://n8n.company.com/webhook/claude/chat",
169+
"local-model": "https://ollama.local:8000/webhook/ollama"
170+
}
171+
```
172+
173+
**Advanced: Using with n8n Workflows for Dynamic Model Selection**
174+
175+
JsonHttpModelLoader can be combined with n8n workflows to implement sophisticated, dynamic model filtering and selection logic. This allows you to:
176+
177+
- **Apply custom business logic** to determine which models are available
178+
- **Filter models by user roles/permissions** - expose different models to different users
179+
- **Dynamically select models** based on tenant, organization, or department
180+
- **Apply additional criteria** like resource availability, licensing, or feature flags
181+
- **Centralize model configuration** while keeping selection logic in n8n
182+
183+
**Architecture:**
184+
185+
```
186+
n8n-openai-bridge
187+
↓ polls every 300s
188+
[n8n Workflow (HTTP Trigger)]
189+
190+
[Filter/Transform Logic]
191+
↓ returns JSON
192+
{ "model-id": "webhook-url", ... }
193+
```
194+
195+
**How It Works:**
196+
197+
1. Configure JsonHttpModelLoader with endpoint pointing to n8n webhook
198+
2. Create n8n workflow with HTTP trigger (webhook node)
199+
3. Workflow receives polling request from bridge
200+
4. Workflow executes custom logic:
201+
- Query model catalog/database
202+
- Apply user/tenant criteria
203+
- Filter by available resources
204+
- Transform data to required format
205+
5. Workflow returns JSON object with model mappings
206+
6. Bridge validates, caches (with hash), and uses for requests
207+
208+
**Example Workflow Setup:**
209+
210+
```
211+
HTTP Trigger (Webhook)
212+
213+
Query Database/API
214+
215+
Filter Models (Apply Criteria)
216+
217+
Transform to Required Format
218+
219+
Return JSON Response
220+
```
221+
222+
**Use Case Examples:**
223+
224+
1. **Role-based Model Availability:**
225+
```json
226+
// For admin users
227+
{
228+
"gpt-4": "https://n8n.../webhook/gpt4",
229+
"gpt-4-turbo": "https://n8n.../webhook/gpt4-turbo",
230+
"claude-3": "https://n8n.../webhook/claude"
231+
}
232+
233+
// For regular users
234+
{
235+
"gpt-3.5-turbo": "https://n8n.../webhook/gpt35"
236+
}
237+
```
238+
239+
2. **Tenant-based Selection:**
240+
- Workflow queries which models customer's subscription includes
241+
- Returns only available models for that tenant
242+
- Prevents unauthorized access to premium models
243+
244+
3. **Resource-aware Selection:**
245+
- Workflow checks available GPU/compute resources
246+
- Only exposes models for available resources
247+
- Prevents overload from unavailable services
248+
249+
4. **Feature Flag Control:**
250+
- Workflow queries feature flags
251+
- Dynamically enables/disables experimental models
252+
- Rollout control without redeploying
253+
254+
**Configuration Example:**
255+
256+
```bash
257+
MODEL_LOADER_TYPE=json-http
258+
JSON_HTTP_ENDPOINT=https://your-n8n.com/webhook/model-selector
259+
JSON_HTTP_POLL_INTERVAL=300 # Recheck available models every 5 minutes
260+
JSON_HTTP_TIMEOUT=10000 # Allow workflow up to 10 seconds to respond
261+
```
262+
263+
**n8n Workflow Development:**
264+
265+
When building your model selection workflow:
266+
267+
1. **Receive request** - HTTP trigger automatically receives polling request
268+
2. **Query your criteria** - Check database, APIs, or configuration
269+
3. **Filter models** - Apply business logic (roles, tenants, resources, etc.)
270+
4. **Transform data** - Build JSON response in required format:
271+
```json
272+
{
273+
"model-id": "https://full-webhook-url-including-domain",
274+
"model-id-2": "https://another-webhook-url"
275+
}
276+
```
277+
5. **Return response** - Respond with 200 and JSON object
278+
279+
**Important Notes:**
280+
281+
- Workflow endpoint must return valid JSON object (not array)
282+
- Each webhook URL must be complete (including domain/protocol)
283+
- Ensure workflow completes within `JSON_HTTP_TIMEOUT` (default 10s)
284+
- Models are cached (hash-based), so frequent polling is efficient
285+
- Workflow errors log warning but keep using previously cached models
286+
- Invalid model entries are filtered with warnings (graceful degradation)
287+
288+
**Advantages Over N8nApiModelLoader:**
289+
290+
| Aspect | N8nApiModelLoader | JsonHttpModelLoader + Workflow |
291+
|--------|-------------------|-------------------------------|
292+
| **Setup Complexity** | Simple (tag-based) | More flexible but requires workflow |
293+
| **Filtering Logic** | Tag-based only | Unlimited custom logic |
294+
| **User-aware** | No | Yes (workflow can access user context) |
295+
| **Tenant Support** | No | Yes (workflow can filter by tenant) |
296+
| **Resource Awareness** | No | Yes (workflow can check resources) |
297+
| **Auth Support** | API key required | Flexible (can use any n8n auth) |
298+
98299
### StaticModelLoader (Type: `static`)
99300

100301
Loads models from environment variable. For testing and development only.
@@ -166,13 +367,14 @@ These are logged as warnings but don't block startup:
166367

167368
## Comparison
168369

169-
| Feature | JsonFileModelLoader | N8nApiModelLoader | StaticModelLoader |
170-
|---------|-------------------|-------------------|-------------------|
171-
| **Type ID** | `file` | `n8n-api` | `static` |
172-
| **Use Case** | Manual configuration | Auto-discovery | Testing only |
173-
| **Startup Speed** | Fast | Depends on API | Fast |
174-
| **Hot-Reload** | File watching | Polling | None |
175-
| **Dependencies** | None | n8n API access | None |
370+
| Feature | JsonFileModelLoader | N8nApiModelLoader | JsonHttpModelLoader | StaticModelLoader |
371+
|---------|-------------------|-------------------|-------------------|-------------------|
372+
| **Type ID** | `file` | `n8n-api` | `json-http` | `static` |
373+
| **Use Case** | Manual configuration | Auto-discovery | Remote config | Testing only |
374+
| **Startup Speed** | Fast | Depends on API | Depends on endpoint | Fast |
375+
| **Hot-Reload** | File watching | Polling | Polling | None |
376+
| **Dependencies** | None | n8n API access | HTTP endpoint | None |
377+
| **Authentication** | N/A | Required (API key) | None (future support) | N/A |
176378

177379
---
178380

@@ -245,7 +447,9 @@ AUTO_DISCOVERY_TAG=openai-model
245447
- **Base Class**: `src/loaders/ModelLoader.js`
246448
- **File Loader**: `src/loaders/JsonFileModelLoader.js`
247449
- **n8n API Loader**: `src/loaders/N8nApiModelLoader.js`
450+
- **JSON HTTP Loader**: `src/loaders/JsonHttpModelLoader.js`
248451
- **Static Loader**: `src/loaders/StaticModelLoader.js`
452+
- **Factory**: `src/factories/ModelLoaderFactory.js`
249453
- **Config Integration**: `src/config.js`
250454
- **Tests**: `tests/loaders/`
251455

src/factories/ModelLoaderFactory.js

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818

1919
const JsonFileModelLoader = require('../loaders/JsonFileModelLoader');
2020
const N8nApiModelLoader = require('../loaders/N8nApiModelLoader');
21+
const JsonHttpModelLoader = require('../loaders/JsonHttpModelLoader');
2122
const StaticModelLoader = require('../loaders/StaticModelLoader');
2223

2324
/**
@@ -38,7 +39,12 @@ class ModelLoaderFactory {
3839
* Registry of available model loaders
3940
* Each loader must have a static TYPE property
4041
*/
41-
static MODEL_LOADERS = [JsonFileModelLoader, N8nApiModelLoader, StaticModelLoader];
42+
static MODEL_LOADERS = [
43+
JsonFileModelLoader,
44+
N8nApiModelLoader,
45+
JsonHttpModelLoader,
46+
StaticModelLoader,
47+
];
4248

4349
/**
4450
* Validate and extract environment variables for a loader

0 commit comments

Comments
 (0)