Skip to content
Open

update #1373

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion Dockerfile.railway
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ FROM ghcr.io/nofxaios/nofx/nofx-frontend:latest AS frontend
# 最终镜像
FROM alpine:latest

RUN apk add --no-cache ca-certificates tzdata sqlite nginx openssl gettext
RUN apk add --no-cache ca-certificates tzdata sqlite nginx openssl gettext wget

# 复制后端二进制
COPY --from=backend /app/nofx /app/nofx
Expand All @@ -23,6 +23,8 @@ RUN ldconfig /usr/local/lib 2>/dev/null || true
COPY --from=frontend /usr/share/nginx/html /usr/share/nginx/html

WORKDIR /app
# /app/data holds data.db and logs. On Railway, mount a Volume at /app/data to persist
# registration and config across deploys. See docs/railway.md.
RUN mkdir -p /app/data

# 启动脚本(包含 nginx 配置生成)
Expand Down
54 changes: 54 additions & 0 deletions Dockerfile.railway.fromsource
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Railway All-in-One: build backend FROM SOURCE (includes 0-candidate fallbacks)
# Use this when ghcr.io/nofxaios/nofx/nofx-backend does not include fallbacks.
# Set railway.toml: [build] dockerfilePath = "Dockerfile.railway.fromsource"

ARG GO_VERSION=1.25-alpine
ARG ALPINE_VERSION=latest
ARG TA_LIB_VERSION=0.4.0

# ─── TA-Lib ───
FROM alpine:${ALPINE_VERSION} AS ta-lib-builder
ARG TA_LIB_VERSION
RUN apk update && apk add --no-cache wget tar make gcc g++ musl-dev autoconf automake
RUN wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-${TA_LIB_VERSION}-src.tar.gz && \
tar -xzf ta-lib-${TA_LIB_VERSION}-src.tar.gz && cd ta-lib && \
if [ "$(uname -m)" = "aarch64" ]; then \
CONFIG_GUESS=$(find /usr/share -name config.guess 2>/dev/null | head -1); \
CONFIG_SUB=$(find /usr/share -name config.sub 2>/dev/null | head -1); \
[ -n "$CONFIG_GUESS" ] && cp "$CONFIG_GUESS" config.guess; \
[ -n "$CONFIG_SUB" ] && cp "$CONFIG_SUB" config.sub; \
chmod +x config.guess config.sub 2>/dev/null || true; \
fi && \
./configure --prefix=/usr/local && make && make install && \
cd .. && rm -rf ta-lib ta-lib-${TA_LIB_VERSION}-src.tar.gz

# ─── Backend from source ───
FROM golang:${GO_VERSION} AS backend-builder
RUN apk update && apk add --no-cache git make gcc g++ musl-dev
COPY --from=ta-lib-builder /usr/local /usr/local
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=1 GOOS=linux CGO_CFLAGS="-D_LARGEFILE64_SOURCE" \
go build -trimpath -ldflags="-s -w" -o nofx .

# ─── Frontend (pre-built) ───
FROM ghcr.io/nofxaios/nofx/nofx-frontend:latest AS frontend

# ─── Final ───
FROM alpine:${ALPINE_VERSION}
RUN apk add --no-cache ca-certificates tzdata sqlite nginx openssl gettext wget
COPY --from=backend-builder /app/nofx /app/nofx
COPY --from=ta-lib-builder /usr/local/lib/libta_lib* /usr/local/lib/
RUN ldconfig /usr/local/lib 2>/dev/null || true
COPY --from=frontend /usr/share/nginx/html /usr/share/nginx/html
WORKDIR /app
RUN mkdir -p /app/data
COPY railway/start.sh /app/start.sh
RUN chmod +x /app/start.sh
ENV DB_PATH=/app/data/data.db
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:${PORT:-8080}/health || exit 1
CMD ["/app/start.sh"]
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,11 @@ Deploy to Railway with one click - no server setup required:

After deployment, Railway will provide a public URL to access your NOFX instance.

> **Persisting data (registration, strategies, etc.) on Railway**
> By default, data is lost on each new deploy because the container filesystem is ephemeral. To keep your data:
> - **Option A – Volume:** In your Railway service, add a **Volume**, set the **mount path** to `/app/data`, and redeploy. See [docs/railway.md](docs/railway.md).
> - **Option B – PostgreSQL:** Add a Postgres database in Railway and set `DB_TYPE=postgres` plus `DB_HOST`, `DB_USER`, `DB_PASSWORD`, `DB_NAME`, `DB_SSLMODE=require` on your NOFX service. See [docs/railway.md](docs/railway.md).

### Docker Compose (Manual)

```bash
Expand Down
46 changes: 44 additions & 2 deletions api/strategy.go
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,8 @@ func (s *Server) handleStrategyTestRun(c *gin.Context) {
}

var req struct {
Config store.StrategyConfig `json:"config" binding:"required"`
Config *store.StrategyConfig `json:"config"`
StrategyID string `json:"strategy_id"`
PromptVariant string `json:"prompt_variant"`
AIModelID string `json:"ai_model_id"`
RunRealAI bool `json:"run_real_ai"`
Expand All @@ -437,12 +438,33 @@ func (s *Server) handleStrategyTestRun(c *gin.Context) {
return
}

// Prefer loading from DB when strategy_id is provided (avoids client serialization issues with coin_source.static_coins)
if req.StrategyID != "" {
strategy, err := s.store.Strategy().Get(userID, req.StrategyID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "Strategy not found"})
return
}
loaded, err := strategy.ParseConfig()
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "Strategy config invalid: " + err.Error()})
return
}
req.Config = loaded
}
if req.Config == nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "config or strategy_id is required"})
return
}

if req.PromptVariant == "" {
req.PromptVariant = "balanced"
}

logger.Infof("[Strategy Test-Run] coin_source: source_type=%q static_coins=%v", req.Config.CoinSource.SourceType, req.Config.CoinSource.StaticCoins)

// Create strategy engine to build prompt
engine := kernel.NewStrategyEngine(&req.Config)
engine := kernel.NewStrategyEngine(req.Config)

// Get candidate coins
candidates, err := engine.GetCandidateCoins()
Expand All @@ -454,6 +476,10 @@ func (s *Server) handleStrategyTestRun(c *gin.Context) {
})
return
}
if len(candidates) == 0 {
logger.Warnf("[Strategy Test-Run] GetCandidateCoins returned 0; using [BTC] as fallback. Check strategy coin_source (source_type, static_coins).")
candidates = []kernel.CandidateCoin{{Symbol: "BTCUSDT", Sources: []string{"static"}}}
}

// Get timeframe configuration
timeframes := req.Config.Indicators.Klines.SelectedTimeframes
Expand Down Expand Up @@ -537,6 +563,12 @@ func (s *Server) handleStrategyTestRun(c *gin.Context) {
// Build System Prompt
systemPrompt := engine.BuildSystemPrompt(1000.0, req.PromptVariant)

// Defensive: ensure testContext.CandidateCoins is never empty before BuildUserPrompt
if len(testContext.CandidateCoins) == 0 {
logger.Warnf("[Strategy Test-Run] testContext.CandidateCoins is empty before BuildUserPrompt; using [BTCUSDT]. Check strategy coin_source.")
testContext.CandidateCoins = []kernel.CandidateCoin{{Symbol: "BTCUSDT", Sources: []string{"fallback"}}}
}

// Build User Prompt (using real market data)
userPrompt := engine.BuildUserPrompt(testContext)

Expand Down Expand Up @@ -632,11 +664,21 @@ func (s *Server) runRealAITest(userID, modelID, systemPrompt, userPrompt string)
}

// Call AI API
logger.Infof("[Strategy Test-Run] Params to AI server: provider=%s model=%s system_prompt=%d chars user_prompt=%d chars",
provider, model.Name, len(systemPrompt), len(userPrompt))
logger.Infof("[Strategy Test-Run] system_prompt: %s", systemPrompt)
logger.Infof("[Strategy Test-Run] user_prompt: %s", userPrompt)

response, err := aiClient.CallWithMessages(systemPrompt, userPrompt)

if err != nil {
logger.Errorf("[Strategy Test-Run] AI server error: %v", err)
return "", fmt.Errorf("AI API call failed: %w", err)
}

logger.Infof("[Strategy Test-Run] Response from AI server: %d chars", len(response))
logger.Infof("[Strategy Test-Run] response: %s", response)

return response, nil
}

2 changes: 1 addition & 1 deletion auth/auth.go
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ func GenerateJWT(userID, email string) (string, error) {
UserID: userID,
Email: email,
RegisteredClaims: jwt.RegisteredClaims{
ExpiresAt: jwt.NewNumericDate(time.Now().Add(24 * time.Hour)), // Expires in 24 hours
ExpiresAt: jwt.NewNumericDate(time.Now().Add(14 * 24 * time.Hour)), // Expires in 10 days
IssuedAt: jwt.NewNumericDate(time.Now()),
NotBefore: jwt.NewNumericDate(time.Now()),
Issuer: "nofxAI",
Expand Down
5 changes: 5 additions & 0 deletions docs/i18n/zh-CN/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,11 @@ curl -fsSL https://raw.githubusercontent.com/NoFxAiOS/nofx/main/install.sh | bas

部署后,Railway 会提供一个公网 URL 访问你的 NOFX 实例。

> **Railway 数据持久化(用户、策略等)**
> 默认每次重新部署后数据会丢失(容器文件系统是临时的)。若要保留数据:
> - **方式一:Volume**:在 Railway 服务中添加 **Volume**,将 **挂载路径** 设为 `/app/data`,然后重新部署。详见 [docs/railway.md](../../railway.md)。
> - **方式二:PostgreSQL**:在 Railway 添加 Postgres 数据库,并在 NOFX 服务中设置 `DB_TYPE=postgres` 以及 `DB_HOST`、`DB_USER`、`DB_PASSWORD`、`DB_NAME`、`DB_SSLMODE=require`。详见 [docs/railway.md](../../railway.md)。

### Docker Compose (手动)

```bash
Expand Down
159 changes: 159 additions & 0 deletions docs/railway.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
# Deploying NOFX on Railway

This guide covers deploying NOFX to [Railway](https://railway.app) and **persisting your data** (user accounts, strategies, traders, etc.) across deploys.

---

## Local vs Railway: How They Differ

| Aspect | **Local** | **Railway** |
|--------|-----------|-------------|
| **Docker** | Optional. You can: (1) run `./nofx` directly (no Docker), or (2) use `./start.sh` → `docker compose` (Docker). | **Always Docker.** Railway builds and runs the image from `Dockerfile.railway`. |
| **Layout** | With `./start.sh`: **2 containers** (backend `nofx` + frontend `nofx-frontend`). With `./nofx`: **1 process**, no frontend container (you serve the built `web/` yourself or use a separate server). | **1 container (all‑in‑one).** `Dockerfile.railway` puts the Go binary + nginx + built frontend in one image. `railway/start.sh` runs nofx on 8081 and nginx on `PORT`. |
| **Port** | Backend: `API_SERVER_PORT` (default 8080) or `NOFX_BACKEND_PORT` in compose. Frontend: `NOFX_FRONTEND_PORT` (default 3000). | Railway sets **`PORT`** (e.g. 8080). Nginx listens on `PORT`; the Go app runs on **8081** via `API_SERVER_PORT=8081` in `railway/start.sh`. |
| **Database** | SQLite: `DB_PATH` (default `data/data.db`). With Docker Compose, `./data` is mounted at `/app/data` so `data/data.db` or `/app/data/data.db` persists on the host. | SQLite: `DB_PATH=/app/data/data.db` in the image. **Without a Volume, the filesystem is ephemeral** — data is lost on redeploy. With a Volume at `/app/data`, it persists. |
| **Env file** | `.env` from the project root (and `env_file` in docker-compose). `godotenv.Load()` in `main.go` also loads `.env` when running `./nofx`. | No `.env` in the image. Set variables in the **Railway service → Variables**. Railway injects them into the container at runtime. |
| **Encryption keys** | `.env`: `JWT_SECRET`, `DATA_ENCRYPTION_KEY`, `RSA_PRIVATE_KEY`. `./start.sh` can generate them if missing. | Same variable names. If **not set** in Railway, `railway/start.sh` **auto‑generates** them at container start. That means keys change on each new container unless you set them explicitly in Railway Variables. |
| **Persistence** | With Docker: `./data` → `/app/data` persists on the host. With `./nofx`: `data/data.db` in the project directory persists. | **By default: none.** Add a **Volume** at `/app/data` or use **PostgreSQL** so data survives redeploys. |

### Variables That Often Differ

| Variable | Local (typical) | Railway (typical) |
|----------|----------------|-------------------|
| `PORT` | Not used by the Go app. | **Set by Railway.** Used by nginx in `railway/start.sh` to listen. |
| `API_SERVER_PORT` | 8080 (or from `.env`). | **Set to 8081** in `railway/start.sh` (nginx proxies `/api/` to 8081). |
| `DB_PATH` | `data/data.db` (relative to workdir) or `/app/data/data.db` in Docker. | **`/app/data/data.db`** (set in `Dockerfile.railway`). Must be under `/app/data` if you use a Volume. |
| `DB_TYPE` | `sqlite` (default). | `sqlite` with a Volume, or `postgres` if you use Railway Postgres. |
| `JWT_SECRET`, `DATA_ENCRYPTION_KEY`, `RSA_PRIVATE_KEY` | In `.env`; often generated once by `./start.sh`. | **Set in Railway Variables** so they stay fixed across deploys. If unset, `railway/start.sh` generates new ones each start → sessions/decryption can break. |
| `TRANSPORT_ENCRYPTION` | Often `false` for local HTTP. | `false` or `true`; does **not** affect DB or “Candidate Coins (0)” (see main README). |
| `NOFX_BACKEND_PORT`, `NOFX_FRONTEND_PORT` | Used by **docker-compose** and `./start.sh` for host port mapping. | **Not used** on Railway; `PORT` is the single public port. |

### Is Railway Using Docker?

Yes. `railway.toml` points to `Dockerfile.railway`. Railway builds that image and runs it in a container. The main difference from “local Docker” is:

- **Local `docker-compose`**: two services (backend + frontend), `.env` and host volume `./data` for persistence.
- **Railway**: one image with backend + nginx + frontend, no `.env` (use Railway Variables), and **you must add a Volume or Postgres** for persistence.

---

## Why does my data disappear on each deploy?

Railway runs your app in a **container**. Each **new deploy** creates a **new container** with a fresh filesystem. Anything written to the container’s local disk (e.g. `data/data.db`) is **lost** when that container is replaced.

To keep your data, you must store it in **persistent storage**. Railway offers two main options:

1. **Volumes** – persistent disk attached to your service (keeps SQLite `data.db` and logs).
2. **PostgreSQL** – managed database; set `DB_TYPE=postgres` and connection env vars.

---

## Option A: Railway Volume (recommended for SQLite)

A **Volume** is a persistent disk. When you mount it at `/app/data`, the database and logs written there **survive redeploys**.

### Steps

1. **Open your Railway project** → select your NOFX **service**.
2. **Add a Volume**
- **Command Palette**: `⌘K` (Mac) or `Ctrl+K` (Windows) → “Add Volume”, or
- **Right‑click** on the project canvas → “Add Volume”.
3. **Attach the volume to your NOFX service** when prompted.
4. **Set the mount path**
- In the volume or service settings, set **Mount Path** to:
```text
/app/data
```
- This must be exactly `/app/data` because:
- `DB_PATH` is `/app/data/data.db` in the Railway image.
- Logs go to `data/` (i.e. `/app/data/`) when the app runs with `WORKDIR /app`.

5. **Redeploy** (or let the next deploy run). From that point on, `data.db` and logs under `/app/data` will persist across deploys.

### Notes

- The **first** time you add the volume, the filesystem at `/app/data` will be empty. The app will create `data.db` on first run. **You will need to register again** (or restore a backup) if you had data in the previous, non‑persistent container.
- After the volume is attached, **new** registrations and data will persist across future deploys.
- Do **not** set `DB_PATH` to something outside `/app/data` if you want it on the volume. The default `DB_PATH=/app/data/data.db` is correct when the volume is at `/app/data`.

---

## Option B: PostgreSQL

If you prefer a managed database, use **PostgreSQL** and switch NOFX to it. Data is stored in Postgres instead of a local file, so it persists regardless of the container.

### Steps

1. **Add PostgreSQL** in Railway
- “New” → “Database” → “PostgreSQL”, or add the Postgres plugin to your project.

2. **Connect it to your NOFX service**
- In the Postgres service, use “Connect” / “Add to project” so your NOFX service can use it. Railway will set `DATABASE_URL` or `PGHOST`, `PGUSER`, etc.
- If you get `DATABASE_URL`, you can derive the individual vars from it, or set them explicitly.

3. **Set environment variables** on your **NOFX service**:

| Variable | Example / description |
|----------------|-------------------------------------------------|
| `DB_TYPE` | `postgres` |
| `DB_HOST` | from Railway (e.g. `containers-us-west-xxx.railway.app`) |
| `DB_PORT` | `5432` (or the port Railway shows) |
| `DB_USER` | from Railway |
| `DB_PASSWORD` | from Railway |
| `DB_NAME` | `railway` or the DB name Railway creates |
| `DB_SSLMODE` | `require` (Railway Postgres typically uses SSL) |

Use the exact values from your Postgres service’s “Variables” or “Connect” tab.

4. **Remove or do not set** `DB_PATH` when using Postgres (it is only for SQLite).

5. **Deploy**. NOFX will create the tables in Postgres. Your users, strategies, and traders will persist across deploys.

---

## Troubleshooting: “Candidate Coins (0)” even with /app/data

A Volume at `/app/data` only **persists** the database. If the **strategy** stored in the DB has **no static coins** (or uses **ai500/oi_top**, which can fail on Railway), you will still see **0 coins** in the user prompt.

### 1. Set Coin Source to **Static** and add coins

1. Open **Strategy Studio**.
2. Select the **strategy** your trader uses.
3. Open the **Coin Source** (币种来源) section.
4. Set **Source Type** to **Static List**.
5. In **Custom Coins**, add at least **BTC** (e.g. type `BTC` or `BTCUSDT` and add). Optionally add ETH, SOL, DOGE.
6. Click **Save** (保存).

### 2. Make sure the trader uses that strategy

In **Config → Traders**, edit your trader and set **Strategy** to the same strategy you just saved. Save the trader.

### 3. If the strategy is **ai500** or **oi_top**

The **default** strategy uses **ai500**. That calls the NofxOS API; on Railway it can fail (no/ invalid NofxOS API key, or network). When it fails, candidate coins can be 0.

- **Preferred:** Switch the strategy to **Static** and add BTC (and others) as above; then Save.
- **Alternatively:** In the strategy’s **Indicators** section, set a valid **NofxOS API Key** and ensure Railway can reach the NofxOS API.

### 4. (Optional) Deploy a build that includes fallbacks

The image built from `Dockerfile.railway` uses the pre-built `ghcr.io/nofxaios/nofx/nofx-backend` binary, which may **not** include the “[BTC] when 0” fallbacks. To include them, build the backend from source:

1. In `railway.toml`, set:
```toml
[build]
dockerfilePath = "Dockerfile.railway.fromsource"
```
2. Redeploy. The build will be slower, but the running app will use the fallbacks (e.g. [BTC] when the strategy returns 0).

---

## Summary

| Goal | Approach |
|-----------------------------|--------------------------------------------------|
| Keep SQLite and logs | Add a **Volume**, mount path **`/app/data`** |
| Use a managed DB | Add **PostgreSQL**, set **`DB_TYPE=postgres`** and DB_* vars |
| Fix “0 coins” in prompt | Strategy: **Static** + add **BTC** (and others) → **Save**; ensure the **trader** uses that strategy |

After that, your registration and other data will **no longer be deleted** on each deploy.
Loading