Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 25 additions & 8 deletions docs/node-migration-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,21 +134,27 @@ You should see your account and registered backup_cli public key listed, somethi

## Step 3: Generate and Set Encryption Key

For additional security, the backup and restore process encrypts keyshares during transport using AES encryption. You need to generate a shared encryption key and set it as an environment variable for both your old and new nodes, as well as for the backup-cli commands.
For additional security, the backup and restore process encrypts keyshares during transport using AES encryption. You need to generate a shared encryption key (32 bytes / 64 hex characters), configure it on both your old and new nodes, and pass it to the backup-cli commands.

**Important:** The `MPC_BACKUP_ENCRYPTION_KEY_HEX` must be the same between the backup-cli and the node it is currently communicating with (e.g., the old node when running `get-keyshares`, and the new node when running `put-keyshares`).
**How you configure the key on a node depends on the deployment type** (see [Step 5](#step-5-prepare-the-new-node) for the exact placement):
- **TDX / CVM node:** set `backup_encryption_key_hex` under `[mpc_node_config.secrets]` in `user-config.toml`. There is **no** `MPC_BACKUP_ENCRYPTION_KEY_HEX` environment-variable pathway into a CVM — the `.env` used by `deploy-launcher.sh` is not read for this key.
- **Non-TEE node:** set the `MPC_BACKUP_ENCRYPTION_KEY_HEX` environment variable.

**Important:** The backup encryption key must be the same between the backup-cli and the node it is currently communicating with (e.g., the old node when running `get-keyshares`, and the new node when running `put-keyshares`). The node requires exactly 64 hex characters (32 bytes) with no surrounding whitespace, and **fails to start** if the configured value is malformed.
Comment on lines +139 to +143


### Retrieve a key from an existing node.

**Note:** If your node has been running without the `MPC_BACKUP_ENCRYPTION_KEY_HEX` environment variable set, the node automatically generates an encryption key and stores it in a file called `backup_encryption_key.hex` in your `$MPC_HOME_DIR` directory. You can retrieve it with:
**Note:** If your node has been running without an encryption key configured, the node automatically generates one and stores it in a file called `backup_encryption_key.hex` in your home directory. On a **non-TEE** node you can retrieve it with:

```bash
export BACKUP_ENCRYPTION_KEY=$(cat $MPC_HOME_DIR/backup_encryption_key.hex)
```

Copy this key and set it as the `BACKUP_ENCRYPTION_KEY` environment variable for the backup-cli when running `get-keyshares`.

> **⚠️ TDX / CVM source nodes:** you **cannot** retrieve an auto-generated key this way — there is no filesystem access into the CVM. If a CVM node was deployed without `backup_encryption_key_hex` set in its `user-config.toml`, the key it generated internally is unrecoverable and the node **cannot be migrated off**. Always set `backup_encryption_key_hex` explicitly when deploying a CVM node so you know the key. (The shipped `user-config.toml` template makes this field a required, non-hex placeholder so a forgotten value fails loudly at startup rather than silently auto-generating.)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh this seems really dangerous. Shouldn't we fail instead if no key is provided?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that would be better, iirc this was just introduced for backwards compatibility when we introduced the field for the 3.0.0 release since we could not add new env variables at that time.




**Note on key differences:**
Expand Down Expand Up @@ -207,12 +213,23 @@ The encrypted keyshares are now stored in `$BACKUP_HOME_DIR/permanent_keys/epoch
Set up your new node on the new host with the following:

1. **Install and configure the MPC node software** on the new host (the new node should use the same NEAR account as the old node)
2. **Set the encryption key**: on the backup-cli and the new node (note: this can be a different key from the one used in the previous steps, but it's safe to re-use the same key).
2. **Set the encryption key** on the backup-cli and the new node. This must be the same key you pass to the backup-cli's `put-keyshares` in [Step 7](#step-7-transfer-keyshares-to-new-node) (it can differ from the key used to back up the old node, but it's safe — and simplest — to re-use the same key throughout).

For the new node, add this to the `.env` file (replace `<value>` with the actual key from Step 3):
```env
MPC_BACKUP_ENCRYPTION_KEY_HEX=<value>
```
**Where to set it depends on how the new node is deployed:**

- **TDX / CVM node (most operators migrating to TDX):** set it in `user-config.toml` under `[mpc_node_config.secrets]`, then deploy (or **redeploy**) the CVM so the new config takes effect — editing a running CVM's config has no effect until it is redeployed:
```toml
[mpc_node_config.secrets]
backup_encryption_key_hex = "<value>" # exactly 64 hex chars (32 bytes), no whitespace
```
The `.env` file consumed by `deploy-launcher.sh` is **not** read for this key — only `user-config.toml` is. There is no `MPC_BACKUP_ENCRYPTION_KEY_HEX` pathway into a CVM.

> **⚠️ Do not leave `backup_encryption_key_hex` unset or commented on a CVM.** If the field is absent, the node **silently generates a fresh random key inside the CVM** and starts up normally — but with a key that does not match the old node, so the transfer in Step 7 will fail to decrypt with no obvious error. Set it explicitly to the intended value.

- **Non-TEE node:** add it to the `.env` file (replace `<value>` with the actual key from Step 3):
```env
MPC_BACKUP_ENCRYPTION_KEY_HEX=<value>
```


3. **Start the node and retrieve the new keys from the new node**: (P2P (TLS) key, NEAR account key)
Expand Down
2 changes: 2 additions & 0 deletions docs/running-an-mpc-node-in-tdx-external-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -737,6 +737,8 @@ Adjust the variables as per your environment.
* `my_near_account_id` — use the NEAR account ID created in the previous step
* `mpc_contract_id` — **v1.signer-prod.testnet** for testnet, **v1.signer** for mainnet
* `migration_web_ui` — bind address for the migration HTTP endpoint, used by the [Node Migration](./node-migration-guide.md) flow. Required. Keep at `0.0.0.0:8079` to match the port-forward and the `--mpc-node-address …:8079` form the migration guide uses.
* `secrets.secret_store_key_hex` — 16-byte (32 hex char) key encrypting locally-stored secrets. Generate with `openssl rand -hex 16`. Set directly in `user-config.toml`; the node fails to start if it is missing or not valid 32-char hex.
* `secrets.backup_encryption_key_hex` — 32-byte (64 hex char) AES key used to encrypt keyshares during a [node migration](./node-migration-guide.md). Generate with `openssl rand -hex 32`. **Set this explicitly in `user-config.toml` — it is the *only* way to supply this key to a CVM** (there is no `MPC_BACKUP_ENCRYPTION_KEY_HEX` env var or `.env` pathway into the CVM). The node requires exactly 64 hex chars with no whitespace and **fails to start** on a malformed value. **Do not leave it unset:** a CVM with no `backup_encryption_key_hex` silently generates a random key internally that you cannot read back (no filesystem access into the CVM) — leaving that node **impossible to migrate off**, and any future migration *to* this node will fail to decrypt unless this key matches the source node's. When migrating, it must equal the source node's backup encryption key.
* `port_mappings` — port forwarding rules for the MPC container. These should be a subset of the port forwarding for the CVM defined in the [Using the Web Interface](#using-the-web-interface) section.
* `tier3_public_addr` *(under `[mpc_node_config.near_init]`)* — `IP:24567` the node advertises for decentralized (Tier3) state-sync responses. **Required — the template ships this as a `REPLACE_WITH_…` placeholder and the node fails to start if it's left unset or left as the placeholder** (intentionally, so state sync never silently runs with an unreachable advertised address). It is especially critical on any host with more than one external IP, or when running [multiple nodes on one host](./running-multiple-mpc-nodes-on-one-host.md): otherwise the node auto-discovers its advertised address as the host's default-route outbound IP, which peers may not be able to reach, and state sync stalls. Applied at first init only; changing later requires a CVM redeploy via the [Node Migration](./node-migration-guide.md) flow.
* `external_storage_fallback_threshold` *(under `[mpc_node_config.near_init]`)* — **required for node versions before 3.12.0; leave at `100`.** Number of decentralized (peer-to-peer) state-sync attempts per state part before falling back to the external storage bucket. Released 3.11.x nodes default this to `0` (bucket-only), which no longer works now that nearcore has stopped serving the centralized buckets — leaving it unset makes the node crash in a state-sync restart loop (last log line `running state sync shard_id=5`). Ignored on 3.12.0+, which always uses peer-to-peer state sync, so it is safe to keep set across the upgrade. Applied at first init only, like `tier3_public_addr`.
Expand Down
Loading