Skip to content

Commit a189446

Browse files
wlawtroethke
andauthored
feat(v2-snapshots): add docs for v2-storage snapshots (#1726)
* feat(v2-snapshots): add docs for v2-storage snapshots * edits * floww * keep parts of the setup * resumeable and migrate-v2 * link base cli, fix step cross-refs * typo * fix typo --------- Co-authored-by: roethke <jon.roethke@coinbase.com>
1 parent 92b6726 commit a189446

1 file changed

Lines changed: 78 additions & 39 deletions

File tree

Lines changed: 78 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Node Snapshots
33
sidebarTitle: Snapshots
4-
description: Download and restore Base node snapshots to significantly reduce initial sync time for both archive and pruned nodes.
4+
description: Download and restore Base node snapshots to significantly reduce initial sync time for nodes.
55
---
66

77
Using a snapshot significantly reduces the initial time required to sync a Base node. Snapshots are updated regularly.
@@ -12,88 +12,127 @@ If you're a prospective or current Base node operator, you can restore from a sn
1212

1313
These steps assume you are in the cloned `node` directory (the one containing `docker-compose.yml`).
1414

15-
1. **Install aria2c**: Snapshot downloads require `aria2c`, a resumable downloader that handles the periodic connection interruptions imposed by Cloudflare. If you don't have it installed:
15+
<Note>
16+
These steps use the `base-reth-node` CLI to download snapshots. If you don't already have it, follow the [installation instructions](https://github.com/base/base#install-binaries) to install it first.
17+
</Note>
1618

17-
<CodeGroup>
18-
```bash macOS
19-
brew install aria2
20-
```
21-
22-
```bash Ubuntu / Debian
23-
sudo apt-get install aria2
24-
```
25-
</CodeGroup>
26-
27-
2. **Prepare Data Directory**:
19+
1. **Prepare Data Directory**:
2820
- **Before running Docker for the first time**, create the data directory on your host machine that will be mapped into the Docker container. This directory must match the `volumes` mapping in the `docker-compose.yml` file.
2921
```bash
3022
mkdir ./reth-data
3123
```
3224
- If you have previously run the node and have an existing data directory, **stop the node** (`docker compose down`), remove the _contents_ of the existing directory (e.g. `rm -rf ./reth-data/*`), and proceed.
3325

34-
3. **Download Snapshot**: Choose the appropriate snapshot for your network and client from the table below. Download it into the `node` directory.
26+
2. **Choosing the chain**: Use the `--chain` flag to select the network
27+
28+
| Network | `--chain` value |
29+
|---------------|-----------------|
30+
| Base Mainnet | `base` |
31+
| Base Sepolia | `base-sepolia` |
32+
33+
3. **Download Snapshot**: Choose the appropriate snapshot for your network and client from the table below.
3534

36-
| Network | Snapshot Type | Download Command |
37-
| -------- | ------------- | ---------------- |
38-
| Testnet | Archive (recommended)| `aria2c -c -x 16 -s 16 "https://sepolia-reth-archive-snapshots.base.org/$(curl -s https://sepolia-reth-archive-snapshots.base.org/latest)"` |
39-
| Testnet | Pruned | `aria2c -c -x 16 -s 16 "https://sepolia-reth-pruned-snapshots.base.org/$(curl -s https://sepolia-reth-pruned-snapshots.base.org/latest)"` |
40-
| Mainnet | Archive (recommended)| `aria2c -c -x 16 -s 16 "https://mainnet-reth-archive-snapshots.base.org/$(curl -s https://mainnet-reth-archive-snapshots.base.org/latest)"` |
41-
| Mainnet | Pruned | `aria2c -c -x 16 -s 16 "https://mainnet-reth-pruned-snapshots.base.org/$(curl -s https://mainnet-reth-pruned-snapshots.base.org/latest)"` |
35+
| Config | Flag | What you get | Use when |
36+
|--------------|-------------|------------------------------------------------------------------------------|-------------------------------------------------------------|
37+
| **Minimal** | `--minimal` | The smallest set needed to boot: latest state + headers (plus the minimum required history). | You want the fastest, smallest download and don't need historical data. |
38+
| **Full** | `--full` | Full-node data matching the default full-node prune settings (state, headers, and a bounded window of transactions, receipts, and history). | You want a standard full node without keeping the entire archive. |
39+
| **Archive** | `--archive` | Everything available — all transactions, receipts, and account/storage history, with no pruning. | You need complete historical data (e.g. archive queries, indexing). |
4240

4341
<Note>
44-
Ensure you have enough free disk space to download the snapshot archive (`.tar.gz` / `.tar.zst` file) _and_ extract its contents. The extracted data will be significantly larger than the archive.
42+
Ensure you have enough free disk space to download the snapshot _and_ extract its contents. The extracted data will be significantly larger than the archive.
4543
</Note>
4644

47-
4. **Extract Snapshot**: Untar the downloaded snapshot archive. Replace `snapshot-filename` with the actual downloaded filename:
48-
4945
```bash
50-
tar -xzvf <snapshot-filename.tar.gz>
46+
# Minimal node on Base Mainnet
47+
base-reth-node download --minimal --datadir ./reth-data --chain base --resumable
5148

52-
# For .tar.zst
53-
tar -I zstd -xvf <snapshot-filename.tar.zst>
49+
# Full node on Base Sepolia
50+
base-reth-node download --full --datadir ./reth-data --chain base-sepolia --resumable
5451
```
5552

56-
5. **Move Data**: The extraction process will likely create a `reth` directory.
53+
Alternatively, for archival nodes only, you may run `base db migrate-v2`. However, this is expected to take **much** longer than downloading. `--resumable` is also not supported in `migrate-v2`.
5754

58-
* Move the *contents* of that directory into the data directory you created in Step 1:
55+
4. **(Optional) - tuning download concurrency:**
5956

60-
```bash
61-
mv ./reth/* ./reth-data/
62-
rm -rf ./reth # Clean up empty extracted folder
63-
```
57+
The `--download-concurrency` flag controls how many simultaneous HTTP downloads run across the whole
58+
snapshot job. It defaults to `8`, which is a good baseline for most machines.
6459

65-
* The goal is to have the chain data directories (e.g., `chaindata`, `nodes`, `segments`, etc.) directly inside `./reth-data`, not in a nested subfolder.
60+
If you have high-end hardware, you can safely increase it to speed up the download. A good rule of
61+
thumb is **2× the number of physical CPU cores**:
6662

67-
6. **Start the Node**: Now that the snapshot data is in place, return the root of your Base node folder and start the node:
63+
```bash
64+
# Example: a 16 physical-core machine
65+
base-reth-node download --full --datadir ./reth-data --chain base --download-concurrency 32
66+
```
67+
68+
5. <span id="start-the-node" />**Start the Node**: Now that the snapshot data is in place, return the root of your Base node folder and start the node:
6869

6970
```bash
7071
cd ..
7172
docker compose up --build
7273
```
7374

74-
Your node should begin syncing from the last block in the snapshot.
75+
Your node should begin syncing from the last block in the snapshot.
7576

76-
7. **Verify and Clean Up**: Monitor the node logs (`docker compose logs -f <service_name>`) or use the [sync monitoring](/base-chain/node-operators/run-a-base-node#syncing) command to ensure the node starts syncing from the snapshot's block height. Once confirmed, you can safely delete the downloaded snapshot archive (`.tar.gz` file) to free up disk space.
77+
6. **Verify**: Monitor the node logs (`docker compose logs -f <service_name>`) or use the [sync monitoring](/base-chain/node-operators/run-a-base-node#syncing) command to ensure the node starts syncing from the snapshot's block height.
7778

7879
## Proofs Snapshots
7980

81+
<Note>
82+
V2 Proofs Snapshots are coming soon.
83+
</Note>
84+
8085
If you are running the [historical proofs ExEx](/base-chain/node-operators/run-a-base-node#enable-historical-proofs-rpcs), snapshots of the proofs database are available to skip the 24-48 hour backfill.
8186

87+
Proofs snapshots are still distributed as archives, so you'll need `aria2c`, a resumable downloader that handles the periodic connection interruptions imposed by Cloudflare. If you don't have it installed:
88+
89+
<CodeGroup>
90+
```bash macOS
91+
brew install aria2
92+
```
93+
94+
```bash Ubuntu / Debian
95+
sudo apt-get install aria2
96+
```
97+
</CodeGroup>
98+
8299
| Network | Download Command |
83100
| ------- | ---------------- |
84101
| Testnet | `aria2c -c -x 16 -s 16 "https://sepolia-reth-proofs-snapshots.base.org/$(curl -s https://sepolia-reth-proofs-snapshots.base.org/latest)"` |
85102
| Mainnet | `aria2c -c -x 16 -s 16 "https://mainnet-reth-proofs-snapshots.base.org/$(curl -s https://mainnet-reth-proofs-snapshots.base.org/latest)"` |
86103

87-
The restore process is the same as above — follow the [Restoring from Snapshot](#restoring-from-snapshot) steps using this archive instead.
104+
<Note>
105+
Ensure you have enough free disk space to download the snapshot archive (`.tar.gz` / `.tar.zst` file) _and_ extract its contents. The extracted data will be significantly larger than the archive.
106+
</Note>
107+
108+
Once downloaded, extract the archive. Replace `snapshot-filename` with the actual downloaded filename:
109+
110+
```bash
111+
tar -xzvf <snapshot-filename.tar.gz>
112+
113+
# For .tar.zst
114+
tar -I zstd -xvf <snapshot-filename.tar.zst>
115+
```
116+
117+
The extraction process will likely create a `reth` directory. Move the _contents_ of that directory into the data directory you created in [**Prepare Data Directory**](#restoring-from-snapshot) in the section above:
118+
119+
```bash
120+
mv ./reth/* ./reth-data/
121+
rm -rf ./reth # Clean up empty extracted folder
122+
```
123+
124+
The goal is to have the chain data directories (e.g., `chaindata`, `nodes`, `segments`, etc.) directly inside `./reth-data`, not in a nested subfolder. Once confirmed, you can safely delete the downloaded snapshot archive (`.tar.gz` file) to free up disk space.
125+
126+
Then continue from [**Start the Node**](#start-the-node) in the section above.
88127

89128
## FAQ
90129

91130
<AccordionGroup>
92-
<Accordion title="Why does Base provide a Pruned snapshot instead of a Full node snapshot?">
131+
<Accordion title="Why does Base's Full snapshot use a different retention window than Reth's --full preset?">
93132

94133
In Reth, a "full" node is just a pruned node with a specific preset rather than a distinct node type. Reth's `--full` preset retains the last **10,064 blocks** (~1.4 days on Ethereum; ~5-6 hours on Base due to faster block times).
95134

96-
Base's _pruned_ snapshot uses a 31-day rolling retention window instead. If a smaller storage footprint is preferred, you can override `reth.toml` to match the 10,064-block preset.
135+
Base's `--full` snapshot uses a 31-day rolling retention window instead. If a smaller storage footprint is preferred, you can override `reth.toml` to match the 10,064-block preset.
97136

98137
</Accordion>
99138
</AccordionGroup>

0 commit comments

Comments
 (0)