Skip to content

Commit a4698a7

Browse files
authored
Merge pull request #131 from netboxlabs/diode-updates
docs: update Diode documentation structure and remove broken links
2 parents 27a3576 + 683dca0 commit a4698a7

File tree

3 files changed

+275
-23
lines changed

3 files changed

+275
-23
lines changed
Lines changed: 243 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,246 @@
1-
# Get started
1+
# Get Started with Diode
22

3-
## Components
3+
This guide will help you set up and start using Diode to ingest data into NetBox.
44

5-
Diode runs as a sidecar service to NetBox and can run anywhere with network connectivity to NetBox. To get started, the following components need to be installed and running:
5+
## Prerequisites
66

7-
1. [Installing the Diode plugin](diode-plugin.md) - installed on the NetBox host
8-
2. [Running the Diode server](diode-server.md) - installed anywhere where Docker is available
9-
3. [Installing the Diode client](diode-client.md) - installed on the client where data will be ingested
7+
Before you begin, ensure you have:
8+
9+
- NetBox version 4.2.3 or later
10+
- Docker version 27.0.3 or newer
11+
- bash 4.x or newer
12+
- jq
13+
- Network connectivity between your NetBox server and the Diode server
14+
- Sufficient permissions to run Docker commands
15+
16+
## Installation Steps
17+
18+
### Deploy Diode server
19+
20+
> **Host**: These steps should be performed on the host where you want to run the Diode server.
21+
22+
> **Note**: For the complete installation instructions, please refer to the [official Diode Server documentation](https://github.com/netboxlabs/diode/tree/develop/diode-server).
23+
24+
We provide a `quickstart.sh` script to automate the setup process. The script will download and configure all necessary files:
25+
26+
- `docker-compose.yaml` — Defines Diode server containers
27+
- `.env` — Environment settings for customization
28+
- `nginx.conf` — Nginx configuration for routing Diode endpoints
29+
- `client-credentials.json` — Defines OAuth2 clients for secure communication
30+
31+
1. Create a working directory:
32+
```bash
33+
mkdir -p /opt/diode
34+
cd /opt/diode
35+
```
36+
37+
2. Download and prepare the quickstart script:
38+
```bash
39+
curl -sSfLo quickstart.sh https://raw.githubusercontent.com/netboxlabs/diode/release/diode-server/docker/scripts/quickstart.sh
40+
chmod +x quickstart.sh
41+
```
42+
43+
3. Run the script with your NetBox server address:
44+
```bash
45+
./quickstart.sh https://<netbox-server>
46+
```
47+
> **Note**: Replace `<netbox-server>` with your actual NetBox server address. Do not include a trailing slash.
48+
> Example: `./quickstart.sh https://netbox.example.com`
49+
50+
This should have created an `.env` file for your environment.
51+
52+
4. Start the Diode server:
53+
```bash
54+
docker compose up -d
55+
```
56+
57+
5. Verify the Diode server is running:
58+
```bash
59+
docker compose ps
60+
```
61+
All services should show as "running" or "healthy".
62+
63+
6. Extract the `netbox-to-diode` client secret. This will be needed for the Diode NetBox plugin installation:
64+
```bash
65+
echo $(jq -r '.[] | select(.client_id == "netbox-to-diode") | .client_secret' /opt/diode/oauth2/client/client-credentials.json)
66+
```
67+
> **Note**: This will return a credential that will be used by the Diode NetBox plugin to connect to the Diode server. Store it safely.
68+
69+
### Install Diode NetBox Plugin
70+
71+
> **Host**: These steps should be performed on the host where NetBox is installed.
72+
73+
> **Note**: For the complete installation instructions, please refer to the [official Diode NetBox Plugin documentation](https://github.com/netboxlabs/diode-netbox-plugin/blob/develop/README.md).
74+
75+
1. **Source the NetBox Python Virtual Environment**
76+
```bash
77+
cd /opt/netbox
78+
source venv/bin/activate
79+
```
80+
81+
2. **Install the Plugin Package**
82+
```bash
83+
pip install netboxlabs-diode-netbox-plugin
84+
```
85+
86+
3. **Configure NetBox Settings**
87+
Add the following to your `configuration.py`:
88+
```python
89+
PLUGINS = [
90+
"netbox_diode_plugin",
91+
]
92+
93+
PLUGINS_CONFIG = {
94+
"netbox_diode_plugin": {
95+
# Diode gRPC target for communication with Diode server
96+
"diode_target_override": "grpc://<diode-server:port>/diode",
97+
# NetBox username associated with changes applied via plugin
98+
"diode_username": "diode",
99+
# netbox-to-diode client secret from earlier step
100+
"netbox_to_diode_client_secret": "<netbox-to-diode-secret>"
101+
},
102+
}
103+
```
104+
> **Note**: Replace `<diode-server:port>` with your Diode server address and port (default: 8080)
105+
> Example: `grpc://diode.example.com:8080/diode`
106+
107+
4. **Apply Database Migrations**
108+
```bash
109+
cd /opt/netbox/netbox
110+
./manage.py migrate netbox_diode_plugin
111+
```
112+
113+
5. **Restart NetBox Services**
114+
```bash
115+
sudo systemctl restart netbox netbox-rq
116+
```
117+
118+
6. **Generate Diode Client Credentials**
119+
> **Note**: These credentials will be used by the Orb agent to send discovery results to NetBox via Diode.
120+
121+
1. Go to your NetBox instance (https://<netbox-server>)
122+
2. In the left-hand pane, navigate to **Diode -> Client Credentials**
123+
3. Click on **+ Add a Credential**
124+
4. For Client Name, enter any name and click **Create**
125+
5. **IMPORTANT**: Copy the _Client ID_ and _Client Secret_ and save them securely
126+
6. Click **Return to List**
127+
128+
You have now created your Diode client credentials. These will be used as environment variables when running the Orb agent.
129+
130+
### Ingest Data with Orb Agent
131+
132+
> **Host**: These steps should be performed on the host where you want to run the Orb agent for network discovery.
133+
134+
> **Note**: For the complete installation instructions, please refer to the [official Orb Agent documentation](https://github.com/netboxlabs/orb-agent).
135+
136+
1. **Export Client Credentials**
137+
```bash
138+
# Export the client credentials you generated in NetBox
139+
export DIODE_CLIENT_ID="<your-client-id>"
140+
export DIODE_CLIENT_SECRET="<your-client-secret>"
141+
```
142+
143+
2. **Create Agent Configuration File**
144+
Create an `agent.yaml` file with the following content:
145+
```yaml
146+
orb:
147+
config_manager:
148+
active: local
149+
backends:
150+
network_discovery: # Enable network discovery backend
151+
common:
152+
diode:
153+
target: grpc://<diode-server:port>/diode
154+
client_id: ${DIODE_CLIENT_ID}
155+
client_secret: ${DIODE_CLIENT_SECRET}
156+
agent_name: my_agent
157+
policies:
158+
network_discovery:
159+
loopback_policy:
160+
config:
161+
scope:
162+
targets:
163+
- 127.0.0.1
164+
```
165+
> **Note**: Replace `<diode-server:port>` with your Diode server address and port (default: 8080)
166+
> Example: `grpc://diode.example.com:8080/diode`
167+
168+
3. **Run the Agent**
169+
170+
Using host network mode (recommended):
171+
```bash
172+
docker run --net=host \
173+
-v $(pwd):/opt/orb/ \
174+
-e DIODE_CLIENT_ID \
175+
-e DIODE_CLIENT_SECRET \
176+
netboxlabs/orb-agent:latest run -c /opt/orb/agent.yaml
177+
```
178+
179+
Alternative using root user:
180+
```bash
181+
docker run -u root \
182+
-v $(pwd):/opt/orb/ \
183+
-e DIODE_CLIENT_ID \
184+
-e DIODE_CLIENT_SECRET \
185+
netboxlabs/orb-agent:latest run -c /opt/orb/agent.yaml
186+
```
187+
188+
> **Note**: The container needs sufficient permissions to send ICMP and TCP packets. This can be achieved either by:
189+
> - Setting the network mode to `host` (recommended)
190+
> - Running the container as root user
191+
192+
4. **Verify Agent Operation**
193+
- Check the agent logs for successful startup
194+
- Verify data appears in NetBox
195+
196+
## Troubleshooting
197+
198+
### Common Issues
199+
200+
1. **Connection Issues**
201+
- Verify network connectivity between Diode and NetBox:
202+
```bash
203+
# From Diode server
204+
curl -v https://<netbox-server>
205+
# From NetBox server
206+
curl -v grpc://<diode-server:port>/diode
207+
```
208+
- Check firewall rules:
209+
```bash
210+
# Check if required ports are open
211+
netstat -tulpn | grep -E '8080|443'
212+
```
213+
- Validate URLs and ports in configuration files:
214+
- Diode server `.env`
215+
- NetBox `configuration.py`
216+
- Orb agent `agent.yaml`
217+
218+
2. **Docker Issues**
219+
- Check Docker service status:
220+
```bash
221+
systemctl status docker
222+
```
223+
- Verify Docker container logs:
224+
```bash
225+
docker compose logs
226+
```
227+
228+
3. **Permission Issues**
229+
- Ensure proper file permissions:
230+
```bash
231+
ls -la /opt/diode/
232+
ls -la /opt/netbox/
233+
```
234+
- Check Docker socket permissions:
235+
```bash
236+
ls -l /var/run/docker.sock
237+
```
238+
239+
### Getting Help
240+
241+
If you encounter issues:
242+
243+
1. Search GitHub: [Issues](https://github.com/netboxlabs/diode/issues)
244+
2. Find us in Slack: [NetDev Community #orb](https://netdev-community.slack.com/)
245+
3. Check the logs:
246+
- Diode server logs: `docker compose logs`

docs/netbox-extensions/diode/index.md

Lines changed: 30 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -6,22 +6,41 @@
66

77
## Overview
88

9-
Diode is a NetBox data ingestion service that aims to simplify and enhance the process to add and update network data in NetBox. The guiding principle behind Diode is to make it as easy as possible to get data into NetBox by removing as much burden as possible from the user while shifting that effort to technology.
9+
Diode is a NetBox data ingestion service that aims to simplify and enhance the process to add and update network data in NetBox. The guiding principle behind Diode has been to make it as easy as possible to get data into NetBox, removing as much burden as possible from the user while shifting that effort to technology.
1010

11-
Diode is a sidecar service to NetBox that provides a gRPC/protobuf API designed for ingestion of common NetBox data models. Diode reduces the need to preprocess data to make it conform to the strict object hierarchy imposed by the NetBox data model. This allows data to be sent to NetBox in a more freeform manner, in blocks that are intuitive for network engineers such as by device or by interface. Related information is treated as attributes or properties of these blocks. Diode takes care of the heavy lifting of transforming this data to make it align with NetBoxs structured and comprehensive data model. Diode will create placeholder objects to compensate for missing information, allowing fragmented or incomplete information about the network to be collected.
11+
Diode is a sidecar service to NetBox that provides a gRPC/protobuf API designed for ingestion of common NetBox data models. Diode reduces the need to preprocess data to make it conform to the strict object hierarchy imposed by the NetBox data model. This allows data to be sent to NetBox in a more freeform manner, in blocks that are intuitive for network engineers such as by device or by interface. Related information is treated as attributes or properties of these blocks. Diode takes care of the heavy lifting of transforming this data to make it align with NetBox's structured and comprehensive data model. Diode will create placeholder objects to compensate for missing information, allowing fragmented or incomplete information about the network to be collected.
1212

13-
## Supported versions of NetBox
13+
## Prerequisites
1414

15-
Diode has been tested with NetBox versions 3.7.2 and above.
15+
* NetBox 4.2.3 or later
16+
* Python 3.8 or later (for Python SDK)
17+
* Go 1.18 or later (for Go SDK)
18+
* Network connectivity between Diode and NetBox
1619

17-
## Diode components
20+
## Components
1821

19-
There are three required components to the Diode service:
22+
There are several components that make up the Diode ecosystem:
2023

21-
1. **Diode NetBox plugin** - This component provides API key management and ORM integration into NetBox for the Diode server. See how to [install the Diode plugin](https://github.com/netboxlabs/diode-netbox-plugin).
22-
2. **Diode server** - This component provides the ingestion and reconciliation services that process the incoming data. See how to [run the Diode server](https://github.com/netboxlabs/diode/tree/develop/diode-server#readme).
23-
3. **Diode client** - This component receives the data to be ingested and forwards that data as gRPC/protobuf to the Diode server. It is implemented as an SDK and can be embedded in scripts and integrations. See how to [install the Diode Python SDK](https://github.com/netboxlabs/diode-sdk-python).
24+
1. **Diode Server** - The core service that provides ingestion and reconciliation services. See [deployment instructions](https://github.com/netboxlabs/diode/tree/develop/diode-server#readme).
2425

25-
## Additional resources
26+
2. **Diode NetBox Plugin** - Required component that provides API key management and ORM integration into NetBox. See [installation instructions](https://github.com/netboxlabs/diode-netbox-plugin).
2627

27-
Additional resources including example scripts can be found in the NetBox Labs [NetBox Learning repository](https://github.com/netboxlabs/netbox-learning/tree/develop/diode).
28+
3. **Data Ingestion Methods**:
29+
* **NetBox Discovery Agent** - Automated network discovery using the Orb agent
30+
* **SDK Integrations**:
31+
* [Python SDK](https://github.com/netboxlabs/diode-sdk-python)
32+
* [Go SDK](https://github.com/netboxlabs/diode-sdk-go)
33+
34+
## Quick Start
35+
36+
For a quick step-by-step guide, see our [Getting Started Guide](diode-get-started.md).
37+
38+
## Additional Resources
39+
40+
* [Diode Protocol Documentation](https://github.com/netboxlabs/diode/tree/develop/diode-proto)
41+
* Example scripts and tutorials can be found in the NetBox Labs [NetBox Learning repository](https://github.com/netboxlabs/netbox-learning/tree/develop/diode)
42+
43+
## Support
44+
45+
* [GitHub Issues](https://github.com/netboxlabs/diode/issues)
46+
* Slack NetDev Community (#orb channel)

mkdocs.yml

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -176,12 +176,8 @@ nav:
176176
- Changelog: "netbox-extensions/changes/changelog.md"
177177
- Diode:
178178
- Overview: "netbox-extensions/diode/index.md"
179-
- Get started:
180-
- Components: "netbox-extensions/diode/diode-get-started.md"
181-
- Installing the Diode plugin: "netbox-extensions/diode/diode-plugin.md"
182-
- Running the Diode server: "netbox-extensions/diode/diode-server.md"
183-
- Installing the Diode client SDK: "netbox-extensions/diode/diode-client.md"
184-
- Product and Feature Lifecycle: "product_feature_lifecycle.md"
179+
- Get Started: "netbox-extensions/diode/diode-get-started.md"
180+
- Product and Feature Lifecycle: "product_feature_lifecycle.md"
185181
extra:
186182
version:
187183
provider: mike

0 commit comments

Comments
 (0)