Skip to content

Commit 48b5205

Browse files
thomasjpfanthomasjpfan
committed
Add docs
1 parent 7ac9ad1 commit 48b5205

File tree

5 files changed

+199
-0
lines changed

5 files changed

+199
-0
lines changed

‎docs/img/flexiable-labeling-targets-page.png‎

125 KB
Loading

‎docs/tutorial-flexible-labeling.md‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Flexible Labeling with Docker Flow Monitor
2+
3+
*Docker Flow Monitor* and *Docker Flow Swarm Listener* can be configured to allow for more flexible labeling of exporters. Please read the [Running Docker Flow Monitor](tutorial.md) tutorial before reading this one. This tutorial focuses on configuring the stacks to allow for flexible labeling.
4+
5+
## Setting Up A Cluster
6+
7+
!!! info
8+
Feel free to skip this section if you already have a Swarm cluster that can be used for this tutorial
9+
10+
We'll create a Swarm cluster consisting of three nodes created with Docker Machine.
11+
12+
```bash
13+
git clone https://github.com/vfarcic/docker-flow-monitor.git
14+
15+
cd docker-flow-monitor
16+
17+
./scripts/dm-swarm.sh
18+
19+
eval $(docker-machine env swarm-1)
20+
```
21+
22+
## Deploying Docker Flow Monitor
23+
24+
We will deploy [stacks/docker-flow-monitor-flexible-labels.yml](https://github.com/vfarcic/docker-flow-monitor/blob/master/stacks/docker-flow-monitor-flexible-labels.yml) stack that contains three services: `monitor`, `alert-manager` and `swarm-listener`. The `swarm-listener` service includes an additional environment variable: `DF_INCLUDE_NODE_IP_INFO=true`. This configures `swarm-listener` to send node and ip information to `mointor`.
25+
26+
The `monitor` service includes the environment variable: `DF_SCRAPE_TARGET_LABELS=env,metricType`. This sets up flexible labeling for exporters. If an exporter defines a deploy label `com.df.env` or `com.df.metricType`, that label will be used by `monitor`.
27+
28+
Let's deploy the `monitor` stack:
29+
30+
```bash
31+
docker network create -d overlay monitor
32+
33+
docker stack deploy \
34+
-c stacks/docker-flow-monitor-flexible-labels.yml \
35+
monitor
36+
```
37+
38+
## Collecting Metrics and Defining Alerts
39+
40+
We will deploy exporters stack defined in [stacks/exporters-tutorial-flexible-labels.yml](https://github.com/vfarcic/docker-flow-monitor/blob/master/stacks/exporters-tutorial-flexible-labels.yml), two containing two services: `cadvisor` and `node-exporter`.
41+
42+
The definition of the `cadvisor` service contains additional deploy labels:
43+
44+
```yaml
45+
cadvisor:
46+
image: google/cadvisor
47+
networks:
48+
- monitor
49+
...
50+
deploy:
51+
mode: global
52+
labels:
53+
...
54+
- com.df.scrapeNetwork=monitor
55+
- com.df.env=prod
56+
- com.df.metricType=system
57+
```
58+
59+
The `com.df.scrapeNetwork` deploy label tells `swarm-listener` to use `cadvisor`'s IP on the `monitor` network. This is important because the `monitor` service is using the `monitor` network to scrape `cadvisor`. The `com.df.env=prod` and `com.df.metricType=system` deploy labels configures flexible labeling for `cadvisor`.
60+
61+
The second service, `node-exporter` is also configured with flexiable labels:
62+
63+
```yaml
64+
node-exporter:
65+
image: basi/node-exporter
66+
networks:
67+
- monitor
68+
...
69+
deploy:
70+
mode: global
71+
labels:
72+
...
73+
- com.df.scrapeNetwork=monitor
74+
- com.df.env=dev
75+
- com.df.metricType=system
76+
```
77+
78+
Let's deploy the `exporter` stack
79+
80+
```bash
81+
docker stack deploy \
82+
-c stacks/exporters-tutorial-flexible-labels.yml \
83+
exporter
84+
```
85+
86+
Please wait until the service in the stack are up-and-running. You can check their status by executing `docker stack ps exporter`.
87+
88+
Now we can open the *Prometheus* targets page from a browser.
89+
90+
> If you're a Windows user, Git Bash might not be able to use the `open` command. If that's the case, replace the `open` command with `echo`. As a result, you'll get the full address that should be opened directly in your browser of choice.
91+
92+
```bash
93+
open "http://$(docker-machine ip swarm-1):9090/targets"
94+
```
95+
96+
You should see a targets page similar to the following:
97+
98+
![Flexiable Labeling Targets Page](img/flexiable-labeling-targets-page.png)
99+
100+
Each service is labeled with its associated `com.df.env` or `com.df.metricType` deploy label. In addition, the `node` label is the hostname the service is running on.
101+
102+
## What Now?
103+
104+
*Docker Flow Monitors*'s flexible labeling feature provides more information about your services. Please consult the documentation for any additional information you might need. Feel free to open [an issue](https://github.com/vfarcic/docker-flow-monitor/issues) if you require additional info, if you find a bug, or if you have a feature request.
105+
106+
Before you go, please remove the cluster we created and free those resources for something else.
107+
108+
```bash
109+
docker-machine rm -f swarm-1 swarm-2 swarm-3
110+
```

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ pages:
44
- Tutorial:
55
- Running Docker Flow Monitor: tutorial.md
66
- Auto-Scaling Services Using Instrumented Metrics: auto-scaling.md
7+
- Flexible Labeling with Docker Flow Monitor: tutorial-flexible-labeling.md
78
- Configuration: config.md
89
- Usage: usage.md
910
- Migration Guide: migration.md

‎stacks/docker-flow-monitor-flexible-labels.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
version: "3"
2+
3+
services:
4+
5+
monitor:
6+
image: vfarcic/docker-flow-monitor:${TAG:-latest}
7+
environment:
8+
- LISTENER_ADDRESS=swarm-listener
9+
- GLOBAL_SCRAPE_INTERVAL=10s
10+
- ARG_ALERTMANAGER_URL=http://alert-manager:9093
11+
- DF_SCRAPE_TARGET_LABELS=env,metricType
12+
networks:
13+
- monitor
14+
ports:
15+
- 9090:9090
16+
17+
alert-manager:
18+
image: vfarcic/alert-manager:slack
19+
networks:
20+
- monitor
21+
22+
swarm-listener:
23+
image: vfarcic/docker-flow-swarm-listener
24+
networks:
25+
- monitor
26+
volumes:
27+
- /var/run/docker.sock:/var/run/docker.sock
28+
environment:
29+
- DF_NOTIFY_CREATE_SERVICE_URL=http://monitor:8080/v1/docker-flow-monitor/reconfigure
30+
- DF_NOTIFY_REMOVE_SERVICE_URL=http://monitor:8080/v1/docker-flow-monitor/remove
31+
- DF_INCLUDE_NODE_IP_INFO=true
32+
deploy:
33+
placement:
34+
constraints: [node.role == manager]
35+
36+
networks:
37+
monitor:
38+
external: true

‎stacks/exporters-tutorial-flexible-labels.yml‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
version: "3"
2+
3+
services:
4+
5+
cadvisor:
6+
image: google/cadvisor
7+
networks:
8+
- monitor
9+
volumes:
10+
- /:/rootfs
11+
- /var/run:/var/run
12+
- /sys:/sys
13+
- /var/lib/docker:/var/lib/docker
14+
deploy:
15+
mode: global
16+
labels:
17+
- com.df.notify=true
18+
- com.df.scrapePort=8080
19+
- com.df.scrapeNetwork=monitor
20+
- com.df.env=prod
21+
- com.df.metricType=system
22+
23+
node-exporter:
24+
image: basi/node-exporter
25+
networks:
26+
- monitor
27+
environment:
28+
- HOST_HOSTNAME=/etc/host_hostname
29+
volumes:
30+
- /proc:/host/proc
31+
- /sys:/host/sys
32+
- /:/rootfs
33+
- /etc/hostname:/etc/host_hostname
34+
deploy:
35+
mode: global
36+
labels:
37+
- com.df.notify=true
38+
- com.df.scrapePort=9100
39+
- com.df.alertName.1=mem_load
40+
- com.df.alertIf.1=(sum by (instance) (node_memory_MemTotal) - sum by (instance) (node_memory_MemFree + node_memory_Buffers + node_memory_Cached)) / sum by (instance) (node_memory_MemTotal) > 0.8
41+
- com.df.alertName.2=diskload
42+
- com.df.alertIf.2=@node_fs_limit:0.8
43+
- com.df.scrapeNetwork=monitor
44+
- com.df.env=dev
45+
- com.df.metricType=system
46+
command: '--path.procfs="/host/proc" --path.sysfs="/host/sys" --collector.filesystem.ignored-mount-points="^/(sys|proc|dev|host|etc)($$|/)" --collector.textfile.directory="/etc/node-exporter/" --collector.conntrack --collector.diskstats --collector.entropy --collector.filefd --collector.filesystem --collector.loadavg --collector.mdadm --collector.meminfo --collector.netdev --collector.netstat --collector.stat --collector.textfile --collector.time --collector.vmstat --collector.ipvs'
47+
48+
networks:
49+
monitor:
50+
external: true

0 commit comments

Comments
 (0)