Skip to content

Commit e6739e4

Browse files
committed
updates some docs and docker details
1 parent 3e40740 commit e6739e4

6 files changed

Lines changed: 249 additions & 23 deletions

File tree

.dockerignore

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Build artifacts
2+
build/
3+
4+
# Git history
5+
.git/
6+
7+
# Test dependency
8+
third_party/googletest/
9+
10+
# Documentation and working docs
11+
docs/
12+
memory/
13+
14+
# Scripts not needed for build
15+
scripts/
16+
17+
# Misc
18+
*.md

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,11 @@ Version numbers follow [Semantic Versioning](https://semver.org/).
1515

1616
### Added
1717

18+
**Docker**
19+
- Added Dockerfile with FZGPUModules pre-built and installed to `/usr/local` (headers, libs, CMake package config); supports local dev, CI/CD, and distribution from a single image
20+
- Added `.dockerignore` to exclude build artifacts, git history, and test dependencies from the build context
21+
- Added `docs/docker.md` covering image build, pre-installed library usage (`find_package` + `nvcc`), local source development, CI/CD patterns, and troubleshooting
22+
1823
**Documentation**
1924
- Added `docs/stages/` — per-stage Doxygen pages covering constraints, behavioral rules, mode details, and usage examples for all eight stages (`LorenzoQuantStage`, `LorenzoStage`, `QuantizerStage`, `DifferenceStage`, `BitshuffleStage`, `RZEStage`, `RLEStage`, `BitpackStage`)
2025
- Updated `Doxyfile` to include `docs/stages/` in `INPUT`; added `\ref stages_overview` link from the mainpage

Dockerfile

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# Base image: NVIDIA CUDA Toolkit (Development environment)
22
# Using 12.6 to match the development version used for FZGPUModules
3+
# Supports local development, CI/CD testing, and distribution/deployment.
4+
# FZGPUModules is pre-built and installed to /usr/local so users can link
5+
# against it without building the library themselves.
36
FROM nvcr.io/nvidia/cuda:12.6.0-devel-ubuntu24.04
47

58
# Prevent interactive prompts during apt installations
@@ -15,9 +18,21 @@ RUN apt-get update -qq && apt-get install -y --no-install-recommends \
1518
libubsan1 \
1619
curl \
1720
wget \
21+
python3 \
22+
python3-pip \
1823
&& rm -rf /var/lib/apt/lists/*
1924

20-
# Set up a working directory
25+
# Pre-build and install FZGPUModules to /usr/local.
26+
# After this step: headers at /usr/local/include/fzgmod/, libraries at /usr/local/lib/,
27+
# and CMake package config at /usr/local/lib/cmake/FZGPUModules/ (find_package ready).
28+
COPY . /tmp/fzgpumodules-src
29+
RUN cd /tmp/fzgpumodules-src && \
30+
cmake --preset release && \
31+
cmake --build build/release -j$(nproc) && \
32+
cmake --install build/release --prefix /usr/local && \
33+
rm -rf /tmp/fzgpumodules-src
34+
35+
# Set up a working directory for user code
2136
WORKDIR /workspace
2237

2338
# By default, open a bash shell

Doxyfile

Lines changed: 16 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -16,35 +16,31 @@ OUTPUT_DIRECTORY = docs/doxygen
1616
INPUT = include/ \
1717
modules/ \
1818
docs/mainpage.md \
19-
docs/cli.md \
2019
docs/building.md \
20+
docs/stages/ \
2121
docs/config_file.md \
22-
docs/fzm_format.md \
23-
docs/architecture.md \
22+
docs/cli.md \
2423
docs/api_reference.md \
24+
docs/architecture.md \
2525
docs/how_to_add_a_stage.md \
26-
docs/stages/
26+
docs/fzm_format.md \
27+
docs/docker.md
2728
RECURSIVE = YES
2829
FILE_PATTERNS = *.h *.md
2930
USE_MDFILE_AS_MAINPAGE = docs/mainpage.md
3031

31-
# Strip the repo root from displayed paths so paths look like include/pipeline/dag.h
3232
STRIP_FROM_PATH = .
3333

3434
#---------------------------------------------------------------------------
3535
# What to extract
3636
#---------------------------------------------------------------------------
37-
# Only show symbols that have a /** */ doc block.
38-
# This keeps the output clean — undocumented internals stay hidden.
37+
# Only show symbols that have a /** */ doc block
3938
EXTRACT_ALL = NO
4039
EXTRACT_PRIVATE = NO
4140
EXTRACT_STATIC = NO
4241
EXTRACT_ANON_NSPACES = NO
4342
HIDE_UNDOC_MEMBERS = YES
4443
HIDE_UNDOC_CLASSES = YES
45-
46-
# Show @internal-marked items in output (they're in public headers;
47-
# developers may still want to read them). Set to YES to hide them.
4844
INTERNAL_DOCS = YES
4945

5046
#---------------------------------------------------------------------------
@@ -57,14 +53,14 @@ GENERATE_XML = NO
5753

5854
HTML_OUTPUT = html
5955
HTML_DYNAMIC_SECTIONS = YES
60-
GENERATE_TREEVIEW = YES # Left-hand navigation panel
56+
GENERATE_TREEVIEW = YES
6157
DISABLE_INDEX = NO
6258
FULL_SIDEBAR = NO
6359

6460
#---------------------------------------------------------------------------
6561
# Source browsing
6662
#---------------------------------------------------------------------------
67-
SOURCE_BROWSER = NO # Don't embed raw source — keep docs focused
63+
SOURCE_BROWSER = NO
6864
INLINE_SOURCES = NO
6965
REFERENCED_BY_RELATION = NO
7066
REFERENCES_RELATION = NO
@@ -74,21 +70,20 @@ REFERENCES_RELATION = NO
7470
#---------------------------------------------------------------------------
7571
HAVE_DOT = YES
7672
DOT_NUM_THREADS = 4
77-
CLASS_GRAPH = YES # Inheritance diagrams
78-
COLLABORATION_GRAPH = NO # Too noisy for this codebase
73+
CLASS_GRAPH = YES
74+
COLLABORATION_GRAPH = NO
7975
INCLUDE_GRAPH = NO
8076
INCLUDED_BY_GRAPH = NO
8177
CALL_GRAPH = NO
8278
CALLER_GRAPH = NO
83-
DOT_IMAGE_FORMAT = svg # SVG scales cleanly on retina displays
79+
DOT_IMAGE_FORMAT = svg
8480
INTERACTIVE_SVG = YES
8581
MAX_DOT_GRAPH_DEPTH = 3
8682

8783
#---------------------------------------------------------------------------
8884
# Preprocessing
8985
#---------------------------------------------------------------------------
90-
# Doxygen doesn't understand CUDA, so suppress CUDA-specific keywords
91-
# so they don't appear in the rendered output.
86+
# Doxygen doesn't understand CUDA
9287
ENABLE_PREPROCESSING = YES
9388
MACRO_EXPANSION = YES
9489
EXPAND_ONLY_PREDEF = YES
@@ -104,17 +99,17 @@ PREDEFINED = __host__= \
10499
#---------------------------------------------------------------------------
105100
QUIET = NO
106101
WARNINGS = YES
107-
WARN_IF_UNDOCUMENTED = NO # Don't warn for intentionally undocumented items
102+
WARN_IF_UNDOCUMENTED = NO
108103
WARN_IF_DOC_ERROR = YES
109104
WARN_NO_PARAMDOC = NO
110105

111106
#---------------------------------------------------------------------------
112107
# Miscellaneous
113108
#---------------------------------------------------------------------------
114-
SORT_MEMBER_DOCS = NO # Keep declaration order — it's intentional
109+
SORT_MEMBER_DOCS = NO
115110
SORT_BRIEF_DOCS = NO
116-
ALPHABETICAL_INDEX = YES
111+
ALPHABETICAL_INDEX = NO
117112
SHOW_USED_FILES = NO
118113
SHOW_FILES = YES
119-
SHOW_NAMESPACES = YES
114+
SHOW_NAMESPACES = NO
120115
TYPEDEF_HIDES_STRUCT = NO

LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ Copyright (c) 2026, UChicago Argonne, LLC, and Indiana University
22

33
All Rights Reserved
44

5-
Software Name: FZGPUModules: A GPU-accelerated Module Library for Graph Based Compressor Composition
5+
Software Name: FZGPUModules: GPU-accelerated graph composable compression pipeline builder for analytical workflows.
66

77
OPEN SOURCE LICENSE
88

docs/docker.md

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
# Docker Setup
2+
3+
This document describes how to use Docker for FZGPUModules development, testing, and deployment.
4+
5+
## Overview
6+
7+
The provided Dockerfile creates a single image that supports all three use cases:
8+
- **Local development**: Full environment for building and developing FZGPUModules
9+
- **CI/CD testing**: Automated testing and validation in containerized environments
10+
- **Distribution/deployment**: Packaged environment with the library pre-installed
11+
12+
The image is based on NVIDIA CUDA 12.6.0 with Ubuntu 24.04 and includes:
13+
- CMake and Ninja build tools
14+
- C++ compiler (g++)
15+
- CUDA development libraries
16+
- Python 3 and pip
17+
- Sanitizer libraries (libasan8, libubsan1) for debugging
18+
19+
**FZGPUModules is pre-built and installed into the image** during the Docker build. After pulling the image, users can immediately write and compile code against the library — no library build step required.
20+
21+
Installed locations:
22+
- Headers: `/usr/local/include/fzgmod/`
23+
- Libraries: `/usr/local/lib/`
24+
- CMake package config: `/usr/local/lib/cmake/FZGPUModules/` (`find_package` ready)
25+
26+
## Building the Docker Image
27+
28+
```bash
29+
docker build -t fzgpumodules:latest .
30+
```
31+
32+
Or with a specific version tag:
33+
34+
```bash
35+
docker build -t fzgpumodules:1.0.0 .
36+
```
37+
38+
The build takes a few minutes — it compiles FZGPUModules during the image build so subsequent runs are instant.
39+
40+
## Using the Pre-Installed Library
41+
42+
### Quick Start
43+
44+
Mount a directory containing your code and compile directly:
45+
46+
```bash
47+
docker run --rm --gpus all \
48+
-v $(pwd):/workspace \
49+
fzgpumodules:latest \
50+
nvcc my_app.cu -o my_app -lfzgmod -L/usr/local/lib
51+
```
52+
53+
### With CMake (Recommended)
54+
55+
Use `find_package(FZGPUModules REQUIRED)` in your `CMakeLists.txt`:
56+
57+
```cmake
58+
cmake_minimum_required(VERSION 3.20)
59+
project(MyApp CUDA CXX)
60+
61+
find_package(FZGPUModules REQUIRED)
62+
63+
add_executable(my_app my_app.cu)
64+
target_link_libraries(my_app PRIVATE FZGMOD::fzgmod)
65+
```
66+
67+
Then build inside the container:
68+
69+
```bash
70+
docker run --rm --gpus all \
71+
-v $(pwd):/workspace \
72+
fzgpumodules:latest \
73+
bash -c "cmake -B build -S . && cmake --build build"
74+
```
75+
76+
### Interactive Shell
77+
78+
Start an interactive session to develop incrementally:
79+
80+
```bash
81+
docker run --rm -it --gpus all \
82+
-v $(pwd):/workspace \
83+
fzgpumodules:latest
84+
```
85+
86+
## Local Development (Building FZGPUModules Itself)
87+
88+
To build and modify FZGPUModules source, mount the repo and build inside the container:
89+
90+
```bash
91+
docker run --rm -it --gpus all \
92+
-v /path/to/FZGPUModules:/workspace/src \
93+
fzgpumodules:latest \
94+
bash -c "cd /workspace/src && \
95+
cmake --preset release -DBUILD_EXAMPLES=ON && \
96+
cmake --build build/release -j$(nproc)"
97+
```
98+
99+
The pre-installed library in the image is unaffected — your source build stays in the mounted directory.
100+
101+
## CI/CD Testing
102+
103+
### Running the Test Suite
104+
105+
```bash
106+
docker run --rm --gpus all \
107+
-v $(pwd):/workspace/src \
108+
fzgpumodules:latest \
109+
bash -c "cd /workspace/src && \
110+
cmake --preset release -DBUILD_TESTING=ON && \
111+
cmake --build build/release -j$(nproc) && \
112+
ctest --test-dir build/release --output-on-failure"
113+
```
114+
115+
### Full Build with All Targets
116+
117+
```bash
118+
docker run --rm --gpus all \
119+
-v $(pwd):/workspace/src \
120+
fzgpumodules:latest \
121+
bash -c "cd /workspace/src && \
122+
cmake --preset release \
123+
-DBUILD_EXAMPLES=ON \
124+
-DBUILD_TESTING=ON \
125+
-DBUILD_PROFILING=ON && \
126+
cmake --build build/release -j$(nproc)"
127+
```
128+
129+
## GPU Support
130+
131+
All commands above use `--gpus all` to enable GPU access. This requires:
132+
- NVIDIA Docker runtime installed on the host
133+
- NVIDIA driver compatible with CUDA 12.6
134+
135+
To verify GPU access in the container:
136+
137+
```bash
138+
docker run --rm --gpus all fzgpumodules:latest nvidia-smi
139+
```
140+
141+
On some systems you may need `--runtime=nvidia` instead of `--gpus all`.
142+
143+
## Development Notes
144+
145+
### Sanitizers
146+
147+
The image includes AddressSanitizer (libasan8) and UndefinedBehaviorSanitizer (libubsan1). To build with sanitizers:
148+
149+
```bash
150+
docker run --rm -it --gpus all \
151+
-v $(pwd):/workspace/src \
152+
fzgpumodules:latest \
153+
bash -c "cd /workspace/src && \
154+
cmake --preset release \
155+
-DCMAKE_CXX_FLAGS='-fsanitize=address,undefined' && \
156+
cmake --build build/release -j$(nproc)"
157+
```
158+
159+
### Python Integration
160+
161+
Python 3 and pip are available for profiling scripts or future bindings:
162+
163+
```bash
164+
docker run --rm -it fzgpumodules:latest python3 --version
165+
```
166+
167+
## Troubleshooting
168+
169+
### GPU Not Detected
170+
171+
If `nvidia-smi` fails inside the container:
172+
- Verify the NVIDIA container toolkit is installed on the host
173+
- Check that your driver version supports CUDA 12.6 (`nvidia-smi` on the host shows the max supported CUDA version)
174+
- Try `--runtime=nvidia` if `--gpus all` is not recognized
175+
176+
### find_package Cannot Find FZGPUModules
177+
178+
The CMake package config is at `/usr/local/lib/cmake/FZGPUModules/`. If `find_package` fails, set the path explicitly:
179+
180+
```bash
181+
cmake -B build -S . -DFZGPUModules_DIR=/usr/local/lib/cmake/FZGPUModules
182+
```
183+
184+
### Build Failures in CI
185+
186+
If the image build fails during the `cmake --install` step:
187+
- Ensure `third_party/tomlplusplus/` is present (it must be vendored, not just a submodule reference)
188+
- Verify `third_party/googletest/` is initialized if running with `-DBUILD_TESTING=ON` during the image build (it is excluded from the default release build)
189+
190+
## See Also
191+
192+
- [Building FZGPUModules](building.md) — General build instructions
193+
- [Architecture Guide](architecture.md) — Project structure and design

0 commit comments

Comments
 (0)