Skip to content

Commit ba85437

Browse files
committed
feat: Add GStreamer video recorder with comprehensive sample application
- Implements GstVideoRecorderOperator for encoding video streams to file - Adds RAII GStreamer wrapper classes (Buffer, Caps, Memory, etc.) - Supports zero-copy tensor-to-buffer conversion for host/CUDA memory - Auto-detects video parameters, encoders, parsers, and muxers - Includes comprehensive sample app with animated test patterns - Provides flexible CLI with encoder properties and format support - Handles H.264/H.265 encoding with MP4/MKV container formats
1 parent 5a16b6d commit ba85437

28 files changed

+3814
-0
lines changed

.devcontainer/devcontainer.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
},
3434
"runArgs": [
3535
"--net=host",
36+
"--runtime=nvidia"
3637
//"<env>"
3738
],
3839
"containerEnv": {

applications/CMakeLists.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,8 @@ add_holohub_application(endoscopy_tool_tracking DEPENDS
5656
yuan_qcap
5757
vtk_renderer)
5858

59+
add_subdirectory(gstreamer)
60+
5961
add_subdirectory(h264)
6062

6163
add_holohub_application(isaac_sim_holoscan_bridge)
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
add_holohub_application(gst_video_recorder)
17+
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
project(gst_video_recorder CXX)
17+
18+
find_package(holoscan REQUIRED CONFIG
19+
PATHS "/opt/nvidia/holoscan" "/workspace/holoscan-sdk/install")
20+
21+
add_executable(gst-video-recorder
22+
gst_video_recorder.cpp
23+
)
24+
25+
target_link_libraries(gst-video-recorder
26+
PRIVATE
27+
holoscan::core
28+
holoscan_gstreamer_bridge
29+
)
30+
31+
# Install target
32+
install(TARGETS gst-video-recorder DESTINATION bin/gst_video_recorder)
33+
34+
Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
4+
# SPDX-License-Identifier: Apache-2.0
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
18+
ARG GPU_TYPE
19+
ARG BASE_SDK_VERSION
20+
ARG BASE_IMAGE=nvcr.io/nvidia/clara-holoscan/holoscan:v${BASE_SDK_VERSION}-${GPU_TYPE}
21+
FROM ${BASE_IMAGE} AS base
22+
23+
ARG DEBIAN_FRONTEND=noninteractive
24+
ARG CMAKE_BUILD_TYPE=Release
25+
26+
# --------------------------------------------------------------------------
27+
#
28+
# Set up prerequisites to run HoloHub CLI
29+
#
30+
# --------------------------------------------------------------------------
31+
FROM base AS holohub-cli-prerequisites
32+
33+
# Install python3 if not present (needed for holohub CLI)
34+
ARG PYTHON_VERSION=python3
35+
RUN if ! command -v python3 >/dev/null 2>&1; then \
36+
apt-get update \
37+
&& apt-get install --no-install-recommends -y \
38+
software-properties-common curl gpg-agent \
39+
&& add-apt-repository ppa:deadsnakes/ppa \
40+
&& apt-get update \
41+
&& apt-get install --no-install-recommends -y \
42+
${PYTHON_VERSION} \
43+
&& apt purge -y \
44+
python3-pip \
45+
software-properties-common \
46+
&& apt-get autoremove --purge -y \
47+
&& rm -rf /var/lib/apt/lists/* \
48+
&& update-alternatives --install /usr/bin/python python /usr/bin/${PYTHON_VERSION} 100 \
49+
&& if [ "${PYTHON_VERSION}" != "python3" ]; then \
50+
update-alternatives --install /usr/bin/python3 python3 /usr/bin/${PYTHON_VERSION} 100 \
51+
; fi \
52+
; fi
53+
ENV PIP_BREAK_SYSTEM_PACKAGES=1
54+
RUN if ! python3 -m pip --version >/dev/null 2>&1; then \
55+
curl -sS https://bootstrap.pypa.io/get-pip.py | ${PYTHON_VERSION} \
56+
; fi
57+
58+
# --------------------------------------------------------------------------
59+
#
60+
# Set up GStreamer-specific dependencies for gst_video_recorder
61+
#
62+
# --------------------------------------------------------------------------
63+
FROM holohub-cli-prerequisites AS holohub-cli
64+
65+
RUN mkdir -p /tmp/scripts
66+
COPY holohub /tmp/scripts/
67+
RUN mkdir -p /tmp/scripts/utilities
68+
COPY utilities /tmp/scripts/utilities/
69+
RUN chmod +x /tmp/scripts/holohub
70+
RUN /tmp/scripts/holohub setup && rm -rf /var/lib/apt/lists/*
71+
72+
# Install GStreamer development packages and pkg-config
73+
RUN apt-get update \
74+
&& apt-get install --no-install-recommends -y \
75+
pkg-config \
76+
libgstreamer1.0-dev \
77+
libgstreamer-plugins-base1.0-dev \
78+
libgstreamer-plugins-bad1.0-dev \
79+
gstreamer1.0-plugins-base \
80+
gstreamer1.0-plugins-bad \
81+
gstreamer1.0-plugins-good \
82+
gstreamer1.0-plugins-ugly \
83+
gstreamer1.0-libav \
84+
&& apt-get autoremove -y \
85+
&& apt-get clean -y \
86+
&& rm -rf /var/lib/apt/lists/*
87+
88+
# Enable autocomplete
89+
RUN echo ". /etc/bash_completion.d/holohub_autocomplete" >> /etc/bash.bashrc
90+
91+
# Set default Holohub data directory
92+
ENV HOLOSCAN_INPUT_PATH=/workspace/holohub/data
93+
94+
# --------------------------------------------------------------------------
95+
#
96+
# Default development stage for GStreamer applications
97+
#
98+
# --------------------------------------------------------------------------
99+
FROM holohub-cli AS holohub-dev
Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,181 @@
1+
# GStreamer Video Recorder
2+
3+
A Holoscan application that demonstrates video recording using the GStreamer encoding pipeline.
4+
5+
## Description
6+
7+
This application showcases how to:
8+
- Generate video frames in Holoscan using the pattern generator
9+
- Feed video frames to GStreamer for encoding
10+
- Record encoded video to files in various formats (MP4, MKV)
11+
- Use different video codecs (H.264, H.265, and other GStreamer-supported codecs)
12+
- Support both host and CUDA device memory for zero-copy operation
13+
14+
## Requirements
15+
16+
- NVIDIA Holoscan SDK
17+
- GStreamer 1.0 with the following plugins:
18+
- gstreamer1.0-plugins-base (videoconvert for host memory support)
19+
- gstreamer1.0-plugins-bad (cudaconvert, nvh264enc, nvh265enc for NVIDIA hardware encoding)
20+
- gstreamer1.0-plugins-good (mp4mux, matroskamux for container formats)
21+
- gstreamer1.0-plugins-ugly (x264enc for CPU-based H.264 encoding)
22+
- Additional codecs available through gstreamer1.0-libav if needed
23+
24+
## Building
25+
26+
### Option 1: Containerized Build (Recommended)
27+
No setup required - all dependencies are included in the container:
28+
29+
```bash
30+
./holohub build gst_video_recorder
31+
```
32+
33+
### Option 2: Local Build
34+
For faster builds and easier debugging. First install dependencies:
35+
36+
```bash
37+
# From the gst_video_recorder directory
38+
./install_deps.sh
39+
40+
# Then build locally
41+
./holohub build --local gst_video_recorder
42+
```
43+
44+
The `install_deps.sh` script installs:
45+
- pkg-config (required for CMake)
46+
- GStreamer development libraries
47+
- All necessary GStreamer plugins for encoding
48+
49+
## Usage
50+
51+
```bash
52+
gst-video-recorder [OPTIONS]
53+
```
54+
55+
### Options
56+
57+
- `-o, --output <filename>` - Output video filename (default: output.mp4)
58+
- Supported formats: .mp4, .mkv
59+
- If no extension, defaults to .mp4
60+
- `-e, --encoder <name>` - Encoder base name (default: nvh264)
61+
- Examples: nvh264, nvh265, x264, x265
62+
- Note: 'enc' suffix is automatically appended
63+
- `-c, --count <number>` - Number of frames to generate (default: unlimited)
64+
- `-w, --width <pixels>` - Frame width (default: 1920)
65+
- `-h, --height <pixels>` - Frame height (default: 1080)
66+
- `-f, --framerate <rate>` - Frame rate as fraction or decimal (default: 30/1)
67+
- Examples: '30/1', '30000/1001', '29.97', '60'
68+
- Use '0/1' for live mode (no throttling, real-time timestamps)
69+
- `--pattern <type>` - Pattern type (default: 0)
70+
- 0 = animated gradient
71+
- 1 = animated checkerboard
72+
- 2 = color bars (SMPTE style)
73+
- `--storage <type>` - Memory storage type (default: 1)
74+
- 0 = host memory
75+
- 1 = device/CUDA memory
76+
- `--property <key=value>` - Set encoder property (can be used multiple times)
77+
- Examples: --property bitrate=8000 --property preset=1
78+
- Property types are automatically detected and converted
79+
- `--help` - Show help message
80+
81+
### Examples
82+
83+
#### Record 10 seconds of video at 30fps (300 frames)
84+
85+
```bash
86+
gst-video-recorder --count 300 -o video.mp4
87+
```
88+
89+
#### Record high quality H.265 video
90+
91+
```bash
92+
gst-video-recorder --count 300 --encoder nvh265 --property bitrate=10000 -o video.mp4
93+
```
94+
95+
#### Record 720p video
96+
97+
```bash
98+
gst-video-recorder --count 300 --width 1280 --height 720 -o video_720p.mp4
99+
```
100+
101+
#### Record using host memory (CPU)
102+
103+
```bash
104+
gst-video-recorder --count 300 --storage 0 --encoder x264 -o video.mp4
105+
```
106+
107+
#### Record with H.265 to MKV container
108+
109+
```bash
110+
gst-video-recorder --count 300 --encoder nvh265 -o video.mkv
111+
```
112+
113+
#### Record animated checkerboard pattern
114+
115+
```bash
116+
gst-video-recorder --count 300 --pattern 1 -o checkerboard.mp4
117+
```
118+
119+
#### Record with custom encoder properties
120+
121+
```bash
122+
gst-video-recorder --count 300 --property bitrate=8000 --property preset=1 --property gop-size=30 -o custom.mp4
123+
```
124+
125+
#### Record with NTSC framerate (29.97 fps)
126+
127+
```bash
128+
gst-video-recorder --count 300 --framerate 30000/1001 -o ntsc.mp4
129+
```
130+
131+
## Architecture
132+
133+
The application consists of two main components:
134+
135+
1. **PatternGenOperator**: Generates animated test patterns as Holoscan entities with tensors
136+
2. **GstVideoRecorderOperator**: Receives video frames, manages the GStreamer pipeline, and handles encoding
137+
138+
### Pipeline Flow
139+
140+
```
141+
PatternGenOperator → GstVideoRecorderOperator → GStreamer Encoding Pipeline → File
142+
```
143+
144+
The GStreamer encoding pipeline is automatically constructed based on the encoder and file format:
145+
146+
- **Pipeline structure**: `[converter] ! [encoder]enc ! [parser] ! [muxer] ! filesink`
147+
- **Converter**: Automatically selected based on memory type (videoconvert for host, cudaconvert for device)
148+
- **Encoder**: Specified via `--encoder` option (nvh264, nvh265, x264, x265, etc.)
149+
- **Parser**: Automatically determined from encoder (h264parse, h265parse, etc.)
150+
- **Muxer**: Automatically determined from file extension (mp4mux for .mp4, matroskamux for .mkv)
151+
152+
Example pipelines:
153+
154+
- **NVIDIA H.264 to MP4**: `cudaconvert ! nvh264enc ! h264parse ! mp4mux ! filesink`
155+
- **NVIDIA H.265 to MKV**: `cudaconvert ! nvh265enc ! h265parse ! matroskamux ! filesink`
156+
- **CPU x264 to MP4**: `videoconvert ! x264enc ! h264parse ! mp4mux ! filesink`
157+
158+
## Performance
159+
160+
The application supports both host and device (CUDA) memory:
161+
162+
- **Device memory** (`--storage 1`, default): Zero-copy operation for better performance when using NVIDIA hardware encoders (nvh264enc, nvh265enc)
163+
- **Host memory** (`--storage 0`): Required for CPU encoders (x264, x265) but involves memory copies
164+
165+
## Notes
166+
167+
- The pattern generator supports three test patterns:
168+
- Animated gradient (default): Colorful sine wave patterns
169+
- Animated checkerboard: Moving checkerboard with variable square size
170+
- Color bars: SMPTE-style color bars (7 colors)
171+
- To use your own video source, replace `PatternGenOperator` with your video capture operator
172+
- The application waits for encoding to complete before exiting to ensure proper file finalization
173+
- EOS (End-Of-Stream) signal is sent automatically when recording completes
174+
- Video parameters (width, height, format, storage) are automatically detected from incoming frames
175+
176+
## See Also
177+
178+
- `holopattern_to_gst` - Similar application that can output to any GStreamer pipeline
179+
- GStreamer documentation: https://gstreamer.freedesktop.org/documentation/
180+
181+

0 commit comments

Comments
 (0)