Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# SPDX-License-Identifier: GPL-2.0-only
# Copyright (C) 2025 Intel Corporation
# These contents may have been developed with support from one or more
# Intel-operated generative artificial intelligence solutions.
.*
*.o
*.o.*
*.a
*.s
*.ko
*.so
*.so.dbg
*.mod
*.mod.c
*.i
*.lst
*.symtypes
*.order
modules.builtin
*.elf
*.bin
*.gz
*.bz2
*.lzma
*.xz
*.lz4
*.lzo
*.patch
*.patch.conv
*.gcno

#
# Top-level generic files
#
/tags
/TAGS
/linux
/vmlinux
/vmlinuz
/System.map
/Module.markers
Module.symvers

# stgit generated dirs
patches-*

# quilt's files
patches
series

# cscope files
cscope.*
ncscope.*

# gnu global files
GPATH
GRTAGS
GSYMS
GTAGS

*.orig
*~
\#*#
177 changes: 177 additions & 0 deletions INSTALL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
# Installation Guide for Acompress Provider

This document provides step-by-step instructions for installing and configuring the Acompress Provider for DPUSM and ZFS.

## Prerequisites

Before you begin, ensure that you have the following:

- A system with Linux Kernel support for the Crypto API's acomp interface and zlib-deflate (versions 6.3 to 6.6).
- Administrative access to the system for installation and configuration tasks.
- Setup top level source directory.

```bash
export HOME_PATH=/devel/zfs
sudo mkdir -p $HOME_PATH
sudo chown -R $USER:$USER $HOME_PATH
```

## Component 1: DPUSM

### Download DPUSM

1. Clone the DPUSM repository:

```bash
cd $HOME_PATH
git clone https://github.com/hpc/dpusm
cd dpusm
```

### Build and Install DPUSM

2. Build DPUSM

```bash
make
```

3. Load the module:

```bash
sudo insmod dpusm.ko
```

4. Verify module is loaded:

```bash
lsmod | grep dpusm
```

Example output:

```bash
lsmod | grep dpusm
dpusm 36864 0
```

## Component 2: ZFS with Z.I.A.

### Obtain ZFS with Z.I.A.

To use ZFS with the ZFS Interface for Accelerators (Z.I.A.), you need to obtain the specific branch that includes Z.I.A. support. Follow these steps:

1. Clone the repository with the Z.I.A.:

```bash
cd $HOME_PATH
git clone https://github.com/hpc/zfs.git
cd zfs
```

### Build and Install ZFS

1. Build ZFS

Notes: If DPUSM is installed to another directory, update the following command accordingly.

```bash
./autogen.sh
./configure --with-zia=$HOME_PATH/dpusm
make -j$(nproc)
```

3. Install ZFS:

```bash
sudo make install
```

4. Load ZFS modules:

```bash
sudo modprobe zfs
```

5. Verify ZFS with Z.I.A was loaded:

```bash
sudo dmesg | grep -E "Z\.I\.A|ZFS"
```

The output will look like:

```bash
user@server:~/zfs$ sudo dmesg | grep -E "Z\.I\.A|ZFS"
[59203.606953] Z.I.A. initialized (0000000029342ebc)
[59204.818383] ZFS: Loaded module v2.3.99-313_g325a5e241, ZFS pool version 5000, ZFS filesystem version 5
```

## Component 3: Acompress Provider

### Clone the Acompress Provider Repository

1. Clone the Acompress Provider repository:

```bash
cd $HOME_PATH
git clone https://github.com/intel/acomp-dpusm.git
cd acomp-dpusm
```

### Build and Install Provider:

1. Install the Acompress Provider:

```bash
make
```

3. Install the Provider:

```bash
sudo insmod acomp-dpusm.ko
```

4. Verify provided was loaded:

```bash
sudo dmesg | grep acomp_dpusm
```

Example output:

```bash
user@server:~/acomp-dpusm$ sudo dmesg | grep acomp_dpusm
[62503.851101] Provider acomp_dpusm supports GZIP Level 1 Compress
[62503.851112] Provider acomp_dpusm supports GZIP Level 2 Compress
[62503.851114] Provider acomp_dpusm supports GZIP Level 3 Compress
[62503.851116] Provider acomp_dpusm supports GZIP Level 4 Compress
[62503.851117] Provider acomp_dpusm supports GZIP Level 5 Compress
[62503.851118] Provider acomp_dpusm supports GZIP Level 6 Compress
[62503.851119] Provider acomp_dpusm supports GZIP Level 7 Compress
[62503.851120] Provider acomp_dpusm supports GZIP Level 8 Compress
[62503.851121] Provider acomp_dpusm supports GZIP Level 9 Compress
[62503.851122] Provider acomp_dpusm supports GZIP Level 1 Decompress
[62503.851123] Provider acomp_dpusm supports GZIP Level 2 Decompress
[62503.851124] Provider acomp_dpusm supports GZIP Level 3 Decompress
[62503.851125] Provider acomp_dpusm supports GZIP Level 4 Decompress
[62503.851126] Provider acomp_dpusm supports GZIP Level 5 Decompress
[62503.851127] Provider acomp_dpusm supports GZIP Level 6 Decompress
[62503.851128] Provider acomp_dpusm supports GZIP Level 7 Decompress
[62503.851129] Provider acomp_dpusm supports GZIP Level 8 Decompress
[62503.851130] Provider acomp_dpusm supports GZIP Level 9 Decompress
[62503.851131] dpusm_provider_register: DPUSM Provider "acomp_dpusm" (00000000863c55ce) added. Now 1 providers registered.
```

## Runtime Configuration

Ensure the zpools are configured to use the provider and enable compression/decompression. The operations would look like:

```bash
sudo $HOME_PATH/zfs/zpool create -f test_zpool /dev/sda
sudo $HOME_PATH/zfs/zpool set zia_provider=acomp_dpusm test_zpool
sudo $HOME_PATH/zfs/zpool zia_compress=on test_zpool
sudo $HOME_PATH/zfs/zpool zia_decompress=on test_zpool
sudo $HOME_PATH/zfs/zfs set compression=gzip test_zpool
```
44 changes: 44 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# SPDX-License-Identifier: GPL-2.0-only
# Copyright (C) 2025 Intel Corporation
# These contents may have been developed with support from one or more
# Intel-operated generative artificial intelligence solutions.

# Define the kernel source directory
KERNELDIR ?= /lib/modules/$(shell uname -r)/build

# Define the module name
MODULE_NAME := acomp-dpusm

# Define the home path for the DPUSM and ZFS source directories
HOME_PATH ?= /devel/zfs

# Define the include paths for DPUSM
DPUSM_INCLUDE := $(HOME_PATH)/dpusm/include

# Define the include paths for ZFS
ZFS_INCLUDE := $(HOME_PATH)/zfs/include

# Define the source files for the module
SRC_FILES := provider.c compress.c

# Define the object files for the module
OBJ_FILES := $(SRC_FILES:.c=.o)

# Define the location of the module symvers file for DPUSM
KBUILD_EXTRA_SYMBOLS := $(HOME_PATH)/dpusm/Module.symvers

ifneq ($(KERNELRELEASE),)
ccflags-y := -I$(DPUSM_INCLUDE) -I$(ZFS_INCLUDE)
obj-m := $(MODULE_NAME).o
$(MODULE_NAME)-y := $(OBJ_FILES)
else
PWD := $(shell pwd)

default:
$(MAKE) -C $(KERNELDIR) M=$(PWD) W=1 modules

clean:
$(MAKE) -C $(KERNELDIR) M=$(PWD) clean
endif

.PHONY: clean
57 changes: 57 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Acompress Provider for DPUSM

## Table of Contents

- [Introduction](#introduction)
- [Licensing](#licensing)
- [Software Requirements](#software-requirements)
- [Limitations](#limitations)
- [Installation](#installation)
- [Legal](#legal)

## Introduction

The Acompress Provider is a plugin designed to integrate with DPUSM (Data Processing Unit Storage Management) and ZFS, utilizing the Z.I.A (ZFS Interface for Accelerators) framework. It enhances data compression capabilities within ZFS environments by leveraging the Crypto API's acomp interface for efficient data processing and storage management.

<img src="docs/images/stack.png" alt="stack diagram" width="500"/>


## Licensing

This project is licensed under the GNU General Public License v2.0. Please see the `LICENSE` file contained in the top-level folder for details.

## Software Requirements

- Linux Kernel with support for the Crypto API's acomp interface and zlib-deflate (currently covering versions 6.3 to 6.6).
Copy link

Copilot AI May 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation lists kernel versions 6.3–6.6, but the PR description specifies 6.4–6.6; update the version range for consistency.

Suggested change
- Linux Kernel with support for the Crypto API's acomp interface and zlib-deflate (currently covering versions 6.3 to 6.6).
- Linux Kernel with support for the Crypto API's acomp interface and zlib-deflate (currently covering versions 6.4 to 6.6).

Copilot uses AI. Check for mistakes.
- ZFS installed and operational configured with support for Z.I.A.
- DPUSM installed and configured.
- zpool configured to use this provider and compression offloading enabled


## Limitations

1. The Acompress Provider currently supports specific compression algorithms available through the Crypto API's acomp interface.
2. Ensure compatibility with your existing DPUSM and ZFS setup before integration.
3. Performance may vary based on hardware and system configuration.

## Installation

For detailed installation instructions, please refer to the `INSTALL.md` file in this repository.

## Legal

Intel&reg; disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

This document contains information on products, services, and/or processes in development. All information provided here is subject to change without notice. Contact your Intel&reg; representative to obtain the latest forecast, schedule, specifications, and roadmaps.

The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request.

Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or by visiting www.intel.com/design/literature.htm.

Intel, the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.

\*Other names and brands may be claimed as the property of others

---

Thank you for using the Acompress Provider! We hope it enhances your data processing and storage management experience.
Loading