Skip to content

Commit 142e305

Browse files
sfirosfionatrahe
authored andcommitted
QATlib: 23.11.0 release
Changes from 23.08.0 to 23.11.0: * Support DC NS (NoSession) APIs * Support Symmetric Crypto SM3 & SM4 * Support Asymmetric Crypto SM2 * DC compressBound APIs * Bug Fixes. See Resolved section in README.md Signed-off-by: Firos Sadarul <[email protected]>
1 parent 7429ee2 commit 142e305

File tree

96 files changed

+15528
-1858
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

96 files changed

+15528
-1858
lines changed

INSTALL

Lines changed: 6 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,4 @@
11
===============================================================================
2-
3-
4-
August 2023
52
===============================================================================
63

74

@@ -94,10 +91,9 @@ Check System Prerequisites
9491
intel_qat
9592
qat_4xxx
9693
They should load by default if using any of the following:
97-
* Linux kernel v5.11+ (This is for crypto, for compression use v5.17+,
98-
for sym-only asym-only, sym;dc and asym;dc use v6.6+, for dcc use v6.7+)
99-
* Fedora 34+ (for compression use 36+)
100-
* RHEL 8.4+ (for compression use 9.0+)
94+
* A recent Linux kernel (see https://intel.github.io/quickassist/qatlib/requirements.html#kernel-firmware-requirements )
95+
* Fedora 36+
96+
* RHEL 8.4+ (for compression use 9.0+)
10197
* each PF device must be bound to the 4xxx driver
10298
Use "ls /sys/bus/pci/drivers/4xxx/" to show the BDFs of each bound PF
10399
* BIOS settings
@@ -169,7 +165,7 @@ Compilation and installation - quickstart instructions
169165

170166
# Install dependencies
171167
sudo dnf install -y gcc systemd-devel automake autoconf libtool
172-
sudo dnf install -y openssl-devel zlib-devel nasm
168+
sudo dnf install -y pkg-config openssl-devel zlib-devel nasm
173169

174170
# Clone QATlib into ~/qatlib, i.e. in your home dir
175171
cd ~
@@ -227,6 +223,7 @@ Compilation and installation - detailed instructions
227223
gcc
228224
make
229225
autotools (automake, autoconf, libtool)
226+
pkg-config
230227
systemd-devel
231228
nasm
232229
Note: If nasm compiler is unavailable see
@@ -720,7 +717,7 @@ Common issues
720717
bound to qat_4xxx:
721718
sudo lspci -vvd:4940 | grep "Kernel driver in use".
722719
sudo lspci -vvd:4942 | grep "Kernel driver in use"
723-
If no driver in use, upgrade to Linux kernel 5.11 or greater.
720+
upgrade to a recent Linux Kernel.
724721
(2) No VFs available. Check VFs are available and bound to vfio-pci
725722
sudo lspci -vvd:4941 | grep "Kernel driver in use"
726723
sudo lspci -vvd:4943 | grep "Kernel driver in use"

Makefile.am

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,8 @@ libadf_la_SOURCES = \
7171
quickassist/lookaside/access_layer/src/qat_direct/vfio/qat_log.c \
7272
quickassist/lookaside/access_layer/src/qat_direct/vfio/vfio_lib.c \
7373
quickassist/lookaside/access_layer/src/qat_direct/vfio/adf_pfvf_proto.c \
74-
quickassist/lookaside/access_layer/src/qat_direct/vfio/adf_pfvf_vf_msg.c
74+
quickassist/lookaside/access_layer/src/qat_direct/vfio/adf_pfvf_vf_msg.c \
75+
quickassist/lookaside/access_layer/src/qat_direct/vfio/adf_vfio_pf.c
7576
libadf_la_CFLAGS = -I$(srcdir)/quickassist/utilities/libusdm_drv \
7677
-I$(srcdir)/quickassist/utilities/osal/include \
7778
-I$(srcdir)/quickassist/utilities/osal/src/linux/user_space/include \
@@ -93,7 +94,8 @@ qatmgr_SOURCES = \
9394
quickassist/lookaside/access_layer/src/qat_direct/vfio/qat_log.c \
9495
quickassist/lookaside/access_layer/src/qat_direct/vfio/adf_pfvf_proto.c \
9596
quickassist/lookaside/access_layer/src/qat_direct/vfio/adf_pfvf_vf_msg.c \
96-
quickassist/lookaside/access_layer/src/qat_direct/vfio/vfio_lib.c
97+
quickassist/lookaside/access_layer/src/qat_direct/vfio/vfio_lib.c \
98+
quickassist/lookaside/access_layer/src/qat_direct/vfio/adf_vfio_pf.c
9799
qatmgr_CFLAGS = -I$(srcdir)/quickassist/lookaside/access_layer/src/qat_direct/vfio \
98100
-I$(srcdir)/quickassist/lookaside/access_layer/include \
99101
-I$(srcdir)/quickassist/include \
@@ -126,7 +128,7 @@ libcrc_SOURCES_DIR = quickassist/lookaside/access_layer/src/common/compression/
126128
%.lo:
127129
@echo " CCAS $(libcrc_SOURCES_DIR)$@"
128130
@$(LIBTOOL) --mode=compile --quiet \
129-
nasm -f elf64 -D LINUX -X gnu $(libcrc_SOURCES_DIR)$(@:.lo=.S) -I$(libcrc_SOURCES_DIR) -o $@ -prefer-non-pic
131+
nasm -f elf64 -D LINUX -X gnu $(top_srcdir)/$(libcrc_SOURCES_DIR)$(@:.lo=.S) -I$(top_srcdir)/$(libcrc_SOURCES_DIR) -o $@ -prefer-non-pic
130132
endif
131133

132134
lib_LTLIBRARIES += lib@[email protected]
@@ -158,6 +160,7 @@ lib@LIBQATNAME@_la_SOURCES = \
158160
quickassist/lookaside/access_layer/src/common/crypto/asym/ecc/lac_ec_nist_curves.c \
159161
quickassist/lookaside/access_layer/src/common/crypto/asym/ecc/lac_ecdh.c \
160162
quickassist/lookaside/access_layer/src/common/crypto/asym/ecc/lac_ecdsa.c \
163+
quickassist/lookaside/access_layer/src/common/crypto/asym/ecc/lac_ecsm2.c \
161164
quickassist/lookaside/access_layer/src/common/crypto/asym/ecc/lac_kpt_ecdsa.c \
162165
quickassist/lookaside/access_layer/src/common/crypto/asym/large_number/lac_ln.c \
163166
quickassist/lookaside/access_layer/src/common/crypto/asym/large_number/lac_ln_interface_check.c \
@@ -261,6 +264,7 @@ pkginclude_HEADERS = \
261264
quickassist/include/lac/cpa_cy_dsa.h \
262265
quickassist/include/lac/cpa_cy_ecdh.h \
263266
quickassist/include/lac/cpa_cy_ecdsa.h \
267+
quickassist/include/lac/cpa_cy_ecsm2.h \
264268
quickassist/include/lac/cpa_cy_ec.h \
265269
quickassist/include/lac/cpa_cy_im.h \
266270
quickassist/include/lac/cpa_cy_key.h \

README.md

Lines changed: 61 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626

2727
| Date | Doc Revision | Version | Details |
2828
|----------|:-------------:|------:|:------|
29+
| November 2023 | 011 | 23.11 | - Support DC NS (NoSession) APIs. <br> - Support DC compressBound APIs. <br> - Support Symmetric Crypto SM3 & SM4. <br> - Support Asymmetric Crypto SM2. <br> - Bug Fixes. See [Resolved Issues](#resolved-issues). |
2930
| August 2023 | 010 | 23.08 | - Removal of following insecure algorithms: Diffie-Hellman and Elliptic curves less than 256-bits. <br> - Additional configuration profiles, including sym which facilitates improved symmetric crypto performance. <br> - DC Chaining (Hash then compress) <br> - Bug Fixes. See [Resolved Issues](#resolved-issues). |
3031
| February 2023 | 009 | 23.02 | - Added configuration option --enable-legacy-algorithms to use these insecure crypto algorithms and disabled them by default (AES-ECB, SHA-1, SHA2-224, SHA3-224, RSA512/1024/1536, DSA)<br>- Refactored code in quickassist/utilities/libusdm_drv<br>- Bugfixes<br>- Updated documentation with configuration and tuning information |
3132
| November 2022 | 008 | 22.07.2 | - Changed from yasm to nasm for assembly compilation<br> - Added configuration option to use C implementation of soft CRC implementation instead of asm<br>- Added support for pkg-config<br>- Added missing lock around accesses to some global data in qatmgr |
@@ -56,9 +57,10 @@ sample codes.
5657
The following services are available in qatlib via the QuickAssist API:
5758
* Symmetric (Bulk) Cryptography
5859
* Ciphers ([AES-ECB](#insecure-algorithms), AES-CBC, AES-CTR (no partials support),
59-
AES-XTS (no partials support), AES-GCM, AES-CCM (192/256)
60+
AES-XTS (no partials support), AES-GCM, AES-CCM (192/256), [SM4-ECB](#insecure-algorithms),
61+
SM4-CBC, SM4-CTR)
6062
* Message digest/hash ([SHA1](#insecure-algorithms), SHA2 ([224](#insecure-algorithms)/256/384/512),
61-
SHA3 ([224](#insecure-algorithms)/256/384/512) (no partials support) and
63+
SHA3 ([224](#insecure-algorithms)/256/384/512) (no partials support), SM3) and
6264
authentication (AES-CBC-MAC, AES-XCBC-MAC)
6365
* Algorithm chaining (one cipher and one hash in a single operation)
6466
* Authenticated encryption (CCM-128 (no partials support),
@@ -77,12 +79,15 @@ The following services are available in qatlib via the QuickAssist API:
7779
* [DSA](#insecure-algorithms) parameter generation and digital signature generation/verification
7880
* Elliptic Curve Cryptography: ECDSA, ECDHE, Edwards Montgomery curves
7981
* Generic point multiply
82+
* SM2
8083
* Compression
8184
* Deflate
8285
* lz4/lz4s
8386
* Compress and Verify (CnV)
8487
* Compress and Verify and Recover (CnVnR)
8588
* End-to-end (E2E) integrity check
89+
* DC compressBound APIs
90+
* DC NS (No Session) APIs
8691
* Compression Chaining (Deflate only)
8792
* Hash then compress
8893

@@ -96,12 +101,13 @@ This package includes:
96101
The following algorithms are considered insecure and are disabled by default.
97102
* AES-ECB
98103
* SHA-1
99-
* SHA2-224
104+
* SHA2-224
100105
* SHA3-224
101106
* RSA512/1024/1536
102107
* DSA
103108
* Diffie-Helman
104109
* Elliptic Curve Cryptography algorithms with less 256 bits
110+
* SM4-ECB
105111

106112
To enable these algorithms, use the following configuration option:
107113
* `--enable-legacy-algorithms`
@@ -122,6 +128,9 @@ supported.
122128
## Limitations
123129
* If an error occurs on the host driver (Heartbeat, Uncorrectable error) it
124130
will not be communicated to the library.
131+
* For simplicity, only one configuration file is used by qatlib. For guidance
132+
on how to use this to allocate resources for processes, please refer to
133+
Configuration and Tuning section in [QATlib User’s Guide](https://intel.github.io/quickassist/qatlib/index.html).
125134

126135
The following features are not currently supported:
127136
* Dynamic instances
@@ -147,8 +156,6 @@ The following assumptions are made concerning the deployment environment:
147156
discovered and initialized the device, exposing the VFs. This driver is
148157
included in the Linux kernel, see [INSTALL](INSTALL) for information about which kernel
149158
to use.
150-
* The library can be used by unprivileged users if that user is included in
151-
the 'qat' group.
152159

153160
## Examples
154161
Example applications that showcase usage of the QAT APIs are included in the
@@ -180,8 +187,7 @@ where: \<Component\> is one of the following:
180187
| QATE-41707 | [CY - Incorrect digest returned when performing a plain hash operation on input data of size 4GB or larger.](#qate-41707) |
181188
| QATE-76073 | [GEN - If PF device configuration is modified without restarting qatmgr, undefined behavior may occur.](#qate-76073) |
182189
| QATE-76698 | [GEN - Multi-process applications running in guest will fail when running with default Policy settings.](#qate-76698) |
183-
| QATE-94286 | [CY - Compression services not detected when crypto-capable VFs are added to VM.](#qate-94286) |
184-
| QATE-94369 | [GEN - SELinux Preventing QAT Service Startup](#qate-94369) |
190+
| QATE-94369 | [GEN - SELinux Preventing QAT Service Startup.](#qate-94369) |
185191

186192
## QATE-3241
187193
| Title | CY - cpaCySymPerformOp when used with parameter checking may reveal the amount of padding. |
@@ -224,20 +230,10 @@ where: \<Component\> is one of the following:
224230
| Affected OS | Linux |
225231
| Driver/Module | CPM-IA - General |
226232

227-
## QATE-94286
228-
| Title | GEN - Compression services not detected when crypto-capable VFs are also added to VM. |
229-
|----------|:-------------
230-
| Reference # | QATE-94286 |
231-
| Description | When configuring a system with different services on different QAT end-points, e.g. asym;sym on one and dc on another, and exposing only one of those Virtual Function (VF) types to the Virtual Machine (VM), the application works as expected. However, when VFs of more than one type are passed to the same VM, the application may only recognize one service-type, e.g. it may detect crypto instances, but not compression instances. There is an assumption that all VFs provide the same services if they come from the same PF. However, detecting which PF they come from is based on domain+bus, which is not always a valid assumption on a VM. |
232-
| Implication | This issue prevents the detection of compression services in a virtualized environment when the default kernel configuration is used, and crypto and dc VFs are passed to the VM, potentially impacting the proper functioning of the system. |
233-
| Resolution | When passing VFs to a guest, the BDFs on the guest should facilitate qatlib recognizing whether VFs are from the same PF or not. See RUNNING IN A VIRTUAL MACHINE / GUEST section of INSTALL for details. |
234-
| Affected OS | Linux |
235-
| Driver/Module | CPM-IA - General |
236-
237233
## QATE-94369
238234
| Title | GEN - SELinux Preventing QAT Service Startup |
239235
|----------|:-------------
240-
| Reference # | QATE-94286 |
236+
| Reference # | QATE-94369 |
241237
| Description | The qat service fails to start due to SELinux preventing the qat_init.sh script and qatmgr from accessing resources. The issue occurs when the system is running with SELinux enabled, causing insufficient permissions for the qat_init.sh script and qatmgr to function correctly. |
242238
| Implication | This issue affects the proper functioning of the qat service on systems with SELinux enabled, potentially preventing QAT virtual functions (VFs) from functioning. |
243239
| Resolution | None available. |
@@ -250,23 +246,45 @@ in this section.
250246

251247
| Issue ID | Description |
252248
|-------------|------------|
253-
| QATE-90845 | [GEN - QAT service fails to start, issue #38](#qate-90845) |
249+
| QATE-94286 | [GEN - Compression services not detected when crypto-capable VFs are added to VM.](#qate-94286) |
250+
| QATE-95905 | [GEN - Fix build when building outside of main directory, issue #56](#qate-95905) |
251+
| QATE-93844 | [DC - cpaDcLZ4SCompressBound is not returning correct value, which could lead to a buffer overflow.](#qate-93844)
254252
| QATE-93278 | [GEN - sample_code potential seg-fault, issue #46](#qate-93278) |
253+
| QATE-90845 | [GEN - QAT service fails to start, issue #38](#qate-90845) |
254+
| QATE-78459 | [DC - cpaDcDeflateCompressBound API returns incorrect output buffer size when input size exceeds 477218588 bytes.](#qate-78459) |
255255
| QATE-76846 | [GEN - Forking and re-initializing use-cases do not work](#qate-76846) |
256-
| QATE-78459 | [DC - cpaDcDeflateCompressBound API returns incorrect output buffer size when input size exceeds 477218588 bytes.](#qate-74786) |
257256
| QATE-12241 | [CY - TLS1.2 with secret key lengths greater than 64 are not supported.](#qate-12241) |
258257

258+
## QATE-94286
259+
| Title | GEN - Compression services not detected when crypto-capable VFs are also added to VM. |
260+
|----------|:-------------
261+
| Reference # | QATE-94286 |
262+
| Description | When configuring a system with different services on different QAT end-points, e.g. asym;sym on one and dc on another, and exposing only one of those Virtual Function (VF) types to the Virtual Machine (VM), the application works as expected. However, when VFs of more than one type are passed to the same VM, the application may only recognize one service-type, e.g. it may detect crypto instances, but not compression instances. There is an assumption that all VFs provide the same services if they come from the same PF. However, detecting which PF they come from is based on domain+bus, which is not always a valid assumption on a VM. |
263+
| Implication | This issue prevents the detection of compression services in a virtualized environment when the default kernel configuration is used, and crypto and dc VFs are passed to the VM, potentially impacting the proper functioning of the system. |
264+
| Resolution | Fixed in 23.11. <br>Temporary solution: use a custom libvirt XML file like QATE-76698 here: https://github.com/intel/qatlib/tree/main#qate-76698 . |
265+
| Affected OS | Linux |
266+
| Driver/Module | CPM-IA - General |
259267

260-
## QATE-90845
261-
| Title | GEN - QAT service fails to start, issue #38 |
268+
## QATE-95905
269+
| Title | GEN - Fix build when building outside of main directory, issue #56 |
262270
|----------|:-------------
263-
| Reference # | QATE-90845 |
264-
| Description | QAT service fails to start. The qat service may fail if the kernel driver's initialization is not fully finished when the service starts. See [issue 38](https://github.com/intel/qatlib/issues/38). |
265-
| Implication | The qatmgr may not detect any or all of the vfio devices. |
266-
| Resolution | Fixed in 23.08. The service waits until the kernel driver has completed initialization of all PFs before starting the service. |
271+
| Reference # | QATE-95905 |<F3>
272+
| Description | Fix build when building outside of main directory. Added changes to autoconfig to be able to build outside main directory. See [issue 56](https://github.com/intel/qatlib/issues/56). |
273+
| Implication | A fatal error occurs when trying to build outside main directory. |
274+
| Resolution | Fixed in 23.11. |
267275
| Affected OS | Linux |
268276
| Driver/Module | CPM-IA - General |
269277

278+
## QATE-93844
279+
| Title | DC - cpaDcLZ4SCompressBound is not returning correct value, which could lead to a buffer overflow. |
280+
|----------|:-------------
281+
| Reference # | QATE-93844 |
282+
| Description | CompressBound API (cpaDcLZ4SCompressBound()) is intended to return the maximum size of the output buffer. However, this API is not returning the correct value, which can lead to a lz4s buffer overflow. |
283+
| Implication | Applications may experience buffer overflows even when using the output of compressBound API to allocate output buffers. |
284+
| Resolution | Fixed in 23.11 |
285+
| Affected OS | Linux |
286+
| Driver/Module | QAT IA - Compression |
287+
270288
## QATE-93278
271289
| Title | GEN - sample_code potential seg-fault, issue #46 |
272290
|----------|:-------------
@@ -277,13 +295,13 @@ in this section.
277295
| Affected OS | Linux |
278296
| Driver/Module | CPM-IA - General |
279297

280-
## QATE-76846
281-
| Title | GEN - Forking and re-initializing use-cases do not work |
298+
## QATE-90845
299+
| Title | GEN - QAT service fails to start, issue #38 |
282300
|----------|:-------------
283-
| Reference # | QATE-76846 |
284-
| Description | Forking and re-initializing use-cases do not work:<br>-icp_sal_userStart()/icp_sal_userStop()/icp_sal_userStart() in single process<br>-icp_sal_userStart()/fork()/icp_sal_userStart() in child.<br> This is the use case in openssh + QAT_Engine. |
285-
| Implication | The process will have undefined behavior in these use-cases. |
286-
| Resolution | This issue is resolved with the 21.08 release. If using release prior to this release and using these flows, call qaeMemDestroy() immediately after icp_sal_userStop() to prevent this issue. |
301+
| Reference # | QATE-90845 |
302+
| Description | QAT service fails to start. The qat service may fail if the kernel driver's initialization is not fully finished when the service starts. See [issue 38](https://github.com/intel/qatlib/issues/38). |
303+
| Implication | The qatmgr may not detect any or all of the vfio devices. |
304+
| Resolution | Fixed in 23.08. The service waits until the kernel driver has completed initialization of all PFs before starting the service. |
287305
| Affected OS | Linux |
288306
| Driver/Module | CPM-IA - General |
289307

@@ -297,13 +315,23 @@ in this section.
297315
| Affected OS | Linux |
298316
| Driver/Module | CPM-IA - Data Compression |
299317

318+
## QATE-76846
319+
| Title | GEN - Forking and re-initializing use-cases do not work |
320+
|----------|:-------------
321+
| Reference # | QATE-76846 |
322+
| Description | Forking and re-initializing use-cases do not work:<br>-icp_sal_userStart()/icp_sal_userStop()/icp_sal_userStart() in single process<br>-icp_sal_userStart()/fork()/icp_sal_userStart() in child.<br> This is the use case in openssh + QAT_Engine. |
323+
| Implication | The process will have undefined behavior in these use-cases. |
324+
| Resolution | Fixed in 21.08. If using release prior to this release and using these flows, call qaeMemDestroy() immediately after icp_sal_userStop() to prevent this issue. |
325+
| Affected OS | Linux |
326+
| Driver/Module | CPM-IA - General |
327+
300328
## QATE-12241
301329
| Title | CY - TLS1.2 with secret key lengths greater than 64 are not supported |
302330
|----------|:-------------
303331
| Reference # | QATE-12241 |
304332
| Description | Algorithms, as with Diffie-Hellman using 8K parameters that can use a secret key length greater than 64 bytes is not supported.|
305333
| Implication | Key generation would fail for TLS1.2 algorithms that use more than 64 bytes secret length keys. |
306-
| Resolution | This is resolved with the 22.07 release. |
334+
| Resolution | Fixed in 22.07. |
307335
| Affected OS | Linux |
308336
| Driver/Module | CPM-IA - Crypto |
309337

0 commit comments

Comments
 (0)