Is this plug in support intel nic supporting rdma?

[thsmfe001]

My server environment will use connectx-7 vpi and intel 810-XXV nic card. So i need to register intel nic into rdma resource pool.
Is it possible to use intel nic? If so, could you provide config.map to filter them to register on rdma resource pool?

[adrianchiris]

if 810-XXV supports RDMA then it should be possible. i have not tested on intel hardware

just specify intel vendor ID and device ID as selectors in the resource pool and try it out :)

[thsmfe001]

Please refer to my configmap.yaml file and plug-in logs.

kind: ConfigMap
metadata:
name: rdma-devices
namespace: kube-system
data:
config.json: |
{
"periodicUpdateInterval": 300,
"configList": [
{
"resourceName": "intel_rdma_devices",
"rdmaHcaMax": 1000,
"selectors": {
"vendors": ["8086"],
"deviceIDs": ["0d00", "b500", "a000"],
"ifNames": ["eno16595", "eno17105", "enp13s0f0", "enp160s0f1"],
"drivers": ["irdma"],
"linkTypes": ["ethernet"]
}
}
]
}

kubectl logs rdma-shared-dp-ds-5gpr9 -n kube-system
2024/06/25 04:11:20 Starting K8s RDMA Shared Device Plugin version= master
2024/06/25 04:11:20 resource manager reading configs
2024/06/25 04:11:20 Reading /k8s-rdma-shared-dev-plugin/config.json
Using Kubelet Plugin Registry Mode
2024/06/25 04:11:20 loaded config: [{ResourceName:intel_rdma_devices ResourcePrefix: RdmaHcaMax:1000 Devices:[] Selectors:{Vendors:[8086] DeviceIDs:[0d00 b500 a000] Drivers:[irdma] IfNames:[eno16595 eno17105 enp13s0f0 enp160s0f1] LinkTypes:[ethernet]}}]
2024/06/25 04:11:20 periodic update interval: +300
2024/06/25 04:11:20 Discovering host devices
2024/06/25 04:11:20 discovering host network devices
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:01:00.0 02 Broadcom Inc. and... NetXtreme BCM5720 Gigabit Ethernet PCIe
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:01:00.1 02 Broadcom Inc. and... NetXtreme BCM5720 Gigabit Ethernet PCIe
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:0d:00.0 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:0d:00.1 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:0d:01.0 02 Intel Corporation Ethernet Adaptive Virtual Function
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:22:00.0 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:22:00.1 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:22:00.2 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:22:00.3 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:b5:00.0 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:b5:00.1 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:11:20 DiscoverHostDevices(): device found: 0000:b5:11.0 02 Intel Corporation Ethernet Adaptive Virtual Function
2024/06/25 04:11:20 Initializing resource servers
2024/06/25 04:11:20 Resource: &{ResourceName:intel_rdma_devices ResourcePrefix:rdma RdmaHcaMax:1000 Devices:[] Selectors:{Vendors:[8086] DeviceIDs:[0d00 b500 a000] Drivers:[irdma] IfNames:[eno16595 eno17105 enp13s0f0 enp160s0f1] LinkTypes:[ethernet]}}
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:01:00.0, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:01:00.1, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:0d:00.0, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:0d:00.1, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:0d:01.0, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:22:00.0, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:22:00.1, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:22:00.2, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:22:00.3, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:b5:00.0, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:b5:00.1, RDMA device "issm" not found"
2024/06/25 04:11:20 error creating new device: "missing RDMA device spec for device 0000:b5:11.0, RDMA device "issm" not found"
2024/06/25 04:11:20 Warning: no devices in device pool, creating empty resource server for intel_rdma_devices
2024/06/25 04:11:20 Warning: no Rdma Devices were found for resource intel_rdma_devices
2024/06/25 04:11:20 Starting all servers...
2024/06/25 04:11:20 starting rdma/intel_rdma_devices device plugin endpoint at: intel_rdma_devices.sock
2024/06/25 04:11:20 rdma/intel_rdma_devices device plugin endpoint started serving
2024/06/25 04:11:20 All servers started.
2024/06/25 04:11:20 Listening for term signals
2024/06/25 04:11:20 Starting OS watcher.
2024/06/25 04:11:21 intel_rdma_devices.sock gets registered successfully at Kubelet
2024/06/25 04:11:21 ListAndWatch called by kubelet for: rdma/intel_rdma_devices
2024/06/25 04:11:21 Updating "rdma/intel_rdma_devices" devices
2024/06/25 04:11:21 exposing "0" devices
2024/06/25 04:16:20 discovering host network devices
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:01:00.0 02 Broadcom Inc. and... NetXtreme BCM5720 Gigabit Ethernet PCIe
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:01:00.1 02 Broadcom Inc. and... NetXtreme BCM5720 Gigabit Ethernet PCIe
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:0d:00.0 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:0d:00.1 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:0d:01.0 02 Intel Corporation Ethernet Adaptive Virtual Function
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:22:00.0 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:22:00.1 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:22:00.2 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:22:00.3 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:b5:00.0 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:b5:00.1 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:16:20 DiscoverHostDevices(): device found: 0000:b5:11.0 02 Intel Corporation Ethernet Adaptive Virtual Function
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:01:00.0, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:01:00.1, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:0d:00.0, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:0d:00.1, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:0d:01.0, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:22:00.0, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:22:00.1, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:22:00.2, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:22:00.3, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:b5:00.0, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:b5:00.1, RDMA device "issm" not found"
2024/06/25 04:16:20 error creating new device: "missing RDMA device spec for device 0000:b5:11.0, RDMA device "issm" not found"
2024/06/25 04:16:20 no changes to devices for "rdma/intel_rdma_devices"
2024/06/25 04:16:20 exposing "0" devices
2024/06/25 04:21:20 discovering host network devices
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:01:00.0 02 Broadcom Inc. and... NetXtreme BCM5720 Gigabit Ethernet PCIe
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:01:00.1 02 Broadcom Inc. and... NetXtreme BCM5720 Gigabit Ethernet PCIe
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:0d:00.0 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:0d:00.1 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:0d:01.0 02 Intel Corporation Ethernet Adaptive Virtual Function
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:22:00.0 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:22:00.1 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:22:00.2 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:22:00.3 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:b5:00.0 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:b5:00.1 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:21:20 DiscoverHostDevices(): device found: 0000:b5:11.0 02 Intel Corporation Ethernet Adaptive Virtual Function
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:01:00.0, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:01:00.1, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:0d:00.0, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:0d:00.1, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:0d:01.0, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:22:00.0, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:22:00.1, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:22:00.2, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:22:00.3, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:b5:00.0, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:b5:00.1, RDMA device "issm" not found"
2024/06/25 04:21:20 error creating new device: "missing RDMA device spec for device 0000:b5:11.0, RDMA device "issm" not found"
2024/06/25 04:21:20 no changes to devices for "rdma/intel_rdma_devices"
2024/06/25 04:21:20 exposing "0" devices
2024/06/25 04:26:20 discovering host network devices
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:01:00.0 02 Broadcom Inc. and... NetXtreme BCM5720 Gigabit Ethernet PCIe
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:01:00.1 02 Broadcom Inc. and... NetXtreme BCM5720 Gigabit Ethernet PCIe
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:0d:00.0 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:0d:00.1 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:0d:01.0 02 Intel Corporation Ethernet Adaptive Virtual Function
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:22:00.0 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:22:00.1 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:22:00.2 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:22:00.3 02 Intel Corporation I350 Gigabit Network Connection
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:b5:00.0 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:b5:00.1 02 Intel Corporation Ethernet Controller E810-XXV for SFP
2024/06/25 04:26:20 DiscoverHostDevices(): device found: 0000:b5:11.0 02 Intel Corporation Ethernet Adaptive Virtual Function
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:01:00.0, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:01:00.1, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:0d:00.0, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:0d:00.1, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:0d:01.0, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:22:00.0, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:22:00.1, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:22:00.2, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:22:00.3, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:b5:00.0, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:b5:00.1, RDMA device "issm" not found"
2024/06/25 04:26:20 error creating new device: "missing RDMA device spec for device 0000:b5:11.0, RDMA device "issm" not found"
2024/06/25 04:26:20 no changes to devices for "rdma/intel_rdma_devices"
2024/06/25 04:26:20 exposing "0" devices

[adrianchiris]

are you using the latest k8s-rdma-shared-device-plugin ? we dont require issm rdma device anymore

[1] https://github.com/Mellanox/k8s-rdma-shared-dev-plugin/blob/master/pkg/resources/rdma_device_spec.go#L19