Skip to content

Conversation

@klihub
Copy link
Member

@klihub klihub commented Apr 8, 2025

This PR adds NRI support for injecting 'linux net devices' a.k.a host side network interfaces into containers, building upon all the work @aojea did to make this possible.

Notes:
To test this in practice, one needs

  • runc or crun with support for LinuxNetDevices in the OCI Spec,
  • a patched runtime,
  • some way to inject linux net devices into containers,
  • and some test workload
    • I tested with a few host side VLANs and a test pod annotated for the device injector plugin
apiVersion: v1
kind: Pod
metadata:
  name: net-dev-test
  annotations:
    network-devices.noderesource.dev/container.c0: |
      - hostIf: ens2.100
        Name: netdev0
    network-devices.noderesource.dev/container.c1: |
      - hostIf: ens2.101
        Name: netdev1
spec:
  containers:
  - name: c0
    image: busybox:latest
    command:
      - sh
      - -c
      - sleep 3600
    resources:
      requests:
        cpu: 100m
        memory: 100M
      limits:
        cpu: 100m
        memory: 100M
    imagePullPolicy: IfNotPresent
  - name: c1
    image: busybox:latest
    command:
      - sh
      - -c
      - sleep 3600
    resources:
      requests:
        cpu: 1
        memory: 100M
      limits:
        cpu: 1
        memory: 100M
    imagePullPolicy: IfNotPresent
  terminationGracePeriodSeconds: 1

@klihub klihub force-pushed the devel/linux-net-devices branch from df6df04 to 02f0b68 Compare April 8, 2025 11:28
@aojea
Copy link
Contributor

aojea commented Apr 8, 2025

❤️

@klihub klihub force-pushed the devel/linux-net-devices branch from 02f0b68 to d886c51 Compare April 9, 2025 08:19
@klihub klihub requested review from fuweid and mikebrow May 2, 2025 08:24
@klihub klihub force-pushed the devel/linux-net-devices branch 2 times, most recently from df21d77 to 20533ae Compare June 24, 2025 11:10
@klihub klihub force-pushed the devel/linux-net-devices branch from 20533ae to 6e14a15 Compare June 24, 2025 11:13
@klihub
Copy link
Member Author

klihub commented Jun 24, 2025

/cc @aojea @tao12345666333 Here is the draft PR proposal #180 was asking for.

@aojea
Copy link
Contributor

aojea commented Jun 25, 2025

This is fantastic @klihub , can we undraft it for final review

@klihub klihub force-pushed the devel/linux-net-devices branch from 6e14a15 to a03dcca Compare June 25, 2025 06:26
@klihub
Copy link
Member Author

klihub commented Jun 25, 2025

This is fantastic @klihub , can we undraft it for final review

We can try to do that to get it reviewed. @mikebrow WDYT?

But I think we won't be able to get this merged before the necessary new bits in runtime-spec gets behind a tag. Both CRI-O and containerd main/HEAD is at [email protected] and I think we usually tend to stick to a tagged version in both even in main.

@aojea
Copy link
Contributor

aojea commented Jun 25, 2025

ok, that is fair

@aojea
Copy link
Contributor

aojea commented Jun 25, 2025

Fixes: #180

@aojea
Copy link
Contributor

aojea commented Oct 14, 2025

Containerd containerd/containerd#12295 is already testing runc 1.4.0-rc.2
CRIO uses crun that already has the feature https://github.com/containers/crun/releases/tag/1.22
OCI spec is close to release the definitive 1.3.0 opencontainers/runtime-spec#1295

@mikebrow
Copy link
Member

Thx for the update @aojea

@klihub klihub force-pushed the devel/linux-net-devices branch 2 times, most recently from 8df6cec to 0a96043 Compare October 15, 2025 14:33
@klihub
Copy link
Member Author

klihub commented Oct 15, 2025

Containerd containerd/containerd#12295 is already testing runc 1.4.0-rc.2 CRIO uses crun that already has the feature https://github.com/containers/crun/releases/tag/1.22 OCI spec is close to release the definitive 1.3.0 opencontainers/runtime-spec#1295

@aojea @mikebrow But I think we'd still need a tagged version of opencontainers/runtime-spec we could point ourselves at.

@aojea
Copy link
Contributor

aojea commented Oct 16, 2025

@klihub yeah, I was using the comment as a breadcrumb , so next time I check I can track teh history, otherwise I forget

I was not trying to push for merging before that, sorry for the confusion

@klihub
Copy link
Member Author

klihub commented Oct 16, 2025

@klihub yeah, I was using the comment as a breadcrumb , so next time I check I can track teh history, otherwise I forget

I was not trying to push for merging before that, sorry for the confusion

@aojea No prob / no offense taken. I just asked to make sure I understand correctly where we are and whether we can try to move this forward yet.

@aojea
Copy link
Contributor

aojea commented Oct 28, 2025

informational update , ETA for 1.3.0 spec is 2025/11/04 opencontainers/runc#4875 (comment) , runc will follow

@klihub
Copy link
Member Author

klihub commented Nov 4, 2025

@aojea @mikebrow With opencontainers/runtime-spec v1.3.0 now tagged and released, unfortunately opencontainer/runtime-tools is now badly out of sync with runtime-spec.

[email protected] brings a flips the linux PID limit setting the from int64 to *int64. While this allows to set an 'unlimited' limit using nil, it is a backward incompatible change and it essentially makes it impossible to import [email protected] and runtimes-tools@current HEAD to the same golang code base.

I filed a PR for fixing this but I'm not sure what are the chances of getting that merged soon

LinuxIOPriority io_priority = 6;
SecurityProfile seccomp_profile = 7;
LinuxSeccomp seccomp_policy = 8;
map<string, LinuxNetDevice> net_devices = 9;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the keys, and why is a map chosen vs. repeated?

Copy link
Contributor

@aojea aojea Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because network interface names are unique, so there is no need to validate or fail at runtime, the keys are the interface name

@aojea
Copy link
Contributor

aojea commented Nov 17, 2025

it seems the dependency problem got unblocked

@klihub klihub force-pushed the devel/linux-net-devices branch from 0a96043 to bf5c053 Compare November 17, 2025 12:07
@klihub klihub marked this pull request as ready for review November 17, 2025 12:10
@klihub klihub requested review from aojea and chrishenzie November 17, 2025 12:10
Allow adding and removing container linux net devices.

Signed-off-by: Krisztian Litkey <[email protected]>
@klihub klihub force-pushed the devel/linux-net-devices branch from bf5c053 to c2faa3d Compare November 17, 2025 12:20
@klihub klihub force-pushed the devel/linux-net-devices branch from c2faa3d to c5f3eae Compare November 17, 2025 14:36
Name: d.Name,
})
if !verbose {
log.Infof("%s: injected network device %q -> %q...", containerName(pod, ctr),
Copy link
Contributor

@chrishenzie chrishenzie Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This is unrelated since I see this is an established pattern, but I'm not following why we would conditionally write info logs. I would think we would always info log, and then conditionally decide whether or not to additionally dump verbose output above

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. It is a bit strange not to log that info in verbose mode. We can fix all instances of that wrong pattern in a separate PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants