how to properly set up gvisor (especially with cilium)?

[xyhhx]

in the gvisor directory's readme, it is documented that unprivileged user namespace creation must be enabled¹. to do that, it simply says to patch the user.max_user_namespaces sysctl.

but there are other docs that describe enabling user namespaces in talos/kubernetes: https://www.talos.dev/v1.9/kubernetes-guides/configuration/usernamespace/ - do we need to do this as well to accomodate gvisor?

then of course there is this issue which suggests that gvisor is possibly not compatible with the latest version(s) of talos?

finally, there are many issues related to setting up gvisor alongside cilium²³ as well as docs explicitly to support this use case⁴. at first, i'm inclined to say this is out of scope for talos documentation; but on the other hand, if a user sets up cilium as documented in talos docs, gvisor doesn't work and from my experimentation it is far from trivial to reconfigure talos/kubernetes/cilium in such a way to handle gvisor resulting in major service disruption if done improperly

it might be a good idea to at least mention that, if a user configured cilium according to the docs, gvisor will not work and further configuration is needed, and care must be taken to do it properly

Footnotes

1. https://github.com/siderolabs/extensions/tree/main/container-runtime/gvisor#usage ↩

2. https://github.com/google/gvisor/issues/6998 ↩

3. https://github.com/cilium/cilium/issues/15626 ↩

4. https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/#socket-loadbalancer-bypass-in-pod-namespace ↩

[smira]

There's a bit of confusion on the term "user namespaces" here, there are two things:

• by default, Talos doesn't allow non-root users to create new Linux namespaces, setting max_user_namespaces reverts this value; creating namespaces by non-root users very often leads to exploitable issues in the Linux kernel, specifically networking stack (as the creator will be effectively able to use any networking stuff in the new namespace)
• user namespace as part of Linux namespaces which allows to remap users inside the namespace to the host, i.e. user ID 0 in the namespace is actually user ID 10000 on the host.

Any improvements to the documentation are very welcome as a PR to this or Talos repo!

[xyhhx]

gotcha, thanks for clearing up the user namespaces stuff!

as for helping out with docs, i still haven't gotten it working; so until then i won't be able to help much 😮‍💨

[xyhhx]

okay i got it working actually, and the good news is most of my pains were mostly about tampering with my cni on an already-deployed cluster; so the updates i'm gonna propose for the docs will be minimal

[Piccirello]

I'm currently running Talos v1.10.2 w/ Cilium 1.17. When switching a pod's runtime class to gvisor, it is no longer able to communicate with other pods. @xyhhx could you share how you were able to get connectivity working?

[github-actions]

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[obeone]

Personally, I’m still waiting for a reply on this. It was just to poke the stale-bot and make sure the issue wouldn’t be closed ;)

[frezbo]

I think this should be asked in upstream cilium/gvisor repository/