-
Notifications
You must be signed in to change notification settings - Fork 428
Include nftables information in Agent supportbundle #7547
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
001ea65 to
6bb8141
Compare
|
/test-all |
6bb8141 to
f6801cb
Compare
pkg/support/dump_others.go
Outdated
| } | ||
|
|
||
| func (d *agentDumper) dumpNFTables(basedir string) error { | ||
| c, err := nftables.New(d.v4Enabled, d.v6Enabled) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If nftables is not supported, we should not fail the suppot bundle collection operation
cc @hongliangl
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think we should just log the error and return nil.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what are all these files for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My apologies. Those were temporary files generated by my local tests that were added by mistake.
I have removed them from the commit in the latest push. Thanks for catching that!
f6801cb to
95e0ce7
Compare
hongliangl
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would save the results of table ip antrea and table ip6 antrea to one file nftables, just like file iptables.
Additionally, could you add the unit tests for this?
pkg/support/dump_others.go
Outdated
| func (d *agentDumper) dumpNFTables(basedir string) error { | ||
| c, err := nftables.New(d.v4Enabled, d.v6Enabled) | ||
| if err != nil { | ||
| klog.Warningf("Skipping nftables dump, client init failed: %v", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't use klog.Warningf. Just use klog.ErrorS or klog.InfoS.
pkg/support/dump_others.go
Outdated
| } | ||
|
|
||
| if d.v4Enabled && c.IPv4 != nil { | ||
| output, err := c.IPv4.DumpTable(context.TODO(), "antrea") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't find the method DumpTable in the interface from nftables lib https://github.com/hongliangl/knftables/blob/e4307300abb50f2d9e7a492c474fe046d777f87f/nftables.go#L30.
Please use the appropriate methods to dump the table.
pkg/support/dump_others.go
Outdated
| if d.v4Enabled && c.IPv4 != nil { | ||
| output, err := c.IPv4.DumpTable(context.TODO(), "antrea") | ||
| if err != nil { | ||
| klog.Warningf("Failed to dump nftables table antrea-v4: %v", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should return an error here since nftables.New succeeded.
pkg/support/dump_others.go
Outdated
| if err := writeFile(d.fs, filepath.Join(basedir, fileName), fileName, output); err != nil { | ||
| klog.Warningf("Failed to write %s: %v", fileName, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto, return the error.
pkg/support/dump_others.go
Outdated
| return nil | ||
| } | ||
|
|
||
| if d.v4Enabled && c.IPv4 != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| if d.v4Enabled && c.IPv4 != nil { | |
| if c.IPv4 != nil { |
pkg/support/dump_others.go
Outdated
| } | ||
| } | ||
|
|
||
| if d.v6Enabled && c.IPv6 != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| if d.v6Enabled && c.IPv6 != nil { | |
| if c.IPv6 != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the detailed feedback! I've pushed the updates incorporating all your suggestions
95e0ce7 to
f41be5c
Compare
pkg/support/dump_others.go
Outdated
| func (d *agentDumper) dumpNFTables(basedir string) error { | ||
| _, err := d.executor.LookPath("nft") | ||
| if err != nil { | ||
| klog.InfoS("Skipping nftables dump, 'nft' command not found", "err", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| klog.InfoS("Skipping nftables dump, 'nft' command not found", "err", err) | |
| klog.ErrorS(err, "'nft' command not found") |
f41be5c to
a6a96a3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM overall from me, only a nit in PR description:
issue : #7546 -> Fixes: #7546
For git commit and PR titles, how about removing the prefix Fix(#7546):?
Defer to @antoninbas @luolanzone to do the final approval.
|
Please resolve the failed checks :
|
a6a96a3 to
c8a0ab9
Compare
c8a0ab9 to
e917f2b
Compare
Go/Golangci-lint I reordered the import statements using goimports -w -local antrea.io/antrea pkg/support/dump_others.go. Go/Unit test I modified the code to not write data when there is no output (empty string), like so: dump_others.go dump_others_test.go DCO I corrected the Signed-off-by line from just a name to the full email address. |
pkg/support/dump_others.go
Outdated
| } | ||
|
|
||
| func (d *agentDumper) dumpNFTables(basedir string) error { | ||
| _, err := d.executor.LookPath("nft") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hongliangl shouldn't we use the wrapper library here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should prioritize using the library. But unfortunately, the library doesn't provide a method to dump all contents of a table. The existing methods for dumping:
// List returns a list of the names of the objects of objectType ("chain", "set",
// "map" or "counter") in the table. If there are no such objects, this will return an empty
// list and no error.
List(ctx context.Context, objectType string) ([]string, error)
// ListRules returns a list of the rules in a chain, in order. If no chain name is
// specified, then all rules within the table will be returned. Note that at the
// present time, the Rule objects will have their `Comment` and `Handle` fields
// filled in, but *not* the actual `Rule` field. So this can only be used to find
// the handles of rules if they have unique comments to recognize them by, or if
// you know the order of the rules within the chain. If the chain exists but
// contains no rules, this will return an empty list and no error.
ListRules(ctx context.Context, chain string) ([]*Rule, error)
// ListElements returns a list of the elements in a set or map. (objectType should
// be "set" or "map".) If the set/map exists but contains no elements, this will
// return an empty list and no error.
ListElements(ctx context.Context, objectType, name string) ([]*Element, error)
// ListCounters returns a list of the counters.
ListCounters(ctx context.Context) ([]*Counter, error)We can use the above methods to dump a table, but it is a little complicated:
- Use
Listto dump all chains, sets, maps - Use
ListRulesto dump all chains - Use
ListElementsto dump all sets and maps
pkg/support/dump_others.go
Outdated
| } | ||
|
|
||
| if data.Len() == 0 { | ||
| klog.InfoS("No Antrea nftables rules found to dump") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would omit this log message
| klog.ErrorS(err, "'nft' command not found") | ||
| return nil | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nft can be found (in the antrea-agent container image), but that doesn't mean it will run successfully: maybe nf_tables is not available.
We should check for compatibility once during initialization, and skip dumpNFTables if there is no support in the host kernel. @hongliangl what do you think i the best way to do that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about this:
- Only initialize the wrapper library
Clientto check if nftables is available. This is because the initialization process does the following checks. See the linkfunc newNFTables(ipFamily knftables.Family) (knftables.Interface, error) {
// knftables.New validates:
// - nft binary is available
// - sufficient permissions
// - kernel version compatibility
// - "nft destroy" support
If an error is got when initializing, just log something that nftables is not available and skip dumping nftables.
- Use the
d.executor.Command("nft", "list", "table", "ip", "antrea").CombinedOutput()to dump the table.
The first step might be confusing, and we need to document it well to state that this step is to verify the availability of nftables.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @antoninbas and @hongliangl,
Just a gentle ping regarding the updates based on your previous discussion.
I've refactored the PR according to the approach you mentioned:
Startup Check: Added nftclient.New in agent.go to verify kernel compatibility during initialization.
Dump Execution: Kept d.executor.Command("nft", ...) for the actual dump since the library doesn't support a full dump.
Log Cleanup: Removed the "No Antrea nftables rules found to dump" log message.
I believe this reflects all the points you raised.
When you have a moment, could you please take a look?
Thanks!
|
In agent.go, I made it check for nftables support once at agent startup using nftclient.New(), save it to nftablesSupported, and added it as an argument to the functions. And I changed the existing LookPath check method to check via nftablesSupported. There were no problems when I ran make golangci and make .linux-test-unit locally, but I don't know if other problems will occur as I lack knowledge. Please review the code, and if you find it strange, I will re-upload a version with only the log removal that the reviewer requested. |
e917f2b to
9008f34
Compare
cmd/antrea-agent/agent.go
Outdated
| var nftablesSupported bool | ||
| if _, err := nftclient.New(networkConfig.IPv4Enabled, networkConfig.IPv6Enabled); err != nil { | ||
| klog.InfoS("nftables is not supported on this Node, skipping nftables-related features", "err", err) | ||
| nftablesSupported = false | ||
| } else { | ||
| klog.InfoS("nftables is supported on this Node") | ||
| nftablesSupported = true | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can just move the logic L338-L346 to function dumpNFTables in dump_others.go, and we don't need the changes of introducing extra parameter nftablesSupported, simplifying the PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review! I reverted the complex changes and moved the check logic directly into 'dumpNFTable's as you suggested.
Now it uses 'nftclient.New' to check for support at the beginning.
I verified that both 'make golangci' and 'make .linux-test-unit' pass locally.
9008f34 to
2c85620
Compare
pkg/support/dump_others.go
Outdated
|
|
||
| func (d *agentDumper) dumpNFTables(basedir string) error { | ||
| if _, err := nftclient.New(d.v4Enabled, d.v6Enabled); err != nil { | ||
| klog.V(4).InfoS("Skipping nftables dump because it is not supported on this Node", "err", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use klog.ErrorS here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the review!
'klog.V(4).InfoS' -> 'ErrorS'
I modified it.
2c85620 to
232b5a3
Compare
hongliangl
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, but you need to resolve the DCO failure in github action.
232b5a3 to
7cd8f1d
Compare
hongliangl
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, defer to @antoninbas and @luolanzone to do final approvals.
|
You have failed unit tests, please correct them first. |
828fa93 to
0a3a75f
Compare
pkg/support/dump_others.go
Outdated
| "strings" | ||
|
|
||
| "k8s.io/klog/v2" | ||
| testingexec "k8s.io/utils/exec/testing" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is never a reason to import this package in a non-test file.
pkg/support/dump_others.go
Outdated
| if _, isFake := d.executor.(*testingexec.FakeExec); !isFake { | ||
| if _, err := nftclient.New(d.v4Enabled, d.v6Enabled); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code is confusing and also doesn't match what was discussed in #7547 (comment)
Ideally we should check for nftables only once, not for every call to dumpNFTables. It was also mentioned that we should comment this code well to explain why we are calling nftclient.New, but not using the returned value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As you advised, we removed the testingexec import. Instead, we separated the kernel inspection called inside dumpNFTables into a variable (var checkNFTablesSupport). And in the test code, we modified this variable to pass the test by mocking it.
0a3a75f to
63d257d
Compare
pkg/support/dump_others.go
Outdated
| var newNFTablesClient = func(v4Enabled, v6Enabled bool) (interface{}, error) { | ||
| return nftclient.New(v4Enabled, v6Enabled) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could we do something like this:
var nftablesIPv4Supported = sync.OnceValue(func() bool {
_, err := newNFTables(knftables.IPv4Family)
return err != nil
})
var nftablesIPv6Supported = sync.OnceValue(func() bool {
_, err := newNFTables(knftables.IPv6Family)
return err != nil
})I am using "sigs.k8s.io/knftables" directly here, instead of "antrea.io/antrea/pkg/agent/util/nftables" which is fine.
This way in dumpNFTables you can use nftablesIPv4Supported() / nftablesIPv6Supported(), and the sync.OnceValue wrapper ensures that we don't check the same thing multiple times over.
You can also easily "mock" them in unit tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review. I've updated the PR based on your feedback.
Signed-off-by: molegit9 <[email protected]>
63d257d to
c1e1cda
Compare
Fixes: #7546
Modification location: pkg/support/dump_others.go
I corrected it because the diagnostic file did not contain the information of 'NFTables'.
' 'pkg/support/dump_others.go' has been added with the function 'dumpNFTables' to allow information to be included in the diagnosis