Skip to content

realtek: dsa: rtl930x: offload ingress cls_flower to PIE - #24994

Open
mab-wien wants to merge 9 commits into
openwrt:mainfrom
mab-wien:realtek-rtl930x-tc-flower
Open

realtek: dsa: rtl930x: offload ingress cls_flower to PIE#24994
mab-wien wants to merge 9 commits into
openwrt:mainfrom
mab-wien:realtek-rtl930x-tc-flower

Conversation

@mab-wien

@mab-wien mab-wien commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Note: the first four preparatory fixes are being reviewed separately in #25024. This branch still contains them for now and will be rebased on top once that PR lands.

Add hardware offload of ingress tc/cls_flower rules on RTL930x DSA
user ports, using the switch's existing Packet Inspection Engine, on top
of the port rate-policing path that is already wired to the RTL93xx
cls_flower ops.

Changes

  • rtldsa_cls_flower_{add,del} route every rule that is not a plain port
    rate limiter to the PIE path; a new rtldsa_cls_flower_stats op
    reports per-rule packet counts.
  • The offload is gated on family_id == RTL9300_FAMILY_ID; RTL931x keeps
    its current behaviour and the RTL83xx SoC code is untouched.
  • The flow-rule parser rejects unsupported dissector keys and partial
    masks up front, and gains generic EtherType matching alongside the
    ARP/IPv4/IPv6 fast paths. pie_rule.ethertype_m is widened to u16;
    rtl930x_pie_verify_template() refuses an EtherType match when the
    active template lacks the field.
  • rtl83xx_validate_flow_actions() rejects nonsensical action
    combinations (multiple forwards; drop mixed with other actions).
  • The rule is bound to the physical ingress port via the source-port
    field. On RTL930x the PIE rule ID is also the LOG counter ID, so the
    packet counter needs no separate allocation.
  • The flow rhashtable is created in rtldsa_93xx_setup() (RTL9300 only)
    and destroyed RCU-safely in rtl83xx_sw_remove() and the probe error
    path.

The first commit is a standalone fix: rtl930x_packet_cntr_clear() wrote
a shared LOG table entry back without reading it first, clobbering the
adjacent counter. It is latent today and becomes reachable once the
offload clears counters at runtime.

Testing

Built for realtek / rtl930x (kernel 6.18); make target/linux/compile
is clean with no new warnings.

Flashed and tested on a Zyxel XGS1210-12:

Rule Result
flower protocol 0x8863 ... action drop (PPPoE discovery) installed, in_hw in_hw_count 1
flower protocol ip ip_proto udp dst_port 67 ... action trap installed, in_hw in_hw_count 1
flower ... action trap on ARP + tc -s filter show 3 real packets counted over 12 s (Sent hardware 0 bytes 3 pkt)
tc filter del + tc qdisc del clsact no residue, no dmesg errors

Note

rtl83xx_setup_tc() (the older flow-block-cb path) has no port_setup_tc
op wired to it and is currently unreachable; it is kept building and now
shares rtl83xx_tc_init().


Discussion / testing feedback: https://forum.openwrt.org/t/tcfilter-luci-app-tcfilter-persistent-tc-ingress-filters-with-hardware-offload-on-realtek-switches/253331

@openwrt openwrt Bot added not following guidelines Pull request does not follow formatting guidelines target/realtek pull request/issue for realtek target labels Sep 2, 2026

@plappermaul plappermaul left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And another surprise out of nowhere. Thanks for the contribution.

Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/dsa.c Outdated
Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/common.c Outdated
Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/tc.c Outdated
@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch from 4020a7a to 71909ac Compare September 2, 2026 19:05
@openwrt openwrt Bot removed the not following guidelines Pull request does not follow formatting guidelines label Sep 2, 2026
@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch from 71909ac to 28a4c14 Compare September 2, 2026 19:18
@mab-wien

mab-wien commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in the latest force-push:

  • family_id gates replaced by a pie_rule_id_is_log_counter bool on struct rtldsa_config.
  • New printouts converted to dev_err()/dev_dbg(priv->dev, ...).
  • New functions renamed to the rtldsa_ prefix (rtldsa_tc_init/cleanup,
    rtldsa_pie_cls_flower_{add,del,stats}, etc.).

@mab-wien
mab-wien requested a review from plappermaul September 2, 2026 19:22

@plappermaul plappermaul left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is too-much-in-one-commit. Start dissecting this ...

Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/tc.c Outdated
Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/rtl-otto.h
Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/rtl930x.c
@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch from 28a4c14 to 0a0ed8d Compare September 2, 2026 19:51
@mab-wien

mab-wien commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Split into logical commits as requested:

  1. rtl930x: don't clobber adjacent LOG counter (was already separate)
  2. rtl930x: bound PIE rule search by the phase block range (block >= max_block)
  3. widen pie_rule.ethertype_m to u16
  4. rtl83xx: give the tc flow hashtable a real lifecycle (rtldsa_tc_init/cleanup)
  5. rtl930x: offload ingress cls_flower to PIE (the feature)

The combined tree is unchanged from the previously built/tested revision.

@mab-wien
mab-wien requested a review from plappermaul September 2, 2026 19:55
@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch from 0a0ed8d to 9d4f409 Compare September 2, 2026 20:11
@mab-wien

mab-wien commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Dissected further into 7 commits:

  1. rtl930x: don't clobber adjacent LOG counter
  2. rtl930x: bound PIE rule search by the phase block range
  3. widen pie_rule.ethertype_m to u16
  4. rtl83xx: set fwd_sel for the tc flower TRAP action (standalone bug fix)
  5. rtl83xx: validate and tighten tc flower match parsing (hardening of the
    existing parser: reject unknown keys/partial masks, fix the ip_proto
    decode, propagate errors)
  6. rtl83xx: give the tc flow hashtable a real lifecycle
  7. rtl930x: offload ingress cls_flower to PIE (the feature itself)

Combined tree unchanged from the built/tested revision.

Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/tc.c
@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch from 9d4f409 to a221fd3 Compare September 2, 2026 20:26
@mab-wien
mab-wien requested a review from plappermaul September 2, 2026 20:27
Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/tc.c
@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch 2 times, most recently from 1322d27 to e1b4036 Compare September 2, 2026 20:44
@mab-wien

mab-wien commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Done. rtldsa_packet_cntr_clear() plus its use in rtldsa_tc_flow_free()
and rtl83xx_delete_flower(), together with the packet_cntr = -1 init, are
now their own pre-commit.

While at it I also moved two stray pr_* -> dev_* conversions into the
commits that already touch those lines, so the final commit is only the
offload itself.

Full series now:

1 rtl930x: don't clobber adjacent LOG counter
2 rtl930x: bound PIE rule search by the phase block range
3 widen pie_rule.ethertype_m to u16
4 rtl83xx: set fwd_sel for the tc flower TRAP action
5 rtl83xx: validate and tighten tc flower match parsing
6 rtl83xx: give the tc flow hashtable a real lifecycle
7 rtl83xx: fix rtl83xx_configure_flower() error unwinding
8 rtl83xx: drop the tc flow hashtable ref before HW teardown
9 rtl83xx: read the tc flow counter outside the RCU lock
10 rtl83xx: clear a flow's LOG counter when it is destroyed
11 rtl930x: offload ingress cls_flower to PIE

1-5 are standalone fixes/hardening; 6-10 give the existing add/del/stats
helpers a proper lifecycle and locking; 11 is the offload (flag + gate,
the rtldsa_pie_cls_flower_* wrappers, cls_flower routing in dsa.c, generic
EtherType matching, source-port binding, PIE-rule-id-as-counter).

The combined tree is unchanged from the revision built for realtek/rtl930x
and tested on a Zyxel XGS1210-12 (EtherType/PPPoE drop, UDP/67 trap, ARP
trap with hardware packet counts via tc -s filter show, clean teardown).

@mab-wien
mab-wien requested a review from plappermaul September 2, 2026 20:46
@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch from e1b4036 to e94446e Compare September 2, 2026 20:51
@openwrt openwrt Bot added GitHub/CI pull requests/issues for GitHub, CI and related stuff not following guidelines Pull request does not follow formatting guidelines labels Sep 2, 2026
@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch from e94446e to 88c1249 Compare September 2, 2026 20:52
@openwrt openwrt Bot removed the not following guidelines Pull request does not follow formatting guidelines label Sep 2, 2026
@mab-wien

mab-wien commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (was 3 commits behind; the 3 new upstream
commits are ath79/mediatek, no realtek changes, no conflicts). No code
changes. Also folded two pr_* -> dev_* one-liners into the commits that
already touch those lines (7 and 9), so commit 11 is purely the offload

@openwrt-ai openwrt-ai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 11 new commits; commit messages match their diffs.


Generated by Claude Code

Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/tc.c
Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/tc.c Outdated
Comment thread target/linux/realtek/files-6.18/drivers/net/dsa/rtl83xx/rtl930x.c Outdated
@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch from 88c1249 to 39884e8 Compare September 3, 2026 07:10
@mab-wien
mab-wien requested a review from openwrt-ai September 6, 2026 23:06
@mab-wien

mab-wien commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main now that #25050 has merged: the two commits carried there (drop the tc flow hashtable ref before HW teardown, verify the MAC template against the mask) drop out as already-applied, leaving 11 commits that replay with no conflicts. git range-diff against the pre-rebase tip is clean (11× =).

Retested on an XGS1210-12 (B1): clean boot on the rebased tree, all four flower match types (0x88e1 / 0x8912 EtherType, IPv4 + IPv6 UDP dst_port) install with in_hw, direct tc filter add/del and the service stop/start cycle behave, and dmesg stays clean (no traces / warnings).

The one open review thread (the tc_initialized re-check) is answered inline. Ready for another look.

@mab-wien
mab-wien force-pushed the realtek-rtl930x-tc-flower branch from 0cc6632 to db99622 Compare September 6, 2026 23:29

@openwrt-ai openwrt-ai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 new commit; no new issues found.


Generated by Claude Code

@plappermaul

Copy link
Copy Markdown
Contributor

Partially reverting patch 1 with patch 10 is bad style. Not so good. Proceed as follows:

@l0rdg3x

l0rdg3x commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

While reading tc.c for #25062 I checked whether its .ndo_setup_tc path can run, and it cannot: rteth_setup_tc() (rtl838x_eth.c:1528) returns -EOPNOTSUPP unless ds->ops->port_setup_tc is set, and nothing in target/linux/realtek sets it. The only two mentions of that op in the whole target are the check and the call inside that function, on main and with this PR applied.

So the series leaves two dispatchers for the same three commands: the live one you add through cls_flower_add/_del/_stats, and rtl83xx_setup_tc_cls_flower(), which this PR keeps current — it now calls rtldsa_configure_flower(), rtldsa_delete_flower() and rtldsa_stats_flower() — but which nothing reaches. Your call what to do with it; a marker comment would at least stop the next reader taking it for a live path.

Related: the !priv->r->packet_cntr_read || !priv->r->packet_cntr_clear guard in rtldsa_configure_flower() covers RTL931x, which has n_counters = 2048 and neither accessor. Without it that would have been a NULL call as soon as the path went live.

Analysis assisted by Claude Opus 5 (Anthropic AI).

mab-wien added a commit to mab-wien/openwrt that referenced this pull request Sep 8, 2026
rtl83xx_parse_flow_rule() accepted whatever it was handed: unknown
dissector keys were ignored, partial EtherType / ip_proto / VLAN masks
were treated as exact, the ip_proto decoding set frame_type_l4 twice for
TCP and never for a UDP-only match, and its return value was dropped with
a "TODO: check error".

Reject what the PIE cannot express before programming it:

  - fail on dissector keys outside the supported set, and on a rule
    combining IPv4 and IPv6 addresses;
  - require a full 0xffff EtherType mask and a full 0xff ip_proto mask,
    decode ip_proto through a switch (adding IGMP), and bail on anything
    else;
  - reject VLAN priority / DEI / ethertype sub-matches, and a VLAN TPID
    other than 802.1Q (cls_flower always sets a full vlan_tpid mask, so
    an unconditional reject would kill the VLAN match arm);
  - add rtldsa_validate_flow_actions() to accept only drop / trap /
    redirect / mirred - FLOW_ACTION_VLAN_PUSH / _POP included in the
    rejection, the ivid / ovid PIE translation in rtl83xx_add_flow() is
    neither programmed nor tested through this offload - honour control
    flags and the hw-stats type, and propagate the parse and action
    errors out of rtl83xx_add_flow().

Unknown EtherTypes are rejected here; generic EtherType matching arrives
with the cls_flower offload series (openwrt#24994), which rebases
on top of this.

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
mab-wien added a commit to mab-wien/openwrt that referenced this pull request Sep 8, 2026
rtl83xx_parse_flow_rule() accepted whatever it was handed: unknown
dissector keys were ignored, partial EtherType / ip_proto / VLAN masks
were treated as exact, the ip_proto decoding set frame_type_l4 twice for
TCP and never for a UDP-only match, and its return value was dropped with
a "TODO: check error".

Reject what the PIE cannot express before programming it:

  - fail on dissector keys outside the supported set, and on a rule
    combining IPv4 and IPv6 addresses;
  - require a full 0xffff EtherType mask and a full 0xff ip_proto mask,
    decode ip_proto through a switch (adding IGMP), and bail on anything
    else;
  - reject VLAN priority / DEI / ethertype sub-matches, and a VLAN TPID
    other than 802.1Q (cls_flower always sets a full vlan_tpid mask, so
    an unconditional reject would kill the VLAN match arm);
  - add rtldsa_validate_flow_actions() to accept only drop / trap /
    redirect / mirred - FLOW_ACTION_VLAN_PUSH / _POP included in the
    rejection, the ivid / ovid PIE translation in rtl83xx_add_flow() is
    neither programmed nor tested through this offload - honour control
    flags and the hw-stats type, and propagate the parse and action
    errors out of rtl83xx_add_flow().

Unknown EtherTypes are rejected here; generic EtherType matching arrives
with the cls_flower offload series (openwrt#24994), which rebases
on top of this.

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
@mab-wien
mab-wien requested a review from openwrt-ai September 8, 2026 15:17
@mab-wien

mab-wien commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main now that #25070 is merged. Its four commits (VID encoding fix, packet counter allocator rework + free helper, configure_flower error unwinding, match/action validation) drop out of this series - patches 1, 3 and 10 were superseded and are gone; the offload commit picks up the rtldsa_packet_cntr_alloc() rename and folds the RTL930x pie-rule-id reservation into the reworked allocator.

Also added your dead-dispatcher point as its own commit ("drop the unreachable tc flower block dispatcher"): rtl83xx_setup_tc() / _block_cb() / cls_flower() are removed, nothing sets ds->ops->port_setup_tc so that whole path was dead; the live path stays the DSA .cls_flower{add,del,stats} ops. rtldsa_tc_init/cleanup() stay.

9 commits, only the five rtl83xx DSA files touched. Builds clean for realtek/rtl930x.

@plappermaul

Copy link
Copy Markdown
Contributor

Next step: Make patch 7/9 and 9/9 a separate PR.

@mab-wien

mab-wien commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Rebased again onto current main - it picked up the realtek: table: refactor (otto_table_read / _otto_table{read,write} replacing rtl_table_read + sw_r32(rtl_table_data())). Only "read a PIE rule's LOG packet counter correctly" conflicted; the counter < RTL930X_PIE_RULE_IDS split is re-expressed in the new API (PIE rule id -> LOG entry at that index, count in word 1; route counter -> halved counter/2 layout). Same table, same index, same word - 1:1 translation. Builds clean for realtek/rtl930x. MERGEABLE now.

@mab-wien

mab-wien commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Done - patch 7/9 (read a PIE rule's LOG packet counter correctly) and 9/9 are now #25076.

9/9 couldn't move as-is: on main rtl83xx_setup_tc_cls_flower() is the only caller of rtl83xx_{configure,delete,stats}flower(), so just dropping the dispatcher orphans them (cascading through add_flow / parse_flow_rule). The whole of tc.c on main is that one never-reachable call graph, so #25076 removes the file outright (tc.o, struct rtl83xx_flow, priv->tc_ht, the decl). This series then adds a working tc.c back through the DSA .cls_flower* ops.

#24994 is 7 commits now (7/9 + 9/9 gone), still builds clean for realtek/rtl930x. Once #25076 lands I'll rebase - tc.c will be a fresh add rather than a diff, so the early lifecycle/locking commits fold into the offload.

@openwrt-ai openwrt-ai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit checks

  • 1be54a5 "realtek: dsa: rtl930x: offload ingress cls_flower to PIE" — after the rebase this commit only adds the EtherType check to rtl930x_pie_verify_template(); the inner-VLAN and L4 source/destination-port checks the third bullet claims, and the dst_port story in the testing paragraph, are already in the base. It also still names the allocator rtl83xx_packet_cntr_alloc(), now rtldsa_packet_cntr_alloc().
  • 5d0f274 "realtek: dsa: rtl83xx: clear a flow's LOG counter when it is destroyed" — the message says it initialises rule.packet_cntr to -1 in rtldsa_configure_flower(), but that init is already in the base and the diff does not touch it; drop that sentence.

Generated by Claude Code

Comment on lines +413 to +417
flow = rhashtable_lookup_fast(&priv->tc_ht, &f->cookie, tc_ht_params);
if (flow) {
pr_info("%s: Got flow\n", __func__);
return -EEXIST;
err = -EEXIST;
goto out_unlock;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tc_ht is keyed on the filter cookie alone, but the caller is now a per-port DSA op that also programs spn. With a shared block (tc qdisc add dev … ingress_block N clsact on two user ports) DSA registers one flow_block_cb per netdev (dsa_user_setup_tc_block(), cb_priv = dev), so the same cookie is offloaded to each port and the second one lands here with -EEXIST — no PIE rule on that port, and a hard failure of the add under skip_sw. Should the hash key be (cookie, ingress port)?


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Real gap. A shared ingress_block across two DSA user ports registers one flow_block_cb per netdev, so the same cookie reaches rtldsa_configure_flower() twice and the second add gets -EEXIST - a hard failure under skip_sw, with no PIE rule on that port. The fix is to key tc_ht on (cookie, ingress_port). I'd rather land that as a follow-up on top of the merged offload than grow this series again; for now a per-port tc qdisc add dev <p> clsact works and the shared-block case fails loudly instead of silently. Will note it as a known limitation.

Comment on lines +442 to +445
if (ingress_port >= 0) {
flow->rule.spn = ingress_port;
flow->rule.spn_m = 0x7f;
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the block dispatcher gone, rtldsa_pie_cls_flower_add() is the only caller and it rejects !ingress, so ingress_port is always a real DSA port and this guard is dead. The same caller also requires pie_rule_id_is_log_counter, which makes the else arm at lines 456-462 (rtldsa_packet_cntr_alloc() + log_data) unreachable too — worth dropping in the same commit, since the "keep it correct if the flow-block-cb path is ever revived" rationale went away with that path.

Suggested change
if (ingress_port >= 0) {
flow->rule.spn = ingress_port;
flow->rule.spn_m = 0x7f;
}
flow->rule.spn = ingress_port;
flow->rule.spn_m = 0x7f;

Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - folded into "drop the unreachable tc flower block dispatcher". With that gone rtldsa_configure_flower() has one caller (rtldsa_pie_cls_flower_add(), RTL930x-only, always a real ingress port), so the ingress_port >= 0 wrapper and the !pie_rule_id_is_log_counter / rtldsa_packet_cntr_alloc() + log_data branch are both removed there (tc.c: -91 lines).

}

static int rtldsa_configure_flower(struct rtl838x_switch_priv *priv,
struct flow_cls_offload *f, int ingress_port)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: patches 1, 2 and 5 each rework rtl83xx_setup_tc() and its helpers — the first_time removal, the rtldsa_ rename, and the ingress_port = -1 call site added for this parameter — and patch 8 then deletes all of it. Sorting that deletion to the front of the series, as was done for the earlier prep patches, removes the churn.


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't sort it to the front here. On main rtl83xx_setup_tc_cls_flower() is the sole caller of rtl83xx_{configure,delete,stats}flower() -> add_flow -> parse_flow_rule, i.e. the whole tc.c call graph; deleting the dispatcher first orphans all of it (-Wunused-function) until patch 5 rewires rtldsa_configure_flower() to the DSA .cls_flower* ops. Moving the deletion up would need patch 5's wiring to move with it - a larger reshuffle than the churn it removes. The first_time / rename / ingress_port touches in patches 1, 2, 5 are each part of turning that path live, not edits to dead code.

@plappermaul

Copy link
Copy Markdown
Contributor

New feature seems to start at patch 5. So make patches 1-4 a separate fix-PR.

@mab-wien

mab-wien commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main now that #25076 is merged. The two counter ops now
take priv; adapted the three call sites (rtldsa_stats_flower, the
rtldsa_packet_cntr_clear wrapper, and the 930x packet_cntr_read/_clear
split) and merged the dev_dbg wording. 9 commits, no functional change; rtl930x
build clean.

@plappermaul

Copy link
Copy Markdown
Contributor

also patch 8 looks very suspicious as it should be part of this pre-Fix series.

The rhashtable that stores offloaded tc flows was initialised lazily from
a "first_time" static in rtl83xx_setup_tc() and never torn down. Add
rtldsa_tc_init() / rtldsa_tc_cleanup() helpers, track initialisation in
the switch private data, set the table up in rtldsa_93xx_setup() and free
it (RCU-safely) from rtl83xx_sw_remove() and the probe error path.

No functional change for existing users; this only moves the table's
init/teardown to the switch lifecycle.

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
The tc cls_flower add / del / stats callbacks reach per-switch state -
the flow hashtable, the PIE rule IDs and the LOG counters - that has no
serialization of its own. Every caller currently runs under rtnl
(dsa_user_setup_tc_block() does not set unlocked_driver_cb, so
tcf_block_bind() bumps block->lockeddevcnt and tc_setup_cb_*() takes
rtnl), so this is not a bug that can be hit today, but nothing in the
driver documents or enforces that. It also leaves
rtldsa_configure_flower() publishing a flow into tc_ht before
pie_rule_add() has assigned its rule ID, where an unlocked delete racing
the add would call pie_rule_rm() with rule.id still 0 and tear down an
unrelated PIE rule.

Add a per-switch tc_flow_lock mutex, set up and torn down alongside the
flow hashtable, and hold it across the whole body of
rtldsa_configure_flower(), rtldsa_delete_flower() and
rtldsa_stats_flower(). This makes the serialization explicit and local
rather than an unstated reliance on the caller, and it is what lets the
next patch drop the hand-rolled rcu_read_lock() sections around the
hashtable lookups. tc_flow_lock is the outermost tc lock - reg_mutex and
pie_mutex are always taken under it, never the other way round.

The three callbacks are reworked here, so also rename them from the
rtl83xx_ to the rtldsa_ prefix to match the surrounding tc flower code.

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
rtldsa_stats_flower() looked the flow up without rcu_read_lock() at all
and then called packet_cntr_read(), which sleeps.

Do the rhashtable lookup in a short RCU section and read the hardware
counter with reg_mutex held once RCU is dropped; tc_flow_lock (held for
the whole callback) keeps the flow alive across the read. Use u32 for the
packet counters, return -ENOENT when the flow is gone, switch the debug
print to dev_dbg(), and report the real packet count instead of a
synthetic byte estimate.

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
A flow that carries a LOG-table packet counter left that counter with its
stale count behind when the flow was removed, so the next flow assigned
the same counter id started from a bogus baseline.

Add rtldsa_packet_cntr_clear() and zero the counter from both flow
teardown paths (rtldsa_tc_flow_free() and rtldsa_delete_flower()); the
helper skips the clear while no counter is assigned (rule.packet_cntr
already starts at -1).

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
The add, delete and stats cls_flower callbacks all run under
tc_flow_lock, but rtldsa_tc_cleanup() walked and destroyed tc_ht
without it and then destroyed the mutex. A callback that raced teardown
could therefore walk a flow that rtldsa_tc_flow_free() is freeing, or
call pie_rule_rm() on a rule this path already removed.

Take tc_flow_lock around rhashtable_free_and_destroy() and clear
tc_initialized under it, so any in-flight callback completes before the
table is torn down. Drop the lock before rcu_barrier() / mutex_destroy().

A callback that was already blocked on tc_flow_lock when teardown ran
would still wake into a freed tc_ht, so the three callbacks re-check
tc_initialized right after they take the lock and return -ENOENT when
it is clear.

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
Offload ingress tc-flower rules on RTL930x DSA user ports to the Packet
Inspection Engine, on top of the port rate-policing path already wired to
the RTL93xx cls_flower ops.

  - rtldsa_cls_flower_{add,del} route every rule that is not a plain port
    rate limiter to rtldsa_pie_cls_flower_*(), and a new
    rtldsa_cls_flower_stats op exposes per-rule packet counts.
  - The offload is gated on a new rtldsa_config flag
    (pie_rule_id_is_log_counter), set for RTL930x only, so RTL931x keeps
    its current behaviour and the RTL83xx SoC code is untouched.
  - Generic EtherType matches (frame_type 1) are added on top of the
    ARP/IPv4/IPv6 fast paths, and rtl930x_pie_verify_template() gains an
    EtherType arm so a rule that needs TEMPLATE_FIELD_ETHERTYPE skips a
    block that lacks it instead of matching on a stale field.
  - The rule is bound to the physical ingress port via the source-port
    field. On RTL930x the PIE rule ID is also the LOG counter ID, so no
    separate counter is allocated; it is cleared when the rule is added.
    rtldsa_packet_cntr_alloc() keeps the PIE rule ID range reserved on
    these SoCs so an L3 route counter cannot alias a flow's LOG entry.

Tested on a Zyxel XGS1210-12: ingress flower rules matching EtherType
(PPPoE discovery), ARP, and a UDP destination port install with in_hw
set, "tc -s filter show" reports hardware packet counts, and removing the
rules leaves no residue.

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
rtl930x_packet_cntr_read() and rtl930x_packet_cntr_clear() always
addressed the LOG table as counter/2, with the low bit selecting one of
the two 32-bit words in the 64-bit entry. That layout only applies to
the L3 route statistics counters that rtldsa_packet_cntr_alloc() hands
out above the PIE rule ID range (base = n_pie_blocks * PIE_BLOCK_SIZE).

A PIE rule instead logs its matched-packet count into the LOG table
entry that carries its own rule ID, in data word 1. Because the old code
folded the rule ID in half, only rule ID 0 read a sane value; every
other offloaded tc-flower rule reported a bogus or zero "tc -s filter"
hardware packet count, and clearing one rule's counter disturbed the
neighbouring rule.

Split the two cases on the rule ID / route counter boundary: address PIE
rule counters directly at entry = id, word 1, and keep the paired
counter/2 read-modify-write for the route counters.

Verified on an RTL9302C (Zyxel XGS1210-12) with several concurrent
skip_sw flower rules: each now reports its own increasing packet count.

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
…llbacks

With tc_flow_lock serializing rtldsa_configure_flower(),
rtldsa_delete_flower() and rtldsa_stats_flower(), and a flow only ever
freed (via kfree_rcu) from rtldsa_delete_flower() / rtldsa_tc_cleanup(),
a looked-up flow cannot be freed while a callback holds tc_flow_lock. The
hand-rolled rcu_read_lock() sections around the rhashtable lookups are
therefore redundant:

  - rhashtable_lookup_fast() / _insert_fast() / _remove_fast() already
    take rcu_read_lock() internally;
  - nothing dereferences a looked-up flow outside tc_flow_lock.

Switch configure_flower() from rhashtable_lookup() to the _fast variant
and drop the explicit rcu_read_lock() / rcu_read_unlock() in all three
callbacks. No functional change.

Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
rtl83xx_setup_tc() and its helpers rtl83xx_setup_tc_block_cb() /
rtl83xx_setup_tc_cls_flower() form a TC_SETUP_BLOCK dispatcher meant to
be reached through .ndo_setup_tc -> rteth_setup_tc() -> ds->ops->
port_setup_tc(). Nothing in target/linux/realtek ever sets
port_setup_tc, and rtl83xx_setup_tc() is assigned to no ops struct
either, so rteth_setup_tc() returns -EOPNOTSUPP and this whole path is
dead.

The cls_flower offload runs entirely through the DSA .cls_flower_add /
_del / _stats switch ops, i.e. rtldsa_pie_cls_flower_{add,del,stats}()
-> rtldsa_{configure,delete,stats}_flower(). Keeping the second,
unreachable dispatcher only invites a reader to mistake it for a live
path.

Remove it. With it gone rtldsa_configure_flower() has a single caller,
rtldsa_pie_cls_flower_add(), which is RTL930x-only and always passes a
real ingress port, so the ingress_port >= 0 guard and the
!pie_rule_id_is_log_counter branch (the rtldsa_packet_cntr_alloc() /
log_data path) are dead too - drop them. rtldsa_tc_init() /
rtldsa_tc_cleanup() stay; they run from the probe path.

Reported-by: Gennaro Cimmino <gcimmino@rayonra.net>
Assisted-by: Claude Code (Anthropic Claude Sonnet 5)
Signed-off-by: Mark Abe <github@mab.wien>
@mab-wien

mab-wien commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Split as requested: patches 1-4 and "hold tc_flow_lock in rtldsa_tc_cleanup()"
are now #25094. This PR keeps only the offload feature - 4 commits:

  • offload ingress cls_flower to PIE
  • read a PIE rule's LOG packet counter correctly (rtl930x)
  • drop redundant rcu_read_lock() in tc flower callbacks
  • drop the unreachable tc flower block dispatcher

Rebases cleanly on #25094; content unchanged from the tested 9-commit stack
(range-diff clean, only the commit split differs). Will rebase onto main once
#25094 lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

GitHub/CI pull requests/issues for GitHub, CI and related stuff target/realtek pull request/issue for realtek target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants