opal/mca/if: posix_ipv4 ignores if_base_retain_loopback (#if 0 since 2010); loopback enumeration differs across Linux/macOS vs the BSDs

[jsquyres]

Summary

The opal/mca/if framework enumerates the IPv4 loopback interface (127.0.0.1) inconsistently across platforms, and the MCA parameter that is supposed to control this (if_base_retain_loopback) is silently ignored on Linux and macOS.

• On the BSDs (FreeBSD/NetBSD/OpenBSD/DragonFly), the IPv4 component bsdx_ipv4 honors if_base_retain_loopback (default false) and drops the loopback interface from opal_if_list.
• On Linux and macOS, the IPv4 component posix_ipv4 has the equivalent loopback check #if 0'd out, so it always keeps loopback regardless of the parameter.

Net effect: for an identical Open MPI configuration, 127.0.0.1/lo is in opal_if_list on Linux/macOS but absent on the BSDs.

This was found while adding an opal_if unit test in #13976: the test failed only on the FreeBSD CI job because 127.0.0.1/lo0 was not enumerated there. (That PR works around it test-side by setting OMPI_MCA_if_base_retain_loopback=1; this issue is about the underlying production behavior.)

Scope: this is IPv4-only. IPv6 loopback (::1) is already dropped on all platforms — linux_ipv6 keeps only global-scope (0x00) addresses and so skips ::1, and bsdx_ipv6 drops it via the same retain_loopback check. So the only live disparity is the IPv4 127.0.0.1 case described here.

Where the disparity lives

if_base_retain_loopback is registered (default false) in
opal/mca/if/base/if_base_components.c (~L50-55), declared in opal/mca/if/if.h.

The IPv4 components diverge:

• opal/mca/if/posix_ipv4/if_posix.c (~L197-201) — used on Linux and macOS — loopback filter compiled out:

  #if 0
          if (!opal_if_retain_loopback && (ifr->ifr_flags & IFF_LOOPBACK) != 0) {
              continue;
          }
  #endif

• opal/mca/if/bsdx_ipv4/if_bsdx.c (~L96-99) — used on the BSDs — filter is live:

  /* skip interface if it is a loopback device (IFF_LOOPBACK set) */
  if (!opal_if_retain_loopback && 0 != (cur_ifaddrs->ifa_flags & IFF_LOOPBACK)) {
      continue;
  }

History

The #if 0 in posix_ipv4 is not recent. It was carried into the framework when the interface code was reorganized in 2010 (commit e96b5f486f, "Reorganize the opal interface code in opal/util/if.c ... Move the interface discovery code into a framework"), i.e. it predates the framework split and has been inert ever since. A 2010 follow-up (c74ce1632a) even renamed the variable inside the already-#if 0'd block, so the dead code has been maintained but never re-enabled. So: ~16 years of posix_ipv4 ignoring if_base_retain_loopback.

Who actually consumes opal_if_* (blast radius)

A whole-tree audit of the opal_if* API shows the impact is far narrower than it looks:

• The TCP BTL (opal/mca/btl/tcp/) is the only production consumer of the opal_if_* API in the entire Open MPI source tree.
• ompi/ and oshmem/ contain zero references to opal_if_* (not even opal_if_list/opal_if_t). The MPI/OpenSHMEM layers never enumerate interfaces directly.
• The TCP BTL only calls the enumeration/index/name/address functions (opal_ifbegin, opal_ifnext, opal_ifcount, opal_ifindextoaddr, opal_ifindextoname, opal_ifindextokindex, opal_ifkindextoname, opal_ifnametokindex). The contract-sensitive query functions have zero production callers: opal_ifislocal, opal_ifaddrtoname, and opal_ifisloopback are referenced only by the new unit test and by a stale contrib/build-mca-comps-outside-of-tree/btl_tcp2_* sample (an out-of-tree-build teaching copy of an older TCP BTL).
• The bundled 3rd-party/ libraries are not consumers: they ship their own interface code (e.g. pmix_if_* in 3rd-party/openpmix/src/util/pmix_if.{c,h}), independent of OPAL's copy.

Consequence of the blast-radius finding: because opal_ifislocal() & friends have no live callers, and the one real consumer (TCP BTL) self-filters loopback (see below), the contract violation has no current runtime impact in Open MPI — it is dormant. That should weigh heavily in deciding how aggressively to fix it.

Observable consequences

1. if_base_retain_loopback does not work on Linux/macOS. Setting if_base_retain_loopback=0 (its documented default) does not remove loopback there, because posix_ipv4 ignores the knob. The parameter only actually does anything on the BSDs.

2. opal_ifislocal()/opal_ifaddrtoname() give different answers across platforms — a documented-contract violation (but dormant). opal/util/if.h documents opal_ifislocal() as returning "true if hostname is local." Because these lookups are backed by opal_if_list:

   • Linux/macOS: opal_ifislocal("127.0.0.1") / opal_ifislocal("localhost") return true.
   • BSD (default config): they return false — even though 127.0.0.1/localhost are unquestionably local.

   This is the same class of localhost-correctness bug as opal_net_islocalhost() (fixed in Add OPAL class/util/datatype unit tests #13976), reached via the interface list rather than address math. It is dormant only because nothing in production calls these functions today.

3. Internal interface indices differ. opal_if_t::if_index is assigned by enumeration order, so on Linux/macOS loopback occupies an index (shifting every other interface's OPAL index relative to the BSDs), and opal_ifcount() differs by one for the same host.

Consequences for the TCP BTL (the lone consumer)

The TCP BTL iterates opal_if_list (mca_btl_tcp_component_create_instances() in opal/mca/btl/tcp/btl_tcp_component.c, ~L840-950) and ships a default btl_tcp_if_exclude = "127.0.0.1/8,sppp" (~L308). That default exclude exists essentially to compensate for this inconsistency — it is what keeps loopback out of the BTL set on Linux/macOS, where loopback is present in opal_if_list. With stock settings both platform families therefore end up not using loopback for TCP, so the bug is masked in the common case. It leaks through once the defaults are touched:

• Overriding btl_tcp_if_exclude without re-listing 127.0.0.1/8 is a cross-platform footgun. E.g. btl_tcp_if_exclude=ib0 to steer TCP off IPoIB: on Linux/macOS, loopback is in opal_if_list, no longer covered by the (replaced) exclude, so the create loop (~L927-949) builds a TCP BTL module for lo/127.0.0.1. The node then advertises a 127.0.0.1 endpoint in its modex; peers on other hosts see a 127.0.0.0/8 address, opal_net_samenetwork() matches it against their own loopback, and connections go to the local machine instead of the intended peer — spurious connect failures/retries/timeouts during wire-up. (The btl_tcp.c ~L95 "Do not create loopback TCP connections" guard only blocks a proc connecting to itself; it does not filter the loopback interface.) On the BSDs the identical setting is harmless, since loopback was never in opal_if_list.
• btl_tcp_if_include referencing loopback differs: btl_tcp_if_include=lo resolves via opal_ifnametokindex() (~L903-913) and builds a loopback BTL on Linux/macOS, while on the BSDs opal_ifnametokindex("lo0") fails and the TCP BTL aborts init with "Unknown interface name".
• Spurious diagnostics on the BSDs: the default btl_tcp_if_exclude=127.0.0.1/8 resolves against opal_if_list; on the BSDs nothing matches it (loopback absent), which can emit "Did not find interface matching this subnet" when btl_tcp_report_all_unfound_interfaces is enabled — for a default setting.

Options and consequences

"Loopback in list" = whether 127.0.0.1 is in opal_if_list by default (Linux/macOS · BSD).

#	Option	Loopback in list (Lin/mac · BSD)	Consistent?	opal_ifislocal("127.0.0.1")	Param honored on all OSes?	Risk
—	Current state	in · out	no	true · false	no (posix ignores)	—
A	Document only (no code change)	in · out	no	true · false	no	none
B	Remove the #if 0 in posix_ipv4; keep default false	out · out	yes	false · false ¹	yes	med
C	Default → true (leave #if 0)	in · in	default only ²	true · true	no	med
D	Remove #if 0 + default → true	in · in	all values	true · true	yes	med
E	Special-case 127/8+::1 as local in opal_ifislocal/opal_ifaddrtoname	in · out (unchanged)	enum: no ³	true · true	n/a	low
F	B + E (drop loopback everywhere and fix query fns)	out · out	yes	true · true	yes	med
G	Harden the TCP BTL to filter loopback internally	in · out (unchanged)	enum: no ³	true · false (unchanged)	n/a	low

¹ B makes opal_ifislocal("127.0.0.1") consistently false — consistent, but the documented "is local" contract is then uniformly violated (still dormant: no production callers).
² C is consistent only at the default value. Set retain_loopback=0 and the split returns, because posix_ipv4 still ignores the parameter.
³ E/G leave interface enumeration platform-dependent, but that no longer affects the locality contract (E) and is already tolerated by the TCP BTL's default exclude.

Cross-cutting consequences:

• Footgun (override of btl_tcp_if_exclude): B, F eliminate it on all platforms (loopback never in the list); C, D spread it to the BSDs (they would now include loopback by default); G removes it everywhere without touching the framework.
• B/F side effect: with loopback gone from the list on Linux/macOS too, the TCP BTL's default 127.0.0.1/8 exclude matches nothing → the "Did not find interface matching this subnet" diagnostic (today BSD-only) could fire on all platforms → that default exclude should be dropped/adjusted as part of the change.
• C/D semantics flip: changes the 16-year-old default meaning of retain_loopback from "drop" to "retain"; anything assuming loopback is excluded by default gets it back.
• Not tabulated: a larger cleanup — consolidating posix_ipv4 + bsdx_ipv4 (both getifaddrs-based) into one component that honors the parameter — would subsume B/D but is a bigger, riskier diff.

Recommendation

• Lowest-risk correctness fix: option E — special-case 127.0.0.0/8 and ::1 in opal_ifislocal()/opal_ifaddrtoname() so locality is correct on every platform, with no change to enumeration or interface selection (and therefore no TCP BTL impact). This directly fixes the documented contract.
• If retain_loopback should also actually work, with full enumeration consistency: option F (= B + E) — accept the Linux/macOS enumeration change and adjust the TCP BTL's default exclude.
• Avoid C alone (leaves the parameter broken and spreads the footgun to the BSDs).
• G is a worthwhile orthogonal hardening if the override-btl_tcp_if_exclude footgun is a concern, regardless of which of A/E/F is chosen.
• Given the dormancy (no production callers; TCP BTL self-filters), A (document) or E are the proportionate responses; a disruptive change (C/D/F) is not justified by urgency.

References

• Found via the opal_if unit test added in Add OPAL class/util/datatype unit tests #13976 (which works around this test-side via OMPI_MCA_if_base_retain_loopback=1).
• Files: opal/mca/if/{posix_ipv4,bsdx_ipv4,bsdx_ipv6,linux_ipv6,base}, opal/util/if.{h,c}, opal/mca/btl/tcp/btl_tcp_component.c, opal/mca/btl/tcp/btl_tcp.c.