Title not aligned with correct column in "show mux config" output

[wangxin]

The titles are not aligned with correct column in table output of "show mux config" command.

{
    "changed": true,
    "stdout": "SWITCH_NAME    PEER_TOR\n-------------  ----------\nvlab-06        10.1.0.33\n                                                      port    state    ipv4    ipv6\n----------  ----  ---------------  -----------------  ------  -------  ------  --------\nEthernet4   auto  192.168.0.2/32   fc02:1000::2/128                            software\nEthernet8   auto  192.168.0.3/32   fc02:1000::3/128                            software\nEthernet12  auto  192.168.0.4/32   fc02:1000::4/128                            software\nEthernet16  auto  192.168.0.5/32   fc02:1000::5/128                            software\nEthernet20  auto  192.168.0.6/32   fc02:1000::6/128                            software\nEthernet24  auto  192.168.0.7/32   fc02:1000::7/128                            software\nEthernet28  auto  192.168.0.8/32   fc02:1000::8/128                            software\nEthernet32  auto  192.168.0.9/32   fc02:1000::9/128                            software\nEthernet36  auto  192.168.0.10/32  fc02:1000::a/128                            software\nEthernet40  auto  192.168.0.11/32  fc02:1000::b/128                            software\nEthernet44  auto  192.168.0.12/32  fc02:1000::c/128                            software\nEthernet48  auto  192.168.0.13/32  fc02:1000::d/128                            software\nEthernet52  auto  192.168.0.14/32  fc02:1000::e/128                            software\nEthernet56  auto  192.168.0.15/32  fc02:1000::f/128                            software\nEthernet60  auto  192.168.0.16/32  fc02:1000::10/128                           software\nEthernet64  auto  192.168.0.17/32  fc02:1000::11/128                           software\nEthernet68  auto  192.168.0.18/32  fc02:1000::12/128                           software\nEthernet72  auto  192.168.0.19/32  fc02:1000::13/128                           software\nEthernet76  auto  192.168.0.20/32  fc02:1000::14/128                           software\nEthernet80  auto  192.168.0.21/32  fc02:1000::15/128                           software\nEthernet84  auto  192.168.0.22/32  fc02:1000::16/128                           software\nEthernet88  auto  192.168.0.23/32  fc02:1000::17/128                           software\nEthernet92  auto  192.168.0.24/32  fc02:1000::18/128                           software\nEthernet96  auto  192.168.0.25/32  fc02:1000::19/128                           software",
    "stderr": "",
    "rc": 0,
    "cmd": "show mux config",
    "start": "2025-07-16 05:40:28.923461",
    "end": "2025-07-16 05:40:29.234765",
    "delta": "0:00:00.311304",
    "msg": "",
    "invocation": {
        "module_args": {
            "_raw_params": "show mux config",
            "_uses_shell": true,
            "warn": false,
            "stdin_add_newline": true,
            "strip_empty_ends": true,
            "argv": null,
            "chdir": null,
            "executable": null,
            "creates": null,
            "removes": null,
            "stdin": null
        }
    },
    "stdout_lines": [
        "SWITCH_NAME    PEER_TOR",
        "-------------  ----------",
        "vlab-06        10.1.0.33",
        "                                                      port    state    ipv4    ipv6",
        "----------  ----  ---------------  -----------------  ------  -------  ------  --------",
        "Ethernet4   auto  192.168.0.2/32   fc02:1000::2/128                            software",
        "Ethernet8   auto  192.168.0.3/32   fc02:1000::3/128                            software",
        "Ethernet12  auto  192.168.0.4/32   fc02:1000::4/128                            software",
        "Ethernet16  auto  192.168.0.5/32   fc02:1000::5/128                            software",
        "Ethernet20  auto  192.168.0.6/32   fc02:1000::6/128                            software",
        "Ethernet24  auto  192.168.0.7/32   fc02:1000::7/128                            software",
        "Ethernet28  auto  192.168.0.8/32   fc02:1000::8/128                            software",
        "Ethernet32  auto  192.168.0.9/32   fc02:1000::9/128                            software",
        "Ethernet36  auto  192.168.0.10/32  fc02:1000::a/128                            software",
        "Ethernet40  auto  192.168.0.11/32  fc02:1000::b/128                            software",
        "Ethernet44  auto  192.168.0.12/32  fc02:1000::c/128                            software",
        "Ethernet48  auto  192.168.0.13/32  fc02:1000::d/128                            software",
        "Ethernet52  auto  192.168.0.14/32  fc02:1000::e/128                            software",
        "Ethernet56  auto  192.168.0.15/32  fc02:1000::f/128                            software",
        "Ethernet60  auto  192.168.0.16/32  fc02:1000::10/128                           software",
        "Ethernet64  auto  192.168.0.17/32  fc02:1000::11/128                           software",
        "Ethernet68  auto  192.168.0.18/32  fc02:1000::12/128                           software",
        "Ethernet72  auto  192.168.0.19/32  fc02:1000::13/128                           software",
        "Ethernet76  auto  192.168.0.20/32  fc02:1000::14/128                           software",
        "Ethernet80  auto  192.168.0.21/32  fc02:1000::15/128                           software",
        "Ethernet84  auto  192.168.0.22/32  fc02:1000::16/128                           software",
        "Ethernet88  auto  192.168.0.23/32  fc02:1000::17/128                           software",
        "Ethernet92  auto  192.168.0.24/32  fc02:1000::18/128                           software",
        "Ethernet96  auto  192.168.0.25/32  fc02:1000::19/128                           software"
    ],
    "stderr_lines": [],
    "_ansible_no_log": null,
    "failed": false
}

The consequence is that the parsed result of "show mux config" is incorrect. The "test_pretest.py" script always fails on dualtor testbed.

        duts_mux_config = duthosts.show_and_parse("show mux config", start_line_index=3)
        upper_tor_mux_config = duts_mux_config[dut_upper_tor.hostname]
        lower_tor_mux_config = duts_mux_config[dut_lower_tor.hostname]
        if upper_tor_mux_config != lower_tor_mux_config:
            err_msg = "'show mux config' output differs between two ToRs {} v.s. {}"\
                .format(upper_tor_mux_config, lower_tor_mux_config)
            return False, err_msg, {}
    
        port_cable_types = {}
    
        for row in upper_tor_mux_config:
            port_name = row["port"]
>           port_idx = str(duts_minigraph_facts[dut_upper_tor.hostname][0][1]['minigraph_port_indices'][port_name])
E           KeyError: ''


_check_mux_status_helper = <function _check_dut_mux_status.<locals>._check_mux_status_helper at 0x7a38fb842040>
_verify_inconsistent_mux_status = <function _check_dut_mux_status.<locals>._verify_inconsistent_mux_status at 0x7a3908077430>
_verify_show_mux_status = <function _check_dut_mux_status.<locals>._verify_show_mux_status at 0x7a38fafc0c10>
dut_lower_tor = <MultiAsicSonicHost vlab-06>
dut_upper_tor = <MultiAsicSonicHost vlab-05>
duthosts   = [<MultiAsicSonicHost vlab-05>, <MultiAsicSonicHost vlab-06>]
duts_minigraph_facts = {'vlab-05': [(0, {'deployment_id': '1', 'dhcp_servers': ['192.0.0.1', '192.0.0.2', '192.0.0.3', '192.0.0.4'], 'dhcpv6_...rs': ['fc02:2000::1', 'fc02:2000::2', 'fc02:2000::3', 'fc02:2000::4'], 'forced_mgmt_routes': ['172.17.0.1/24'], ...})]}
duts_mux_config = {'vlab-05': [{'': 'fc02:1000::2/128', 'ipv4': '', 'ipv6': 'software', 'port': '', ...}, {'': 'fc02:1000::3/128', 'ipv4... 'ipv6': 'software', 'port': '', ...}, {'': 'fc02:1000::7/128', 'ipv4': '', 'ipv6': 'software', 'port': '', ...}, ...]}
kwargs     = {'stage': 'stage_pre_test'}
lower_tor_mux_config = [{'': 'fc02:1000::2/128', 'ipv4': '', 'ipv6': 'software', 'port': '', ...}, {'': 'fc02:1000::3/128', 'ipv4': '', 'ipv6..., 'ipv6': 'software', 'port': '', ...}, {'': 'fc02:1000::7/128', 'ipv4': '', 'ipv6': 'software', 'port': '', ...}, ...]
port_cable_types = {}
port_name  = ''
row        = {'': 'fc02:1000::2/128', 'ipv4': '', 'ipv6': 'software', 'port': '', ...}
upper_tor_mux_config = [{'': 'fc02:1000::2/128', 'ipv4': '', 'ipv6': 'software', 'port': '', ...}, {'': 'fc02:1000::3/128', 'ipv4': '', 'ipv6..., 'ipv6': 'software', 'port': '', ...}, {'': 'fc02:1000::7/128', 'ipv4': '', 'ipv6': 'software', 'port': '', ...}, ...]

Full error log: https://elastictest.org/scheduler/publictestplan/687732ac0e60738d0bec029c?testcase=test_pretest.py%7C%7C%7Cvms-kvm-dual-t0_12938&type=log

Version:

    {
        "cmd": "show version",
        "cmd_with_timeout": "",
        "err_msg": "",
        "rc": 0,
        "stdout_lines": [
            "",
            "SONiC Software Version: SONiC.master-22686.901184-40e4dcef9",
            "SONiC OS Version: 12",
            "Distribution: Debian 12.11",
            "Kernel: 6.1.0-29-2-amd64",
            "Build commit: 40e4dcef9",
            "Build date: Wed Jul 16 04:21:20 UTC 2025",
            "Built by: azureuser@e842f359c000000",

Because of this issue (and some other issues), the sonic-utilities submodule advance PR was stuck for almost 2 months: sonic-net/sonic-buildimage#22686

It's highly possible that the issue was introduced by this PR: #3884

[zjswhhh]

@harjotsinghpawra - can you check and prioritize fix for this?

[harjotsinghpawra]

@zjswhhh Sure jing looking into it

[wangxin]

@harjotsinghpawra The sonic-utilities submodule advance PR is blocked. I am going to merge the reverting PR. Please submit a new PR after you fixed the issue. Thanks!

[harjotsinghpawra]

@wangxin @zjswhhh Hi,

I tested in mock test of utilites it was passing because one of the ports had soc_ipv4 .
the problem in script is about soc_ipv4 field .Out of all mux enteries if even one of them has a soc_ipv4 then we get output in this format . But if soc_ipv4 is not given in any port it automatically assumes its in active-standby which according to me is wrong assumption . Now in code based on the soc_ipv4 we detect the format if all headers need to be displayed or subset . Which again raises new question why such assumption was not made for soc_ipv6 .

port        state    ipv4      ipv6      cable_type      soc_ipv4    soc_ipv6    prober_type
----------  -------  --------  --------  --------------  ----------  ----------  -------------
Ethernet0   active   10.2.1.1  e800::46                                          software
Ethernet4   auto     10.3.1.1  e801::46                              e801::47    software
Ethernet8   active   10.4.1.1  e802::46                                          hardware
Ethernet12  active   10.4.1.1  e802::46                                          software
Ethernet16  standby  10.1.1.1  fc00::75  active-standby                          software
Ethernet28  manual   10.1.1.1  fc00::75                                          software
Ethernet32  auto     10.1.1.1  fc00::75  active-active   10.1.1.2    fc00::76    software

We have following formats and check

    if soc_ipv4_value is not None:
        port_list.append(soc_ipv4_value)
        is_dualtor_active_active[0] = True

     if is_dualtor_active_active[0]:
         headers = ['port', 'state', 'ipv4', 'ipv6', 'cable_type', 'soc_ipv4', 'soc_ipv6', 'prober_type']
      else:
         headers = ['port', 'state', 'ipv4', 'ipv6']

I can modify the code to only show headers based on the fields in config_db rather than hardcoded headers. or we can keep just one format. But why did we used two different formats to display same command output . Is it for some legacy infra support ? Do we need in future ? can we finalize one output format?

Before i make changes further if you have some context can you please help me answer these ?

[zjswhhh]

@harjotsinghpawra - soc IPs were introduced with active-active dualtor. For active-standby, we weren't using it. That was why.

I can modify the code to only show headers based on the fields in config_db rather than hardcoded headers. or we can keep just one format.

I don't have a preference on this. But we should be careful and shouldn't break any existing mgmt tests. If you want to change to one standard format, please evaluate the sonic-mgmt tests. Please also double check the json format.