X410 SPI mode: using more than 2 peripherals simultaneously

[inets-rwth]

We are currently working on a testbed that involves a USRP X410 receiving a signal over all the four different channels in GNURadio. The X410 and UHD enables control of external components (transceivers in our case) via SPI and the HDMI front-panel connectors. For that, one has to set the pins on the desired GPIO port and configure the data direction. According to the manual page for the X410 (link), up to 4 peripherals are supported. A vector of peripheral configurations has to be passed to the spi_iface_getter.

Our aim was to connect 2 transceivers to GPIO0 (left hand side HDMI connector) and the other two transceivers to GPIO1 (right hand side HDMI connector). Each of the transceiver uses 4 pins, so in total we want to use 16 out of the 24 pins from both connectors (eight on each GPIO0/GPIO1). We were trying to initialize all 4 transceivers with the following line:

spi_ref = spi_getter_iface.get_spi_ref(periph_cfgs)

periph_cfgs is a vector of the four peripheral configurations. However, this returns an error as we are only allowed to push two peripheral configurations (two of our transceivers) into periph_cfgs. The get_spi_ref only accepts a vector of size 1/2. In each vector entry, we define the CLK/SDI/SDO/CS pins.

A peripheral configuration perip_cfg has been created as such:

spi_pconf periph_cfg;
periph_cfg.periph_clk = SPI_CLK_PIN_0;
periph_cfg.periph_sdi = SPI_SDI_PIN_0;
periph_cfg.periph_sdo = SPI_SDO_PIN_0;
periph_cfg.periph_cs  = SPI_CS_PIN_0;

periph_cfgs.push_back(periph_cfg);

To circumvent this error our idea was to initialize two different spi_getter_iface objects, each with two peripheral configurations. We attempted to get the spi_getter_iface_0 and get the spi_getter_iface_1 objects from different radio controllers with the following lines:

spi_getter_iface_0 = usrp->get_radio_control(0).get_feature<spi_getter_iface>
spi_ref_0 = spi_getter_iface_0.get_spi_ref(periph_cfgs_0)

spi_getter_iface_1 = usrp->get_radio_control(1).get_feature<spi_getter_iface>
spi_ref_1 = spi_getter_iface_1.get_spi_ref(periph_cfgs_1)

However, our SPI commands are only executed on the spi_ref_1, not on spi_ref_0. If we reverse the order of the commands of spi_ref_0/spi_ref_1, it's the other way around, spi_ref_0 gets executed, spi_ref_1 not. Thus, it seems like only the "last" command is executed.

We have also verified, that both HDMI ports can be used (that they are not physically turned off), by having one configuration for GPIO0 and one for GPOI1 in the periph_cfgs_1 (the one executed last and thus working). This also works but the other two configs in periph_cfgs_0 are still not (as before with calling both spi_getter_iface_0/spi_getter_iface_1.

Is there any way to use multiple (more than 2) configurations via both GPIO0/GPIO1 at the same time?

[twilsonSCT]

Summary:
I am seeing it as well, on UHD 4.10 specifically.

It looks like commits f49dce2 61c48b7 48b76f4 did some significant rework of x400 SPI & GPIO. Did it break the multi-SPI interface capabilities?

Details:
I can confirm that we have been using a similar approach (two SPI ports per get_spi_ref call, 4 in total). This was working up until UHD 4.9 but I am currently investigating an issue where it is not working on UHD 4.10. The symptom we have is that only the first two of the SPI ports that are configured and used are working and the two that are configured second are dead, confirmed with logic analyzer. Are you also seeing this on 4.10 only?

Here is a normal 4.8 initialization from our system, where a GPIO reset is used to reset the connected peripheral.

And then here is a 4.10 plot of the offending channel. It gets an extra reset pulse (from where?), and the SPI lines are inactive.

Debug info:

Slot A:

[DEBUG] [sc2430] Available MB GPIO Bank: GPIO0
[DEBUG] [sc2430] Available MB GPIO Bank: GPIO1
[DEBUG] [sc2430] Available MB GPIO Source: PS
[DEBUG] [sc2430] Available MB GPIO Source: MPM
[DEBUG] [sc2430] Available MB GPIO Source: USER_APP
[DEBUG] [sc2430] Available MB GPIO Source: DB0_RF0
[DEBUG] [sc2430] Available MB GPIO Source: DB0_RF1
[DEBUG] [sc2430] Available MB GPIO Source: DB0_SPI
[DEBUG] [sc2430] Available MB GPIO Source: DB1_RF0
[DEBUG] [sc2430] Available MB GPIO Source: DB1_RF1
[DEBUG] [sc2430] Available MB GPIO Source: DB1_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 0 Value: DB0_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 1 Value: DB0_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 2 Value: DB0_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 3 Value: DB0_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 4 Value: DB0_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 5 Value: DB0_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 6 Value: DB0_RF0
[DEBUG] [sc2430] Current MB GPIO Source: 7 Value: DB0_RF1
[DEBUG] [sc2430] Current MB GPIO Source: 8 Value: DB0_RF0
[DEBUG] [sc2430] Current MB GPIO Source: 9 Value: DB0_RF0
[DEBUG] [sc2430] Current MB GPIO Source: 10 Value: DB0_RF0
[DEBUG] [sc2430] Current MB GPIO Source: 11 Value: DB0_RF0
[DEBUG] [sc2430] Available Radio GPIO Bank: GPIO
[DEBUG] [sc2430] Available Radio GPIO Attribute: CTRL Value: 0xc0
[DEBUG] [sc2430] Available Radio GPIO Attribute: DDR Value: 0xbf9
[DEBUG] [sc2430] Available Radio GPIO Attribute: OUT Value: 0x0
[DEBUG] [sc2430] Available Radio GPIO Attribute: ATR_0X Value: 0x0
[DEBUG] [sc2430] Available Radio GPIO Attribute: ATR_RX Value: 0x0
[DEBUG] [sc2430] Available Radio GPIO Attribute: ATR_TX Value: 0xc0
[DEBUG] [sc2430] Available Radio GPIO Attribute: ATR_XX Value: 0xc0
[DEBUG] [sc2430] Available Radio GPIO Attribute: READBACK Value: 0x800000
[DEBUG] [sc2430] Read Slot Detect 0x2020
[DEBUG] [sc2430] Read PID 0x901
[DEBUG] [sc2430] Wrote scratch: 0xb0f2
[DEBUG] [sc2430] Read scratch: 0xb0f2
[DEBUG] [sc2430] HW Version: 1.0
[DEBUG] [sc2430] FW Version: 1.26
[DEBUG] [sc2430] Initialized Slot A

Slot B:

[DEBUG] [sc2430] Available MB GPIO Bank: GPIO0
[DEBUG] [sc2430] Available MB GPIO Bank: GPIO1
[DEBUG] [sc2430] Available MB GPIO Source: PS
[DEBUG] [sc2430] Available MB GPIO Source: MPM
[DEBUG] [sc2430] Available MB GPIO Source: USER_APP
[DEBUG] [sc2430] Available MB GPIO Source: DB0_RF0
[DEBUG] [sc2430] Available MB GPIO Source: DB0_RF1
[DEBUG] [sc2430] Available MB GPIO Source: DB0_SPI
[DEBUG] [sc2430] Available MB GPIO Source: DB1_RF0
[DEBUG] [sc2430] Available MB GPIO Source: DB1_RF1
[DEBUG] [sc2430] Available MB GPIO Source: DB1_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 0 Value: DB1_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 1 Value: DB1_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 2 Value: DB1_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 3 Value: DB1_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 4 Value: DB1_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 5 Value: DB1_SPI
[DEBUG] [sc2430] Current MB GPIO Source: 6 Value: DB1_RF0
[DEBUG] [sc2430] Current MB GPIO Source: 7 Value: DB1_RF1
[DEBUG] [sc2430] Current MB GPIO Source: 8 Value: DB1_RF0
[DEBUG] [sc2430] Current MB GPIO Source: 9 Value: DB1_RF0
[DEBUG] [sc2430] Current MB GPIO Source: 10 Value: DB1_RF0
[DEBUG] [sc2430] Current MB GPIO Source: 11 Value: DB1_RF0
[DEBUG] [sc2430] Available Radio GPIO Bank: GPIO
[DEBUG] [sc2430] Available Radio GPIO Attribute: CTRL Value: 0xc0000
[DEBUG] [sc2430] Available Radio GPIO Attribute: DDR Value: 0xbf9bf9
[DEBUG] [sc2430] Available Radio GPIO Attribute: OUT Value: 0x0
[DEBUG] [sc2430] Available Radio GPIO Attribute: ATR_0X Value: 0x0
[DEBUG] [sc2430] Available Radio GPIO Attribute: ATR_RX Value: 0x0
[DEBUG] [sc2430] Available Radio GPIO Attribute: ATR_TX Value: 0xc0000
[DEBUG] [sc2430] Available Radio GPIO Attribute: ATR_XX Value: 0xc0000
[DEBUG] [sc2430] Available Radio GPIO Attribute: READBACK Value: 0x0
[DEBUG] [sc2430] Read Slot Detect 0x0
[DEBUG] [sc2430] Read PID 0x0

[manderseck]

Thanks for reporting the issue @inets-rwth and thanks for confirming it @twilsonSCT. I'm working on a fix, it will be available in UHD 4.11 for sure, a bit earlier in this repo. Will report back once I have it.

[manderseck]

From your descriptions I read that either only the first configuration works or the second. I'm trying to reproduce the issue and for this used the spi example which I modified a bit to use the second GPIO port, too (see attached). I connected the clock pin to a scope to see if my SPI transaction is working at all and I can confirm that with this I'm able to see the clock whenever the SPI transaction is called. I will still verify this against UHD <4.10 to see the differences described. Do you maybe have a small example that you can share, too? Thanks!

#include <uhd/features/spi_getter_iface.hpp>
#include <uhd/usrp/multi_usrp.hpp>
#include <uhd/utils/safe_main.hpp>
#include <stdlib.h>
#include <boost/program_options.hpp>
#include <iostream>

static const size_t SPI_DEFAULT_CLK_PIN        = 0;
static const size_t SPI_DEFAULT_SDI_PIN        = 1;
static const size_t SPI_DEFAULT_SDO_PIN        = 2;
static const size_t SPI_DEFAULT_CS_PIN         = 3;
static const size_t SPI_DEFAULT_PAYLOAD_LENGTH = 32;
static const std::string SPI_DEFAULT_PAYLOAD   = "0xfefe";
static const size_t SPI_DEFAULT_CLK_DIVIDER    = 4;
static const size_t SPI_GPIO1_SPI_INDEX_OFFSET = 16;
static const size_t SPI_GPIO1_DDR_INDEX_OFFSET = 12;

namespace po = boost::program_options;

int UHD_SAFE_MAIN(int argc, char* argv[])
{
    // variables to be set by po
    std::string args;
    size_t clk;
    size_t sdi;
    size_t sdo;
    size_t cs;
    size_t payload_length;
    size_t clk_divider;
    std::string payload_str;
    uint32_t payload;

    // setup the program options
    po::options_description desc("Allowed options");
    // clang-format off
    desc.add_options()
        ("help", "help message")
        ("args", po::value<std::string>(&args)->default_value(""), "multi uhd device address args")
        ("list-banks", "print list of banks before running tests")
        ("clk", po::value<size_t>(&clk)->default_value(SPI_DEFAULT_CLK_PIN), "number of pin for SPI clock")
        ("sdo", po::value<size_t>(&sdo)->default_value(SPI_DEFAULT_SDO_PIN), "number of pin for serial data out")
        ("sdi", po::value<size_t>(&sdi)->default_value(SPI_DEFAULT_SDI_PIN), "number of pin for serial data in")
        ("cs", po::value<size_t>(&cs)->default_value(SPI_DEFAULT_CS_PIN), "number of pin for chip select")
        ("payload", po::value<std::string>(&payload_str)->default_value(SPI_DEFAULT_PAYLOAD), "payload as integer value")
        ("length", po::value<size_t>(&payload_length)->default_value(SPI_DEFAULT_PAYLOAD_LENGTH), "payload length in bits")
        ("clk-div", po::value<size_t>(&clk_divider)->default_value(SPI_DEFAULT_CLK_DIVIDER), "clock divider for SPI")
    ;
    // clang-format on
    po::variables_map vm;
    po::store(po::parse_command_line(argc, argv, desc), vm);
    po::notify(vm);

    // print the help message
    if (vm.count("help")) {
        std::cout << argv[0] << " " << desc << std::endl;
        return ~0;
    }

    // create a usrp device
    std::cout << std::endl;
    std::cout << "Creating the usrp device with: " << args << "..." << std::endl;
    auto usrp = uhd::usrp::multi_usrp::make(args);

    if (vm.count("list-banks")) {
        std::cout << "Available GPIO banks: " << std::endl;
        auto banks = usrp->get_gpio_banks(0);
        for (auto& bank : banks) {
            std::cout << "* " << bank << std::endl;
        }
    }

    // Get the SPI getter interface from where we'll get the SPI interface itself
    if (!usrp->get_radio_control().has_feature<uhd::features::spi_getter_iface>()) {
        std::cout << "Error: Could not find SPI_Getter_Iface. Please check if your FPGA "
                     "image is up to date.\n";
        return EXIT_FAILURE;
    }
    auto& spi_getter_iface =
        usrp->get_radio_control().get_feature<uhd::features::spi_getter_iface>();

    // FIXME: Ran on X420, so for any other X4xx, get_radio_control needs a channel number from the second radio here
    auto& spi_getter_iface1 =
        usrp->get_radio_control(1).get_feature<uhd::features::spi_getter_iface>();

    // Set all available pins to SPI for GPIO0 and GPIO1
    std::vector<std::string> sources(12, "DB0_SPI");
    std::vector<std::string> sources1(12, "DB1_SPI");
    usrp->set_gpio_src("GPIO0", sources);
    usrp->set_gpio_src("GPIO1", sources1);

    // Create peripheral configuration per peripheral
    uhd::features::spi_periph_config_t periph_cfg;
    periph_cfg.periph_clk = clk;
    periph_cfg.periph_sdi = sdi;
    periph_cfg.periph_sdo = sdo;
    periph_cfg.periph_cs  = cs;

    uhd::features::spi_periph_config_t periph_cfg1;
    // GPIO1 uses a different index space for the SPI engine than for DDR bits.
    periph_cfg1.periph_clk = clk + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg1.periph_sdi = sdi + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg1.periph_sdo = sdo + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg1.periph_cs  = cs + SPI_GPIO1_SPI_INDEX_OFFSET;

    // The vector holds the peripheral configs with index=peripheral number
    std::vector<uhd::features::spi_periph_config_t> periph_cfgs;
    periph_cfgs.push_back(periph_cfg);
    std::vector<uhd::features::spi_periph_config_t> periph_cfgs1;
    periph_cfgs1.push_back(periph_cfg1);

    // Set the data direction register
    uint32_t outputs = 0x0;
    outputs |= 1 << periph_cfg.periph_clk;
    outputs |= 1 << periph_cfg.periph_sdo;
    outputs |= 1 << periph_cfg.periph_cs;
    uint32_t outputs1 = 0x0;
    outputs1 |= 1 << (clk + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (sdo + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (cs + SPI_GPIO1_DDR_INDEX_OFFSET);
    usrp->set_gpio_attr("GPIOA", "DDR", outputs, outputs);
    usrp->set_gpio_attr("GPIOB", "DDR", outputs1, outputs1);
    auto spi_ref  = spi_getter_iface.get_spi_ref(periph_cfgs);
    auto spi_ref1 = spi_getter_iface1.get_spi_ref(periph_cfgs1);

    std::cout << "Using pins: " << std::endl
              << "  Clock = " << (int)(periph_cfg.periph_clk) << std::endl
              << "  SDO   = " << (int)(periph_cfg.periph_sdo) << std::endl
              << "  SDI   = " << (int)(periph_cfg.periph_sdi) << std::endl
              << "  CS    = " << (int)(periph_cfg.periph_cs) << std::endl
              << std::endl;

    payload = strtoul(payload_str.c_str(), NULL, 0);
    std::cout << "Writing payload: 0x" << std::hex << payload << " with length "
              << std::dec << payload_length << " bits" << std::endl;
    // The spi_config_t holds items like the clock divider and the SDI and SDO edges
    uhd::spi_config_t config;
    config.divider            = clk_divider;
    config.use_custom_divider = true;
    config.mosi_edge          = config.EDGE_RISE;
    config.miso_edge          = config.EDGE_FALL;

    // Do the SPI transaction. There are write() and read() methods available, too.
    std::cout << "Performing SPI transaction..." << std::endl;
    uint32_t read_data = spi_ref->transact_spi(0, config, payload, payload_length, true);
    uint32_t read_data1 =
        spi_ref1->transact_spi(0, config, payload, payload_length, true);
    std::cout << "Data read: 0x" << std::hex << read_data << std::endl;

    return EXIT_SUCCESS;
}

[twilsonSCT]

From your descriptions I read that either only the first configuration works or the second. I'm trying to reproduce the issue and for this used the spi example which I modified a bit to use the second GPIO port, too (see attached). I connected the clock pin to a scope to see if my SPI transaction is working at all and I can confirm that with this I'm able to see the clock whenever the SPI transaction is called. I will still verify this against UHD <4.10 to see the differences described. Do you maybe have a small example that you can share, too? Thanks!

#include <uhd/features/spi_getter_iface.hpp>
#include <uhd/usrp/multi_usrp.hpp>
#include <uhd/utils/safe_main.hpp>
#include <stdlib.h>
#include <boost/program_options.hpp>
#include <iostream>

static const size_t SPI_DEFAULT_CLK_PIN        = 0;
static const size_t SPI_DEFAULT_SDI_PIN        = 1;
static const size_t SPI_DEFAULT_SDO_PIN        = 2;
static const size_t SPI_DEFAULT_CS_PIN         = 3;
static const size_t SPI_DEFAULT_PAYLOAD_LENGTH = 32;
static const std::string SPI_DEFAULT_PAYLOAD   = "0xfefe";
static const size_t SPI_DEFAULT_CLK_DIVIDER    = 4;
static const size_t SPI_GPIO1_SPI_INDEX_OFFSET = 16;
static const size_t SPI_GPIO1_DDR_INDEX_OFFSET = 12;

namespace po = boost::program_options;

int UHD_SAFE_MAIN(int argc, char* argv[])
{
    // variables to be set by po
    std::string args;
    size_t clk;
    size_t sdi;
    size_t sdo;
    size_t cs;
    size_t payload_length;
    size_t clk_divider;
    std::string payload_str;
    uint32_t payload;

    // setup the program options
    po::options_description desc("Allowed options");
    // clang-format off
    desc.add_options()
        ("help", "help message")
        ("args", po::value<std::string>(&args)->default_value(""), "multi uhd device address args")
        ("list-banks", "print list of banks before running tests")
        ("clk", po::value<size_t>(&clk)->default_value(SPI_DEFAULT_CLK_PIN), "number of pin for SPI clock")
        ("sdo", po::value<size_t>(&sdo)->default_value(SPI_DEFAULT_SDO_PIN), "number of pin for serial data out")
        ("sdi", po::value<size_t>(&sdi)->default_value(SPI_DEFAULT_SDI_PIN), "number of pin for serial data in")
        ("cs", po::value<size_t>(&cs)->default_value(SPI_DEFAULT_CS_PIN), "number of pin for chip select")
        ("payload", po::value<std::string>(&payload_str)->default_value(SPI_DEFAULT_PAYLOAD), "payload as integer value")
        ("length", po::value<size_t>(&payload_length)->default_value(SPI_DEFAULT_PAYLOAD_LENGTH), "payload length in bits")
        ("clk-div", po::value<size_t>(&clk_divider)->default_value(SPI_DEFAULT_CLK_DIVIDER), "clock divider for SPI")
    ;
    // clang-format on
    po::variables_map vm;
    po::store(po::parse_command_line(argc, argv, desc), vm);
    po::notify(vm);

    // print the help message
    if (vm.count("help")) {
        std::cout << argv[0] << " " << desc << std::endl;
        return ~0;
    }

    // create a usrp device
    std::cout << std::endl;
    std::cout << "Creating the usrp device with: " << args << "..." << std::endl;
    auto usrp = uhd::usrp::multi_usrp::make(args);

    if (vm.count("list-banks")) {
        std::cout << "Available GPIO banks: " << std::endl;
        auto banks = usrp->get_gpio_banks(0);
        for (auto& bank : banks) {
            std::cout << "* " << bank << std::endl;
        }
    }

    // Get the SPI getter interface from where we'll get the SPI interface itself
    if (!usrp->get_radio_control().has_feature<uhd::features::spi_getter_iface>()) {
        std::cout << "Error: Could not find SPI_Getter_Iface. Please check if your FPGA "
                     "image is up to date.\n";
        return EXIT_FAILURE;
    }
    auto& spi_getter_iface =
        usrp->get_radio_control().get_feature<uhd::features::spi_getter_iface>();

    // FIXME: Ran on X420, so for any other X4xx, get_radio_control needs a channel number from the second radio here
    auto& spi_getter_iface1 =
        usrp->get_radio_control(1).get_feature<uhd::features::spi_getter_iface>();

    // Set all available pins to SPI for GPIO0 and GPIO1
    std::vector<std::string> sources(12, "DB0_SPI");
    std::vector<std::string> sources1(12, "DB1_SPI");
    usrp->set_gpio_src("GPIO0", sources);
    usrp->set_gpio_src("GPIO1", sources1);

    // Create peripheral configuration per peripheral
    uhd::features::spi_periph_config_t periph_cfg;
    periph_cfg.periph_clk = clk;
    periph_cfg.periph_sdi = sdi;
    periph_cfg.periph_sdo = sdo;
    periph_cfg.periph_cs  = cs;

    uhd::features::spi_periph_config_t periph_cfg1;
    // GPIO1 uses a different index space for the SPI engine than for DDR bits.
    periph_cfg1.periph_clk = clk + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg1.periph_sdi = sdi + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg1.periph_sdo = sdo + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg1.periph_cs  = cs + SPI_GPIO1_SPI_INDEX_OFFSET;

    // The vector holds the peripheral configs with index=peripheral number
    std::vector<uhd::features::spi_periph_config_t> periph_cfgs;
    periph_cfgs.push_back(periph_cfg);
    std::vector<uhd::features::spi_periph_config_t> periph_cfgs1;
    periph_cfgs1.push_back(periph_cfg1);

    // Set the data direction register
    uint32_t outputs = 0x0;
    outputs |= 1 << periph_cfg.periph_clk;
    outputs |= 1 << periph_cfg.periph_sdo;
    outputs |= 1 << periph_cfg.periph_cs;
    uint32_t outputs1 = 0x0;
    outputs1 |= 1 << (clk + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (sdo + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (cs + SPI_GPIO1_DDR_INDEX_OFFSET);
    usrp->set_gpio_attr("GPIOA", "DDR", outputs, outputs);
    usrp->set_gpio_attr("GPIOB", "DDR", outputs1, outputs1);
    auto spi_ref  = spi_getter_iface.get_spi_ref(periph_cfgs);
    auto spi_ref1 = spi_getter_iface1.get_spi_ref(periph_cfgs1);

    std::cout << "Using pins: " << std::endl
              << "  Clock = " << (int)(periph_cfg.periph_clk) << std::endl
              << "  SDO   = " << (int)(periph_cfg.periph_sdo) << std::endl
              << "  SDI   = " << (int)(periph_cfg.periph_sdi) << std::endl
              << "  CS    = " << (int)(periph_cfg.periph_cs) << std::endl
              << std::endl;

    payload = strtoul(payload_str.c_str(), NULL, 0);
    std::cout << "Writing payload: 0x" << std::hex << payload << " with length "
              << std::dec << payload_length << " bits" << std::endl;
    // The spi_config_t holds items like the clock divider and the SDI and SDO edges
    uhd::spi_config_t config;
    config.divider            = clk_divider;
    config.use_custom_divider = true;
    config.mosi_edge          = config.EDGE_RISE;
    config.miso_edge          = config.EDGE_FALL;

    // Do the SPI transaction. There are write() and read() methods available, too.
    std::cout << "Performing SPI transaction..." << std::endl;
    uint32_t read_data = spi_ref->transact_spi(0, config, payload, payload_length, true);
    uint32_t read_data1 =
        spi_ref1->transact_spi(0, config, payload, payload_length, true);
    std::cout << "Data read: 0x" << std::hex << read_data << std::endl;

    return EXIT_SUCCESS;
}

Thank you for your time investigating this @manderseck.

May I suggest the same test but with four periph_cfg (two for GPIO0, two for GPIO1)? I am able to get it to work with only two periph_cfg but it becomes an issue for at 4 specifically.

[manderseck]

Thanks @twilsonSCT for your feedback. Based on that I extended the spi example a bit more (see below). In this example I left all pins the same for all configs except for the chip select which got an offset of 1, so I can distinguish them. I tested it both with UHD 4.9 and 4.10 and I could indeed find a difference: The SPI index offset seems to have changed. In 4.9, the pins at the second connector had an offset of 12 while for some reason from 4.10 on it is an offset of 16. When configuring the DDR with set_gpio_attr() however, the offset is still 12. So we definitely have an issue here, but I'm not entirely sure that this is the issue you are seeing. Can you please check if this is indeed "your" issue or if there is more, so we can find the right fix? Thanks!

#include <uhd/features/spi_getter_iface.hpp>
#include <uhd/usrp/multi_usrp.hpp>
#include <uhd/utils/safe_main.hpp>
#include <stdlib.h>
#include <boost/program_options.hpp>
#include <iostream>

static const size_t SPI_DEFAULT_CLK_PIN        = 0;
static const size_t SPI_DEFAULT_SDI_PIN        = 1;
static const size_t SPI_DEFAULT_SDO_PIN        = 2;
static const size_t SPI_DEFAULT_CS_PIN         = 4;
static const size_t SPI_DEFAULT_PAYLOAD_LENGTH = 32;
static const std::string SPI_DEFAULT_PAYLOAD   = "0xfefe";
static const size_t SPI_DEFAULT_CLK_DIVIDER    = 4;
static const size_t SPI_GPIO1_SPI_INDEX_OFFSET = 16; // was 12 in UHD < v4.10
static const size_t SPI_GPIO1_DDR_INDEX_OFFSET = 12;

namespace po = boost::program_options;

int UHD_SAFE_MAIN(int argc, char* argv[])
{
    // variables to be set by po
    std::string args;
    size_t clk;
    size_t sdi;
    size_t sdo;
    size_t cs;
    size_t payload_length;
    size_t clk_divider;
    std::string payload_str;
    uint32_t payload;

    // setup the program options
    po::options_description desc("Allowed options");
    // clang-format off
    desc.add_options()
        ("help", "help message")
        ("args", po::value<std::string>(&args)->default_value(""), "multi uhd device address args")
        ("list-banks", "print list of banks before running tests")
        ("clk", po::value<size_t>(&clk)->default_value(SPI_DEFAULT_CLK_PIN), "number of pin for SPI clock")
        ("sdo", po::value<size_t>(&sdo)->default_value(SPI_DEFAULT_SDO_PIN), "number of pin for serial data out")
        ("sdi", po::value<size_t>(&sdi)->default_value(SPI_DEFAULT_SDI_PIN), "number of pin for serial data in")
        ("cs", po::value<size_t>(&cs)->default_value(SPI_DEFAULT_CS_PIN), "number of pin for chip select")
        ("payload", po::value<std::string>(&payload_str)->default_value(SPI_DEFAULT_PAYLOAD), "payload as integer value")
        ("length", po::value<size_t>(&payload_length)->default_value(SPI_DEFAULT_PAYLOAD_LENGTH), "payload length in bits")
        ("clk-div", po::value<size_t>(&clk_divider)->default_value(SPI_DEFAULT_CLK_DIVIDER), "clock divider for SPI")
    ;
    // clang-format on
    po::variables_map vm;
    po::store(po::parse_command_line(argc, argv, desc), vm);
    po::notify(vm);

    // print the help message
    if (vm.count("help")) {
        std::cout << argv[0] << " " << desc << std::endl;
        return ~0;
    }

    // create a usrp device
    std::cout << std::endl;
    std::cout << "Creating the usrp device with: " << args << "..." << std::endl;
    auto usrp = uhd::usrp::multi_usrp::make(args);

    if (vm.count("list-banks")) {
        std::cout << "Available GPIO banks: " << std::endl;
        auto banks = usrp->get_gpio_banks(0);
        for (auto& bank : banks) {
            std::cout << "* " << bank << std::endl;
        }
    }

    // Get the SPI getter interface from where we'll get the SPI interface itself
    if (!usrp->get_radio_control().has_feature<uhd::features::spi_getter_iface>()) {
        std::cout << "Error: Could not find SPI_Getter_Iface. Please check if your FPGA "
                     "image is up to date.\n";
        return EXIT_FAILURE;
    }
    auto& spi_getter_iface =
        usrp->get_radio_control().get_feature<uhd::features::spi_getter_iface>();

    auto& spi_getter_iface1 = usrp->get_radio_control(usrp->get_rx_num_channels() - 1)
                                  .get_feature<uhd::features::spi_getter_iface>();

    // Set all available pins to SPI for GPIO0 and GPIO1
    std::vector<std::string> sources(12, "DB0_SPI");
    std::vector<std::string> sources1(12, "DB1_SPI");
    usrp->set_gpio_src("GPIO0", sources);
    usrp->set_gpio_src("GPIO1", sources1);


    // Create peripheral configuration per peripheral
    uhd::features::spi_periph_config_t periph_cfg0;
    periph_cfg0.periph_clk = clk;
    periph_cfg0.periph_sdi = sdi;
    periph_cfg0.periph_sdo = sdo;
    periph_cfg0.periph_cs  = cs;

    uhd::features::spi_periph_config_t periph_cfg1;
    periph_cfg1.periph_clk = clk;
    periph_cfg1.periph_sdi = sdi;
    periph_cfg1.periph_sdo = sdo;
    periph_cfg1.periph_cs  = cs + 1;

    uhd::features::spi_periph_config_t periph_cfg2;
    // GPIO1 uses a different index space for the SPI engine than for DDR bits.
    periph_cfg2.periph_clk = clk + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg2.periph_sdi = sdi + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg2.periph_sdo = sdo + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg2.periph_cs  = cs + SPI_GPIO1_SPI_INDEX_OFFSET;

    uhd::features::spi_periph_config_t periph_cfg3;
    periph_cfg3.periph_clk = clk + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg3.periph_sdi = sdi + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg3.periph_sdo = sdo + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg3.periph_cs  = cs + SPI_GPIO1_SPI_INDEX_OFFSET + 1;

    // The vector holds the peripheral configs with index=peripheral number
    std::vector<uhd::features::spi_periph_config_t> periph_cfgs;
    periph_cfgs.push_back(periph_cfg0);
    periph_cfgs.push_back(periph_cfg1);
    std::vector<uhd::features::spi_periph_config_t> periph_cfgs1;
    periph_cfgs1.push_back(periph_cfg2);
    periph_cfgs1.push_back(periph_cfg3);

    // Set the data direction register
    uint32_t outputs = 0x0;
    outputs |= 1 << periph_cfg0.periph_clk;
    outputs |= 1 << periph_cfg0.periph_sdo;
    outputs |= 1 << periph_cfg0.periph_cs;
    outputs |= 1 << periph_cfg1.periph_cs;

    uint32_t outputs1 = 0x0;
    outputs1 |= 1 << (clk + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (sdo + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (cs + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (cs + SPI_GPIO1_DDR_INDEX_OFFSET + 1);

    usrp->set_gpio_attr("GPIOB", "DDR", outputs, outputs);
    usrp->set_gpio_attr("GPIOA", "DDR", outputs1, outputs1);
    auto spi_ref  = spi_getter_iface.get_spi_ref(periph_cfgs);
    auto spi_ref1 = spi_getter_iface1.get_spi_ref(periph_cfgs1);

    std::cout << "Using pins: " << std::endl
              << "  Clock = " << (int)(periph_cfg0.periph_clk) << std::endl
              << "  SDO   = " << (int)(periph_cfg0.periph_sdo) << std::endl
              << "  SDI   = " << (int)(periph_cfg0.periph_sdi) << std::endl
              << "  CS    = " << (int)(periph_cfg0.periph_cs) << std::endl
              << std::endl;

    payload = strtoul(payload_str.c_str(), NULL, 0);
    std::cout << "Writing payload: 0x" << std::hex << payload << " with length "
              << std::dec << payload_length << " bits" << std::endl;
    // The spi_config_t holds items like the clock divider and the SDI and SDO edges
    uhd::spi_config_t config;
    config.divider            = clk_divider;
    config.use_custom_divider = true;
    config.mosi_edge          = config.EDGE_RISE;
    config.miso_edge          = config.EDGE_FALL;

    // Do the SPI transaction. There are write() and read() methods available, too.
    std::cout << "Performing SPI transaction..." << std::endl;
    uint32_t read_data = spi_ref->transact_spi(0, config, payload, payload_length, true);
    spi_ref->transact_spi(1, config, payload, payload_length, true);
    spi_ref1->transact_spi(0, config, payload, payload_length, true);
    spi_ref1->transact_spi(1, config, payload, payload_length, true);
    std::cout << "Data read: 0x" << std::hex << read_data << std::endl;

    return EXIT_SUCCESS;
}

[inets-rwth]

Hello @twilsonSCT , hello @manderseck,

Thanks for looking into that issue. With our initial test we were still running on UHD 4.6. For some validation, we also modified the SPI example as shown below. Some weeks ago, this example lead to the behavior above (only peripherals at spi_ref_1 were working). Weirdly enough, today suddenly all four peripherals were giving good feedback on our SPI command and we are not exactly sure why. Instead, when we tried to use daughterboard 1 today as the GPIO source by setting DB1_SPI for the set_gpio_src() command, it did not work. This daughterboard also didn't work for us when using only one peripheral but maybe that's another issue unrelated to what we described before.

Please let us know if you want us to test on newer UHD versions.

//
// Copyright 2022 Ettus Research, a National Instruments Brand
//
// SPDX-License-Identifier: GPL-3.0-or-later
//

// Example for SPI testing.
//
// This example shows how to work with SPI which is based on the GPIO
// interface of the X410.

// Modified by Berk Acikgoez to act as a script to check if all four antenna arrays are changing beams, 04.09.2025.

#include <uhd/features/spi_getter_iface.hpp>
#include <uhd/features/gpio_power_iface.hpp>
#include <uhd/usrp/multi_usrp.hpp>
//#include <uhd/usrp/subdev_spec.hpp>
#include <uhd/utils/safe_main.hpp>
#include <stdlib.h>
#include <iostream>

static const size_t SPI_CLK_DIVIDER           = 24;
static const std::string USRP_DEVICE_ARGS     = "addr=192.168.100.45";
static const int SPI_PERIPH_NO                = 4;

std::string to_bit_string(uint32_t val, const size_t num_bits)
{
    std::string out;
    for (int i = num_bits - 1; i >= 0; i--) {
        std::string bit = ((val >> i) & 1) ? "1" : "0";
        out += "  ";
        out += bit;
    }
    return out;
}

uint32_t SPI_RW_WRAPPER(uint32_t payload, uint8_t payload_length, uhd::spi_config_t config, uhd::spi_iface::sptr & spi_ref, uint8_t periph_id) {
    std::cout << "Writing payload: 0x" << std::hex << payload << " with length "
              << std::dec << int(payload_length) << " bits" << std::endl;
              
    uint32_t read_data = spi_ref->transact_spi(periph_id, config, payload, payload_length, true);
    std::cout << "Data read: 0x" << std::hex << read_data << std::endl;
    
    return read_data;
}

int UHD_SAFE_MAIN(int argc, char* argv[])
{  
    /********************* set the USRP up ******************************/
    std::cout << "Creating the first USRP device with argument: " << USRP_DEVICE_ARGS << std::endl;
    auto usrp = uhd::usrp::multi_usrp::make(USRP_DEVICE_ARGS);
    
    // Get the SPI getter interface from where we'll get the SPI interface itself
    if (!usrp->get_radio_control().has_feature<uhd::features::spi_getter_iface>()) {
        std::cout << "Error: Could not find SPI_Getter_Iface. Please check if your FPGA "
                     "image is up to date.\n";
        return EXIT_FAILURE;
    }
    
    // set GPIO pin sources
    std::vector<std::string> sources0{"DB0_SPI", "DB0_SPI", "DB0_SPI", "DB0_SPI", 
                                     "DB0_SPI", "DB0_SPI", "DB0_SPI", "DB0_SPI", 
                                     "DB0_SPI", "DB0_SPI", "DB0_SPI", "DB0_SPI"};
    //std::vector<std::string> sources1{"DB1_SPI", "DB1_SPI", "DB1_SPI", "DB1_SPI", 
    //                                 "DB1_SPI", "DB1_SPI", "DB1_SPI", "DB1_SPI", 
    //                                 "DB1_SPI", "DB1_SPI", "DB1_SPI", "DB1_SPI"};
    usrp->set_gpio_src("GPIO0", sources0);
    usrp->set_gpio_src("GPIO1", sources0);
            
    // set GPIO logic voltage level (3V3)
    auto & gpio_handle = usrp->get_mb_controller().get_feature<uhd::features::gpio_power_iface>();
    gpio_handle.set_port_voltage("GPIO0", "3V3");
    gpio_handle.set_port_voltage("GPIO1", "3V3");
          
    // change these for different ports/pins              
    std::vector<int> SPI_CLK_list = {0, 11, 0, 11};
    std::vector<int> SPI_SDI_list = {2, 9, 2, 9};
    std::vector<int> SPI_SDO_list = {1, 10, 1, 10};
    std::vector<int> SPI_CS_list = {3, 7, 3, 7};
    std::vector<std::string> SPI_GPIO_list = {"GPIO0", "GPIO0", "GPIO1", "GPIO1"};
            
    uint32_t outputs = 0x0;
    uhd::features::spi_periph_config_t periph_cfg;
    std::vector<uhd::features::spi_periph_config_t> periph_cfgs_0, periph_cfgs_1;
        
    for (int i = 0; i < SPI_PERIPH_NO; i++) {
        // add 12 to all pin numbers if GPIO1 is used
        int shift_if_GPIO1;
        if (SPI_GPIO_list[i] == "GPIO1") {
            shift_if_GPIO1 = 12;
        } else {
            shift_if_GPIO1 = 0;
        }
            
        // Create peripheral configuration per peripherals
        periph_cfg.periph_clk = SPI_CLK_list[i] + shift_if_GPIO1;
        periph_cfg.periph_sdi = SPI_SDI_list[i] + shift_if_GPIO1;
        periph_cfg.periph_sdo = SPI_SDO_list[i] + shift_if_GPIO1;
        periph_cfg.periph_cs  = SPI_CS_list[i] + shift_if_GPIO1;
                
        outputs |= 1 << periph_cfg.periph_clk;
        outputs |= 1 << periph_cfg.periph_sdo;
        outputs |= 1 << periph_cfg.periph_cs;
        
        if (i < 2)        
            periph_cfgs_0.push_back(periph_cfg);
        else
            periph_cfgs_1.push_back(periph_cfg);
        std::cout << "Peripheral " << i << " created and successfully pushed back." << std::endl;
    }
        
    // Set the data direction register
    usrp->set_gpio_attr("GPIOA", "DDR", outputs & 0xFFFFFF);
            
    // set control register
    usrp->set_gpio_attr("GPIOA", "CTRL", 0, 0xFFFFFF);
            
    // get first SPI controller
    uhd::spi_iface::sptr spi_ref_0, spi_ref_1;
    auto & spi_get_iface_0 = usrp->get_radio_control(0).get_feature<uhd::features::spi_getter_iface>();
    spi_ref_0 = spi_get_iface_0.get_spi_ref(periph_cfgs_0);
    
    if (SPI_PERIPH_NO > 2) {
        // get second SPI controller
        auto & spi_get_iface_1 = usrp->get_radio_control(1).get_feature<uhd::features::spi_getter_iface>();
        spi_ref_1 = spi_get_iface_1.get_spi_ref(periph_cfgs_1);
    }
      
    /********************* END - set the USRP up ******************************/  
          
    // The spi_config_t holds items like the clock divider and the SDI and SDO edges
    uhd::spi_config_t config;
    config.divider            = SPI_CLK_DIVIDER;
    config.use_custom_divider = true;
    config.mosi_edge          = config.EDGE_RISE;
    config.miso_edge          = config.EDGE_RISE;
    
    // Some example payloads:
    //
    // read bf_rx_awv_ptr: 0b0000 1110 0001 0100 0000 0000 -> 0x0e1400
    // 13-bit addr: 0x1C2 -> 00001 1100 0010, 3-bit command (read): 0b100, dummy 8 bits for 8-bit read size: 0000 0000
    // Expected read output after setting beam 21: 0x95 (set bit 7 0x80 + 0x15 beam no)
    //
    // raw write bf_rx_awv_ptr (beam no) = 21: 0b0000 1110 0001 0000 1001 0101 0000 0000 -> 0x0e109500
    // 13-bit addr: 0x1C2 -> 00001 1100 0010, 3-bit command (raw write): 0b000, 8-bit data to be written: 1001 0101, dummy 8 bits for 8 termination cycles
    
    uint32_t readback, rb_beam_id, beam_id, payload;
    while (1) {
        std::cout << "Enter RX beam ID: ";
        std::cin >> beam_id;
        
        if ((0 <= beam_id) && (beam_id <= 21)) {
            for (int i = 0; i < SPI_PERIPH_NO; i++) {
                std::cout << "raw write bf_rx_awv_ptr for channel " << std::dec << i << ":\n";
                payload = 0x0e108000;
                payload |= ((beam_id << 8) & 0x3f00);
                
                if (i == 0) {
                    SPI_RW_WRAPPER(payload, 32, config, spi_ref_0, 0);
                } else if (i == 1) {
                    SPI_RW_WRAPPER(payload, 32, config, spi_ref_0, 1);
                } else if (i == 2) {
                    SPI_RW_WRAPPER(payload, 32, config, spi_ref_1, 0);
                } else { // i == 3
                    SPI_RW_WRAPPER(payload, 32, config, spi_ref_1, 1);
                }
                
                std::cout << "read bf_rx_awv_ptr for channel " << std::dec << i << ":\n";
                if (i == 0) {
                    readback = SPI_RW_WRAPPER(0x0e1400, 24, config, spi_ref_0, 0);
                } else if (i == 1) {
                    readback = SPI_RW_WRAPPER(0x0e1400, 24, config, spi_ref_0, 1);
                } else if (i == 2) {
                    readback = SPI_RW_WRAPPER(0x0e1400, 24, config, spi_ref_1, 0);
                } else { // i == 3
                    readback = SPI_RW_WRAPPER(0x0e1400, 24, config, spi_ref_1, 1);
                }
                
                rb_beam_id = readback & 0x3f;
                if (beam_id == rb_beam_id) {
                    std::cout << "SPI write successful for channel " << std::dec << i << "!\n\n";
                } else {
                    std::cout << "SPI write failed for channel " << std::dec << i << "!\n\n";
                }
            }
        }
        else {
            std::cout << "Invalid beam ID, try again!\n\n";
        }
    }
    
    return EXIT_SUCCESS;
}

[twilsonSCT]

Thanks @twilsonSCT for your feedback. Based on that I extended the spi example a bit more (see below). In this example I left all pins the same for all configs except for the chip select which got an offset of 1, so I can distinguish them. I tested it both with UHD 4.9 and 4.10 and I could indeed find a difference: The SPI index offset seems to have changed. In 4.9, the pins at the second connector had an offset of 12 while for some reason from 4.10 on it is an offset of 16. When configuring the DDR with set_gpio_attr() however, the offset is still 12. So we definitely have an issue here, but I'm not entirely sure that this is the issue you are seeing. Can you please check if this is indeed "your" issue or if there is more, so we can find the right fix? Thanks!

#include <uhd/features/spi_getter_iface.hpp>
#include <uhd/usrp/multi_usrp.hpp>
#include <uhd/utils/safe_main.hpp>
#include <stdlib.h>
#include <boost/program_options.hpp>
#include <iostream>

static const size_t SPI_DEFAULT_CLK_PIN        = 0;
static const size_t SPI_DEFAULT_SDI_PIN        = 1;
static const size_t SPI_DEFAULT_SDO_PIN        = 2;
static const size_t SPI_DEFAULT_CS_PIN         = 4;
static const size_t SPI_DEFAULT_PAYLOAD_LENGTH = 32;
static const std::string SPI_DEFAULT_PAYLOAD   = "0xfefe";
static const size_t SPI_DEFAULT_CLK_DIVIDER    = 4;
static const size_t SPI_GPIO1_SPI_INDEX_OFFSET = 16; // was 12 in UHD < v4.10
static const size_t SPI_GPIO1_DDR_INDEX_OFFSET = 12;

namespace po = boost::program_options;

int UHD_SAFE_MAIN(int argc, char* argv[])
{
    // variables to be set by po
    std::string args;
    size_t clk;
    size_t sdi;
    size_t sdo;
    size_t cs;
    size_t payload_length;
    size_t clk_divider;
    std::string payload_str;
    uint32_t payload;

    // setup the program options
    po::options_description desc("Allowed options");
    // clang-format off
    desc.add_options()
        ("help", "help message")
        ("args", po::value<std::string>(&args)->default_value(""), "multi uhd device address args")
        ("list-banks", "print list of banks before running tests")
        ("clk", po::value<size_t>(&clk)->default_value(SPI_DEFAULT_CLK_PIN), "number of pin for SPI clock")
        ("sdo", po::value<size_t>(&sdo)->default_value(SPI_DEFAULT_SDO_PIN), "number of pin for serial data out")
        ("sdi", po::value<size_t>(&sdi)->default_value(SPI_DEFAULT_SDI_PIN), "number of pin for serial data in")
        ("cs", po::value<size_t>(&cs)->default_value(SPI_DEFAULT_CS_PIN), "number of pin for chip select")
        ("payload", po::value<std::string>(&payload_str)->default_value(SPI_DEFAULT_PAYLOAD), "payload as integer value")
        ("length", po::value<size_t>(&payload_length)->default_value(SPI_DEFAULT_PAYLOAD_LENGTH), "payload length in bits")
        ("clk-div", po::value<size_t>(&clk_divider)->default_value(SPI_DEFAULT_CLK_DIVIDER), "clock divider for SPI")
    ;
    // clang-format on
    po::variables_map vm;
    po::store(po::parse_command_line(argc, argv, desc), vm);
    po::notify(vm);

    // print the help message
    if (vm.count("help")) {
        std::cout << argv[0] << " " << desc << std::endl;
        return ~0;
    }

    // create a usrp device
    std::cout << std::endl;
    std::cout << "Creating the usrp device with: " << args << "..." << std::endl;
    auto usrp = uhd::usrp::multi_usrp::make(args);

    if (vm.count("list-banks")) {
        std::cout << "Available GPIO banks: " << std::endl;
        auto banks = usrp->get_gpio_banks(0);
        for (auto& bank : banks) {
            std::cout << "* " << bank << std::endl;
        }
    }

    // Get the SPI getter interface from where we'll get the SPI interface itself
    if (!usrp->get_radio_control().has_feature<uhd::features::spi_getter_iface>()) {
        std::cout << "Error: Could not find SPI_Getter_Iface. Please check if your FPGA "
                     "image is up to date.\n";
        return EXIT_FAILURE;
    }
    auto& spi_getter_iface =
        usrp->get_radio_control().get_feature<uhd::features::spi_getter_iface>();

    auto& spi_getter_iface1 = usrp->get_radio_control(usrp->get_rx_num_channels() - 1)
                                  .get_feature<uhd::features::spi_getter_iface>();

    // Set all available pins to SPI for GPIO0 and GPIO1
    std::vector<std::string> sources(12, "DB0_SPI");
    std::vector<std::string> sources1(12, "DB1_SPI");
    usrp->set_gpio_src("GPIO0", sources);
    usrp->set_gpio_src("GPIO1", sources1);


    // Create peripheral configuration per peripheral
    uhd::features::spi_periph_config_t periph_cfg0;
    periph_cfg0.periph_clk = clk;
    periph_cfg0.periph_sdi = sdi;
    periph_cfg0.periph_sdo = sdo;
    periph_cfg0.periph_cs  = cs;

    uhd::features::spi_periph_config_t periph_cfg1;
    periph_cfg1.periph_clk = clk;
    periph_cfg1.periph_sdi = sdi;
    periph_cfg1.periph_sdo = sdo;
    periph_cfg1.periph_cs  = cs + 1;

    uhd::features::spi_periph_config_t periph_cfg2;
    // GPIO1 uses a different index space for the SPI engine than for DDR bits.
    periph_cfg2.periph_clk = clk + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg2.periph_sdi = sdi + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg2.periph_sdo = sdo + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg2.periph_cs  = cs + SPI_GPIO1_SPI_INDEX_OFFSET;

    uhd::features::spi_periph_config_t periph_cfg3;
    periph_cfg3.periph_clk = clk + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg3.periph_sdi = sdi + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg3.periph_sdo = sdo + SPI_GPIO1_SPI_INDEX_OFFSET;
    periph_cfg3.periph_cs  = cs + SPI_GPIO1_SPI_INDEX_OFFSET + 1;

    // The vector holds the peripheral configs with index=peripheral number
    std::vector<uhd::features::spi_periph_config_t> periph_cfgs;
    periph_cfgs.push_back(periph_cfg0);
    periph_cfgs.push_back(periph_cfg1);
    std::vector<uhd::features::spi_periph_config_t> periph_cfgs1;
    periph_cfgs1.push_back(periph_cfg2);
    periph_cfgs1.push_back(periph_cfg3);

    // Set the data direction register
    uint32_t outputs = 0x0;
    outputs |= 1 << periph_cfg0.periph_clk;
    outputs |= 1 << periph_cfg0.periph_sdo;
    outputs |= 1 << periph_cfg0.periph_cs;
    outputs |= 1 << periph_cfg1.periph_cs;

    uint32_t outputs1 = 0x0;
    outputs1 |= 1 << (clk + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (sdo + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (cs + SPI_GPIO1_DDR_INDEX_OFFSET);
    outputs1 |= 1 << (cs + SPI_GPIO1_DDR_INDEX_OFFSET + 1);

    usrp->set_gpio_attr("GPIOB", "DDR", outputs, outputs);
    usrp->set_gpio_attr("GPIOA", "DDR", outputs1, outputs1);
    auto spi_ref  = spi_getter_iface.get_spi_ref(periph_cfgs);
    auto spi_ref1 = spi_getter_iface1.get_spi_ref(periph_cfgs1);

    std::cout << "Using pins: " << std::endl
              << "  Clock = " << (int)(periph_cfg0.periph_clk) << std::endl
              << "  SDO   = " << (int)(periph_cfg0.periph_sdo) << std::endl
              << "  SDI   = " << (int)(periph_cfg0.periph_sdi) << std::endl
              << "  CS    = " << (int)(periph_cfg0.periph_cs) << std::endl
              << std::endl;

    payload = strtoul(payload_str.c_str(), NULL, 0);
    std::cout << "Writing payload: 0x" << std::hex << payload << " with length "
              << std::dec << payload_length << " bits" << std::endl;
    // The spi_config_t holds items like the clock divider and the SDI and SDO edges
    uhd::spi_config_t config;
    config.divider            = clk_divider;
    config.use_custom_divider = true;
    config.mosi_edge          = config.EDGE_RISE;
    config.miso_edge          = config.EDGE_FALL;

    // Do the SPI transaction. There are write() and read() methods available, too.
    std::cout << "Performing SPI transaction..." << std::endl;
    uint32_t read_data = spi_ref->transact_spi(0, config, payload, payload_length, true);
    spi_ref->transact_spi(1, config, payload, payload_length, true);
    spi_ref1->transact_spi(0, config, payload, payload_length, true);
    spi_ref1->transact_spi(1, config, payload, payload_length, true);
    std::cout << "Data read: 0x" << std::hex << read_data << std::endl;

    return EXIT_SUCCESS;
}

Hello @manderseck, I was able to confirm that by shifting the spi bits to 16 and keeping the ddr bits at 12 on GPIO1, the spi works again on UHD 4.10.

[manderseck]

@inets-rwth Thanks for the additional information! Knowing that you worked on UHD 4.6 is an important data point. There has been some rework of GPIO for UHD 4.10 and @twilsonSCT confirmed that we seem to have introduced an issue here. He also confirmed that it was working fine with UHD 4.8 and that it is working in 4.10 with the SPI bit offset of 16 instead of 12. So together with our internal testing it looks like we don't have a general issue.
For your particular example you're getting the spi_getter_iface from radio0 twice:

if (SPI_PERIPH_NO > 2) {
        // get second SPI controller
        auto & spi_get_iface_1 = usrp->get_radio_control(1).get_feature<uhd::features::spi_getter_iface>();
        spi_ref_1 = spi_get_iface_1.get_spi_ref(periph_cfgs_1);
    }

The important thing here is that in the MultiUSRP API, get_radio_control() takes the channel number and not the radio index as argument. As a consequence, depending on the device you have, you'll get radio0 again when passing 1 as argument. Since I did my initial tests on an X420, I used channel index 1 because we only have one channel per radio. But I marked that as FIXME in my example code. In my second snippet which I used on X410 and X420 I replaced it with

auto& spi_getter_iface1 = usrp->get_radio_control(usrp->get_rx_num_channels() - 1)
                              .get_feature<uhd::features::spi_getter_iface>();

...which in a single-device scenario will definitely return radio1.

Summary:

@inets-rwth: Please use a channel from the radio1 to get radio1 via get_radio_control(), that should resolve your issues in UHD <4.10.
@twilsonSCT: In UHD 4.10, please use the SPI bit offset of 16 for the second connector. We'll get the correct offset of 12 back in UHD 4.11.

[manderseck]

A fix has been created in our internal repo and will go live with our next push to public master. For the time being, this is the diff:

diff --git a/host/lib/include/uhdlib/usrp/cores/spi_core_4000.hpp b/host/lib/include/uhdlib/usrp/cores/spi_core_4000.hpp
index 19423f25bc..f1140f12ee 100644
--- a/host/lib/include/uhdlib/usrp/cores/spi_core_4000.hpp
+++ b/host/lib/include/uhdlib/usrp/cores/spi_core_4000.hpp
@@ -14,6 +14,13 @@
 #include <memory>

 namespace uhd { namespace cores {
+// There are two ports, each with 12 user-configurable pins (GPIO0: [0..11], GPIO1:
+// [12..23])
+constexpr size_t NUM_PINS_PER_PORT = 12;
+
+// In the FPGA register map, Port B pin indices start at 16 (not 12)
+constexpr size_t PORT_NUMBER_OFFSET = 16;
+
 class spi_core_4000 : uhd::noncopyable, public uhd::spi_iface
 {
 public:
diff --git a/host/lib/usrp/cores/spi_core_4000.cpp b/host/lib/usrp/cores/spi_core_4000.cpp
index e135ca6f9f..e6179564bf 100644
--- a/host/lib/usrp/cores/spi_core_4000.cpp
+++ b/host/lib/usrp/cores/spi_core_4000.cpp
@@ -73,14 +73,26 @@ public:
             periph_ctrl |= (1 << 26);
         }
         periph_ctrl |= ((num_bits & 0x3F) << 20);
+
+        /* In the FPGA, the offset between Port A and B is 16, but we only have 12 pins
+         * per port. To align this with the rest of the GPIO API, we need to adjust the
+         * pin numbers accordingly.
+         */
+        const auto get_bitfield = [](uint32_t pin, uint32_t offset) -> uint32_t {
+            const uint32_t adj = pin
+                                 + ((pin >= NUM_PINS_PER_PORT) ? static_cast<uint32_t>(
+                                        PORT_NUMBER_OFFSET - NUM_PINS_PER_PORT)
+                                                               : 0u);
+            return (adj & 0x1F) << offset;
+        };
         // periph_cs (which GPIO line for CS signal)
-        periph_ctrl |= (_spi_periph_config[which_periph].periph_cs & 0x1F) << 15;
+        periph_ctrl |= get_bitfield(_spi_periph_config[which_periph].periph_cs, 15);
         // periph_sdi (which GPIO line for serial data in from peripheral)
-        periph_ctrl |= (_spi_periph_config[which_periph].periph_sdi & 0x1F) << 10;
+        periph_ctrl |= get_bitfield(_spi_periph_config[which_periph].periph_sdi, 10);
         // periph_sdo (which GPIO line for serial data out to peripheral)
-        periph_ctrl |= (_spi_periph_config[which_periph].periph_sdo & 0x1F) << 5;
+        periph_ctrl |= get_bitfield(_spi_periph_config[which_periph].periph_sdo, 5);
         // periph_clk (which GPIO line for clk signal)
-        periph_ctrl |= (_spi_periph_config[which_periph].periph_clk & 0x1F) << 0;
+        periph_ctrl |= get_bitfield(_spi_periph_config[which_periph].periph_clk, 0);

         // conditionally send peripheral control
         if (_periph_ctrl_cache[which_periph] != periph_ctrl) {

[twilsonSCT]

+++ b/host/lib/include/uhdlib/usrp/cores/spi_core_4000.hpp

Thank you @manderseck for your support. We will implement the workaround.