Skip to content

Cross-compilation support, broken into small incremental commits#163

Open
randomizedcoder wants to merge 17 commits into
nvmd:developfrom
randomizedcoder:cross-compile-incremental
Open

Cross-compilation support, broken into small incremental commits#163
randomizedcoder wants to merge 17 commits into
nvmd:developfrom
randomizedcoder:cross-compile-incremental

Conversation

@randomizedcoder
Copy link
Copy Markdown

Hey @nvmd! 👋

First off — really awesome to see the progress you've been making since issue #154! The ffmpeg-8 addition, the refactor-linux branch work, the firmware sources extraction, and the cross-compilation support you've already started building out — all great stuff. It's clear you've been thinking hard about the right architecture, and it shows. Thanks for being so welcoming about merging these changes back!

What's in this PR

As we discussed in #154, I've taken the big cross-compile branch and broken it up into 17 small, incremental commits that (hopefully!) each make sense on their own and step-by-step build towards the full cross-compilation solution. The idea is that you can review each commit individually, and at every step the repo should be in a coherent state.

Here's the rough progression:

  1. Formatting — nixfmt-rfc-style applied first, so it doesn't pollute later diffs
  2. Logo + docs — the RPi logo images, ffmpeg docs
  3. FFmpeg cleanup — remove deprecated ffmpeg 4/6, add mkFfmpegRpi helper
  4. Pisugar-3 refactor + sysctl module
  5. Kernel/firmware refactoring (commits 6a–6e) — this is the big architectural piece, done incrementally
  6. Overlay + module registries
  7. Dev-shells + tooling (Makefile, etc.)
  8. Board module compatibility — dual pkgs.* + nixos-raspberrypi.packages support
  9. Cross-compilation infrastructure — the main event!
  10. Flake.nix refactor — wiring it all together
  11. Documentation — cross-compilation guide, README updates

Kernel/firmware organization — two approaches

This is probably the most important architectural decision in the PR, so I wanted to walk through the thinking. When looking at how to organize kernel versions, firmware, and their relationships, we considered two approaches:

Your current approach (modular/hierarchical)

This is what's on develop and refactor-linux today — kernel data is split across multiple files in a directory structure:

  • pkgs/linux-rpi/{package.nix, kernels.nix, linux-sources.nix, linux-patches.nix} (4 files)
  • overlays/firmware-sources.nix (separate firmware version list)
  • Overlay ordering is implicit in flake.nix
  • ~13 total files, ~963 lines

Our proposed approach (data-driven/consolidated)

Instead of a directory hierarchy, we encode the kernel/firmware relationships as explicit data tables:

  • kernelMatrix in vendor-kernel.nix — a single table mapping every kernel version to its supported board models. Adding a new kernel = adding one row.
  • bundleSpecs in linux-and-firmware.nix — explicitly pairs each kernel with its matching firmware. The relationships are visible at a glance.
  • firmwareVersions / wirelessFirmwareVersions in vendor-firmware.nix — structured attrsets with mkFirmware/mkWirelessFirmware constructors instead of procedural definitions.
  • commonFixupStructuredConfig + mkFixupStructuredConfig in kernels.nix — deduplicates kernel config fixups that were previously repeated across every kernel definition.
  • overlays/default.nix — explicit overlay registry with documented ordering (cross-fixes → pkgs → bootloader → vendor-kernel → vendor-firmware → kernel-and-firmware → vendor-pkgs).
  • pkgs/linux-rpi.nix — single consolidated file instead of the directory
  • ~9 total files, ~800 lines

Comparison

Criterion Your approach (modular) Our proposal (data-driven)
Files to touch for new kernel 4 3
Files to touch for new firmware 2–3 2
Total files / lines ~13 / ~963 ~9 / ~800
Code deduplication Good Better (commonFixupStructuredConfig)
Discoverability Data scattered across files Co-located in data tables
Overlay ordering Implicit in flake.nix Explicit + documented in overlays/default.nix
Cross-compile readiness Needs additions cross-fixes overlay built-in, first in chain
Self-documenting Minimal Data tables show version↔board↔firmware relationships

Why we recommend the data-driven approach

The key insight is that kernel/firmware management is fundamentally a data problem — you have a matrix of versions × boards × firmware pairings. Encoding this as explicit data tables (rather than spreading it across a directory hierarchy) makes the relationships visible at a glance and reduces the number of places you need to touch when adding new versions.

It also makes the overlay ordering explicit and documented, which is especially important for cross-compilation where cross-fixes must come first in the chain.

The commits 6a–6e introduce this incrementally so you can see exactly what each step changes:

  • 6a: Consolidates pkgs/linux-rpi/ directory back to single pkgs/linux-rpi.nix
  • 6b: Extracts commonFixupStructuredConfig helper
  • 6c: Data-driven firmware versions
  • 6d: Data-driven vendor-kernel with kernelMatrix
  • 6e: Data-driven linux-and-firmware with bundleSpecs

Regarding commonFixupStructuredConfig

I totally hear your point from #154 about potentially auto-generating fixups in the future. The commonFixupStructuredConfig helper is designed to be easy to replace — if you build a generator tool later, you'd just swap out the data it produces. The helper is really just factoring out the duplication that exists today, not locking in any particular approach.

Board module compatibility

The board modules now support both access patterns:

  • pkgs.* via overlays (needed for cross-compilation to work)
  • nixos-raspberrypi.packages as fallback (backward compatibility)

This is done with: { nixos-raspberrypi ? null, self ? nixos-raspberrypi, lib, pkgs, ... }:

Testing

The cross-compilation has been tested building installer images from x86_64:

nix build .#installerImagesCross.x86_64-linux.rpi5
nix build .#packages.x86_64-linux.linux_rpi5

AI assistance (Claude) was used for code organization, formatting, and drafting this PR — but the architectural decisions, Nix patterns, and cross-compilation approach are all human-designed and tested on real hardware.

How to review

I'd suggest reviewing commit-by-commit (the GitHub "Commits" tab works well for this). Each commit message explains what it does and why. If any particular commit doesn't make sense or you'd prefer a different approach, happy to rework it!

Also totally fine if you want to cherry-pick just some of these commits, or if you'd prefer I split this into multiple smaller PRs (e.g., formatting + ffmpeg cleanup as one, kernel refactoring as another, cross-compilation as a third). Whatever works best for your workflow! 🙂

Closes #154

🤖 Generated with Claude Code

randomizedcoder and others added 17 commits March 19, 2026 17:45
Formatting-only changes across modules, packages, and overlays.
No logic changes.

- Expand function parameter lists to multi-line
- Normalize empty attrset spacing: {} → { }
- Add missing EOF newlines
- Reformat let..in blocks and list continuations
- Add inherit (lib) pattern in bootloader module

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add official Raspberry Pi logo images and display the logo
at the top of the README.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
FFmpeg 4.x reached EOL, and FFmpeg 6.x is superseded by 7 and 8.
Only FFmpeg 7 and 8 RPi forks are maintained upstream.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document the RPi-optimized FFmpeg packages: available versions,
RPi-specific features, and usage patterns for NixOS configurations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace repetitive per-version ffmpeg blocks with a data-driven
mkFfmpegRpi helper that maps over version list. Also make --enable-sand
unconditional (all RPi ffmpeg forks support it), use enableFeature
helper consistently in kodi flags, and remove withSand/withVoutDrm/
withV4l2Request overrides from ffmpeg_8 (defaults are correct).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… module

pisugar-3: Add configurable options for i2cBus and i2cAddr instead of
hardcoded values. Use direct callPackage for kernel module instead of
overlay injection. This makes the module self-contained and simpler.

sysctl: Add network sysctl tuning module for improved TCP performance
on Raspberry Pi (keepalive, buffer sizes, congestion control).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidate the 4-file pkgs/linux-rpi/ directory (package.nix,
kernels.nix, linux-sources.nix, linux-patches.nix) back to a single
pkgs/linux-rpi.nix package builder and overlays/kernels.nix data file.

The directory split didn't reduce complexity — all data is co-located
and easier to maintain in fewer files. The kernel version data, patches,
and config fixups are now in overlays/kernels.nix alongside the overlays
that consume them.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add commonFixupStructuredConfig and mkFixupStructuredConfig helpers
to deduplicate the per-SoC kernel config fixups that were repeated
for v6_12_25 and v6_6_31. The common config enforces RPi defconfig
defaults (preemption, NFS, CMA size, etc.) that NixOS overrides.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace procedural firmware definitions with firmwareVersions and
wirelessFirmwareVersions data tables plus mkFirmware/mkWirelessFirmware
constructors. Firmware version data is now inline instead of in a
separate firmware-sources.nix file.

Adding a new firmware version is now a single attrset entry instead
of copying a multi-line block.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the explicit list of mkLinuxFor calls with a kernelMatrix
data table that maps kernel versions to supported RPi models.
This makes the version-to-model relationship self-documenting
and reduces the diff when adding new kernel versions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace explicit mkBundle calls with a bundleSpecs data table that
maps kernel versions to compatible firmware dates. Add stable alias
alongside default and latest. mkBundle now takes a spec record
instead of separate arguments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add overlays/default.nix with explicit overlay registry and documented
ordering. Add modules/default.nix as the nixosModules output registry.

These registries centralize what was previously inline in flake.nix,
making the overlay dependency chain and module exports discoverable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename devshells/ to dev-shells/ for consistency with the flake output
name. Add ASCII art shell hook, jp2a dependency, and a Makefile with
fmt/lint/check/update targets. Update .envrc to watch dev-shells/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…rrypi.packages

Board modules now reference kernel and firmware packages via pkgs.*
(injected through overlays) instead of nixos-raspberrypi.packages.
This is required for cross-compilation where the package set must
be constructed with the correct build/host platform pair.

The nixos-raspberrypi special arg is kept as an optional parameter
(defaulting to null) with self as a fallback, maintaining backward
compatibility with existing configurations that pass it via
specialArgs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add the core infrastructure for building RPi packages and installer
images from x86_64 without binfmt emulation:

- lib/systems.nix: define rpiSystems, buildSystems
- lib/pkgs.nix: memoized nixpkgs instances for native and cross builds
- lib/default.nix: mkBuilder factory with buildPlatform threading
- lib/internal.nix: inject nixpkgs.buildPlatform into NixOS modules
- overlays/cross-fixes.nix: fix gnutls doc build under cross-compilation
- overlays/default.nix: add cross-fixes as first overlay in chain
- pkgs/default.nix: flat packages output with cross-compilation support
- installers/default.nix: native + cross installer image builders

Usage:
  nix build .#packages.x86_64-linux.ffmpeg_8  # cross-compile from x86
  nix build .#installerImagesCross.x86_64-linux.rpi5  # cross SD image

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… infra

Replace inline flake output definitions with imports from the new
modular registries:
- overlays/ for overlay definitions and ordering
- modules/ for NixOS module exports
- lib/pkgs.nix for package set construction
- pkgs/ for the packages output
- installers/ for installer image builders
- dev-shells/ for development shells

The flake now exposes cross-compilation outputs:
- packages.x86_64-linux.* for cross-compiled aarch64 packages
- installerImagesCross.x86_64-linux.* for cross-compiled SD images
- checks.*.all-installer-images for CI validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add comprehensive cross-compilation documentation covering:
- Quick start for building from x86_64
- Package and installer image build commands
- NixOS configuration with buildPlatform
- How cross-compilation works under the hood
- Troubleshooting guide

Update README with table of contents, quick start section,
cross-compilation overview, and project structure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ivanoovii
Copy link
Copy Markdown

Hi! Thanks for this great PR! I've already used it to create installers for my RPi4 and 5. However, what I find rather inconvenient is that I have to clone the repo to change custom-user-config, since it is defined in in the let block in installers/default.nix. I've created an issue #170 which describes this problem in the original codebase, but because this PR might be merged soon, maybe we also should discuss exposing the config here.

@blaggacao
Copy link
Copy Markdown

blaggacao commented Apr 10, 2026

@randomizedcoder To help ensure there's no misunderstanding waiting to be discovered, I had read:

I've scrolled through all of it. I would carefully say, that it would make sense to potentially accept all of it, but definitely in several PRs [PRs not commits].

That's also what I would expect as a maintainer to be able to review one change at a time.

@nvmd Don't know, maybe formatting first (at some checkpoint that is suitable to you) is a better approach since these AI tools seem to always be eager to format each commit and it might be hard to separate this out (or teach the agent to not touch formatting) 🤷 - not sure though.

@nvmd
Copy link
Copy Markdown
Owner

nvmd commented Apr 16, 2026

@randomizedcoder To help ensure there's no misunderstanding waiting to be discovered, I had read:

I've scrolled through all of it. I would carefully say, that it would make sense to potentially accept all of it, but definitely in several PRs [PRs not commits].

That's also what I would expect as a maintainer to be able to review one change at a time.

Thanks @blaggacao for noticing that. I guess I got overwhelmed while reviewing the PR, so that I've forgotten to bring this to the attention.

@randomizedcoder thank you for splitting this into the commits! It's also very important that those will be also split across multiple PRs, like we discussed earlier, so that we could review and accept them independently and incrementally.

As for the for formatting, I'm not yet sure how to deal with it.
I guess we should stick with the code style used in the community at large (and nixpkgs) at some point (even though I personally don't like the RFC formatting for several reasons).

On the other hand doing a global reformatting will somewhat obscure with git history, especially in the short term. To mitigate them, I would prefer the gradual approach of integrating the meaningful changes with the formatting updates. As far as starting on the so to say, function level, for example.

That can be, however, quite labor-intensive: "as simple as" adding only the selected portion of the file to the commit after the reformat for the humans, but probably more complicated to "tame" the AI agent to do that right.
However, I wouldn't want to sacrifice the tokens spent by the AI at the expense of maintainers (being human or AI) in the future.

The resolution being for now, let the formatting be the separate PR that we'll discuss after we're done with the meaningful changes.

@nvmd
Copy link
Copy Markdown
Owner

nvmd commented Apr 16, 2026

For the @randomizedcoder and the Claude Code concerning linux/firmware approach: I don't think the lists of kernel and firmware sources will go away because they help separate the concerns better. More precisely the logic of generating and organizing derivations from the data being the sources for such derivations.

@doronbehar doronbehar mentioned this pull request Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cross compiling

4 participants