Cross-compilation support, broken into small incremental commits#163
Cross-compilation support, broken into small incremental commits#163randomizedcoder wants to merge 17 commits into
Conversation
Formatting-only changes across modules, packages, and overlays.
No logic changes.
- Expand function parameter lists to multi-line
- Normalize empty attrset spacing: {} → { }
- Add missing EOF newlines
- Reformat let..in blocks and list continuations
- Add inherit (lib) pattern in bootloader module
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add official Raspberry Pi logo images and display the logo at the top of the README. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
FFmpeg 4.x reached EOL, and FFmpeg 6.x is superseded by 7 and 8. Only FFmpeg 7 and 8 RPi forks are maintained upstream. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document the RPi-optimized FFmpeg packages: available versions, RPi-specific features, and usage patterns for NixOS configurations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace repetitive per-version ffmpeg blocks with a data-driven mkFfmpegRpi helper that maps over version list. Also make --enable-sand unconditional (all RPi ffmpeg forks support it), use enableFeature helper consistently in kodi flags, and remove withSand/withVoutDrm/ withV4l2Request overrides from ffmpeg_8 (defaults are correct). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… module pisugar-3: Add configurable options for i2cBus and i2cAddr instead of hardcoded values. Use direct callPackage for kernel module instead of overlay injection. This makes the module self-contained and simpler. sysctl: Add network sysctl tuning module for improved TCP performance on Raspberry Pi (keepalive, buffer sizes, congestion control). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidate the 4-file pkgs/linux-rpi/ directory (package.nix, kernels.nix, linux-sources.nix, linux-patches.nix) back to a single pkgs/linux-rpi.nix package builder and overlays/kernels.nix data file. The directory split didn't reduce complexity — all data is co-located and easier to maintain in fewer files. The kernel version data, patches, and config fixups are now in overlays/kernels.nix alongside the overlays that consume them. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add commonFixupStructuredConfig and mkFixupStructuredConfig helpers to deduplicate the per-SoC kernel config fixups that were repeated for v6_12_25 and v6_6_31. The common config enforces RPi defconfig defaults (preemption, NFS, CMA size, etc.) that NixOS overrides. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace procedural firmware definitions with firmwareVersions and wirelessFirmwareVersions data tables plus mkFirmware/mkWirelessFirmware constructors. Firmware version data is now inline instead of in a separate firmware-sources.nix file. Adding a new firmware version is now a single attrset entry instead of copying a multi-line block. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the explicit list of mkLinuxFor calls with a kernelMatrix data table that maps kernel versions to supported RPi models. This makes the version-to-model relationship self-documenting and reduces the diff when adding new kernel versions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace explicit mkBundle calls with a bundleSpecs data table that maps kernel versions to compatible firmware dates. Add stable alias alongside default and latest. mkBundle now takes a spec record instead of separate arguments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add overlays/default.nix with explicit overlay registry and documented ordering. Add modules/default.nix as the nixosModules output registry. These registries centralize what was previously inline in flake.nix, making the overlay dependency chain and module exports discoverable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename devshells/ to dev-shells/ for consistency with the flake output name. Add ASCII art shell hook, jp2a dependency, and a Makefile with fmt/lint/check/update targets. Update .envrc to watch dev-shells/. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…rrypi.packages Board modules now reference kernel and firmware packages via pkgs.* (injected through overlays) instead of nixos-raspberrypi.packages. This is required for cross-compilation where the package set must be constructed with the correct build/host platform pair. The nixos-raspberrypi special arg is kept as an optional parameter (defaulting to null) with self as a fallback, maintaining backward compatibility with existing configurations that pass it via specialArgs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add the core infrastructure for building RPi packages and installer images from x86_64 without binfmt emulation: - lib/systems.nix: define rpiSystems, buildSystems - lib/pkgs.nix: memoized nixpkgs instances for native and cross builds - lib/default.nix: mkBuilder factory with buildPlatform threading - lib/internal.nix: inject nixpkgs.buildPlatform into NixOS modules - overlays/cross-fixes.nix: fix gnutls doc build under cross-compilation - overlays/default.nix: add cross-fixes as first overlay in chain - pkgs/default.nix: flat packages output with cross-compilation support - installers/default.nix: native + cross installer image builders Usage: nix build .#packages.x86_64-linux.ffmpeg_8 # cross-compile from x86 nix build .#installerImagesCross.x86_64-linux.rpi5 # cross SD image Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… infra Replace inline flake output definitions with imports from the new modular registries: - overlays/ for overlay definitions and ordering - modules/ for NixOS module exports - lib/pkgs.nix for package set construction - pkgs/ for the packages output - installers/ for installer image builders - dev-shells/ for development shells The flake now exposes cross-compilation outputs: - packages.x86_64-linux.* for cross-compiled aarch64 packages - installerImagesCross.x86_64-linux.* for cross-compiled SD images - checks.*.all-installer-images for CI validation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add comprehensive cross-compilation documentation covering: - Quick start for building from x86_64 - Package and installer image build commands - NixOS configuration with buildPlatform - How cross-compilation works under the hood - Troubleshooting guide Update README with table of contents, quick start section, cross-compilation overview, and project structure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Hi! Thanks for this great PR! I've already used it to create installers for my RPi4 and 5. However, what I find rather inconvenient is that I have to clone the repo to change |
|
@randomizedcoder To help ensure there's no misunderstanding waiting to be discovered, I had read:
That's also what I would expect as a maintainer to be able to review one change at a time. @nvmd Don't know, maybe formatting first (at some checkpoint that is suitable to you) is a better approach since these AI tools seem to always be eager to format each commit and it might be hard to separate this out (or teach the agent to not touch formatting) 🤷 - not sure though. |
Thanks @blaggacao for noticing that. I guess I got overwhelmed while reviewing the PR, so that I've forgotten to bring this to the attention. @randomizedcoder thank you for splitting this into the commits! It's also very important that those will be also split across multiple PRs, like we discussed earlier, so that we could review and accept them independently and incrementally. As for the for formatting, I'm not yet sure how to deal with it. On the other hand doing a global reformatting will somewhat obscure with git history, especially in the short term. To mitigate them, I would prefer the gradual approach of integrating the meaningful changes with the formatting updates. As far as starting on the so to say, function level, for example. That can be, however, quite labor-intensive: "as simple as" adding only the selected portion of the file to the commit after the reformat for the humans, but probably more complicated to "tame" the AI agent to do that right. The resolution being for now, let the formatting be the separate PR that we'll discuss after we're done with the meaningful changes. |
|
For the @randomizedcoder and the Claude Code concerning linux/firmware approach: I don't think the lists of kernel and firmware sources will go away because they help separate the concerns better. More precisely the logic of generating and organizing derivations from the data being the sources for such derivations. |
Hey @nvmd! 👋
First off — really awesome to see the progress you've been making since issue #154! The ffmpeg-8 addition, the
refactor-linuxbranch work, the firmware sources extraction, and the cross-compilation support you've already started building out — all great stuff. It's clear you've been thinking hard about the right architecture, and it shows. Thanks for being so welcoming about merging these changes back!What's in this PR
As we discussed in #154, I've taken the big
cross-compilebranch and broken it up into 17 small, incremental commits that (hopefully!) each make sense on their own and step-by-step build towards the full cross-compilation solution. The idea is that you can review each commit individually, and at every step the repo should be in a coherent state.Here's the rough progression:
mkFfmpegRpihelperpkgs.*+nixos-raspberrypi.packagessupportKernel/firmware organization — two approaches
This is probably the most important architectural decision in the PR, so I wanted to walk through the thinking. When looking at how to organize kernel versions, firmware, and their relationships, we considered two approaches:
Your current approach (modular/hierarchical)
This is what's on
developandrefactor-linuxtoday — kernel data is split across multiple files in a directory structure:pkgs/linux-rpi/{package.nix, kernels.nix, linux-sources.nix, linux-patches.nix}(4 files)overlays/firmware-sources.nix(separate firmware version list)flake.nixOur proposed approach (data-driven/consolidated)
Instead of a directory hierarchy, we encode the kernel/firmware relationships as explicit data tables:
kernelMatrixinvendor-kernel.nix— a single table mapping every kernel version to its supported board models. Adding a new kernel = adding one row.bundleSpecsinlinux-and-firmware.nix— explicitly pairs each kernel with its matching firmware. The relationships are visible at a glance.firmwareVersions/wirelessFirmwareVersionsinvendor-firmware.nix— structured attrsets withmkFirmware/mkWirelessFirmwareconstructors instead of procedural definitions.commonFixupStructuredConfig+mkFixupStructuredConfiginkernels.nix— deduplicates kernel config fixups that were previously repeated across every kernel definition.overlays/default.nix— explicit overlay registry with documented ordering (cross-fixes → pkgs → bootloader → vendor-kernel → vendor-firmware → kernel-and-firmware → vendor-pkgs).pkgs/linux-rpi.nix— single consolidated file instead of the directoryComparison
commonFixupStructuredConfig)overlays/default.nixcross-fixesoverlay built-in, first in chainWhy we recommend the data-driven approach
The key insight is that kernel/firmware management is fundamentally a data problem — you have a matrix of versions × boards × firmware pairings. Encoding this as explicit data tables (rather than spreading it across a directory hierarchy) makes the relationships visible at a glance and reduces the number of places you need to touch when adding new versions.
It also makes the overlay ordering explicit and documented, which is especially important for cross-compilation where
cross-fixesmust come first in the chain.The commits 6a–6e introduce this incrementally so you can see exactly what each step changes:
pkgs/linux-rpi/directory back to singlepkgs/linux-rpi.nixcommonFixupStructuredConfighelperkernelMatrixbundleSpecsRegarding
commonFixupStructuredConfigI totally hear your point from #154 about potentially auto-generating fixups in the future. The
commonFixupStructuredConfighelper is designed to be easy to replace — if you build a generator tool later, you'd just swap out the data it produces. The helper is really just factoring out the duplication that exists today, not locking in any particular approach.Board module compatibility
The board modules now support both access patterns:
pkgs.*via overlays (needed for cross-compilation to work)nixos-raspberrypi.packagesas fallback (backward compatibility)This is done with:
{ nixos-raspberrypi ? null, self ? nixos-raspberrypi, lib, pkgs, ... }:Testing
The cross-compilation has been tested building installer images from x86_64:
AI assistance (Claude) was used for code organization, formatting, and drafting this PR — but the architectural decisions, Nix patterns, and cross-compilation approach are all human-designed and tested on real hardware.
How to review
I'd suggest reviewing commit-by-commit (the GitHub "Commits" tab works well for this). Each commit message explains what it does and why. If any particular commit doesn't make sense or you'd prefer a different approach, happy to rework it!
Also totally fine if you want to cherry-pick just some of these commits, or if you'd prefer I split this into multiple smaller PRs (e.g., formatting + ffmpeg cleanup as one, kernel refactoring as another, cross-compilation as a third). Whatever works best for your workflow! 🙂
Closes #154
🤖 Generated with Claude Code