Proposal for examples

[cmahnke]

In the last few weeks I was using this code to create some media stations for a exhibition. Some things were harder than the probably they should have been, but I was convinced by the approach and the speed of this coder (as compared to pi-gen).
From my experience I would like to see a few more examples addressing the following use cases:

• Integration with Raspberry Pi Imager : This tool allows setting WiFi credentials while writing an image, it would be great to habe an example doing this. Maybe even with WPA3 Enterprise
• Adding custom package sources (with https, trusted=yes etc).
• The old settings / variables have been misleading: For example setting a variable did't guarantee that the implied changes would be implemented since the user has been responsible for providing the required dependencies as well. One example (with the current code) is setting the console language, this won't change the keyboard layout by itself when console-setup hasn't been enabled via systemd...

Please keep in mind that there is #79 now, so this should target the result of it.

[learmj]

Hi @cmahnke

Thanks for the suggestions and proposals.

You make a good point about provisioning assets / information which would loosely be termed 'secret' and about them ideally not being present in the config file. That's not to say they couldn't be - it's just that you wouldn't normally store 'sensitive' information like that in a build configuration file. With that said, rpi-imager writes such credentials to the device filesystem anyway and if not running the image with LUKS / dm-crypt, the secret is in the clear. The more appropriate way to handle that sort of material (IMHO) is to provision it on the device after creating the image. That's how we intend on handling other types of sensitive material. That's one of the reasons we have rpi-sb-provisioner which provisions an image on the device and enables authenticated boot and encrypted filesystems. rpi-image-gen doesn't prevent users from storing their wifi creds in the image at build time - there are plenty ways to hook that up via a custom layer/script if you wanted to.

Re: custom package sources, the Docker layers already do this today. They fetch the key used to authenticate the packages, install it into the keydir apt uses to create the chroot (setup hook), install the packages from the docker repo, then add an entry to /etc/apt/sources.list.d/ in the chroot afterwards (customize hook) to make sure the docker repo is usable on the device when running the image.

I think I understand what you mean regarding variables. Locale settings are not hard coupled to (optional) system services that may be responsible for managing console language, keyboard, etc, and while systemd is the popular sysv init of choice, you're right that unless its (a) installed and (b) running its console service, changes to language/layout may not be reflected on the console. When adding the current selection of layers (which are no means set in stone) I tried to keep separation of functionality clean and minimal. User have always been able to create their own layers but the changes coming in via #79 will allow users to chain layers together as dependencies. rpi-image-gen will discover those layers so if you wanted to add a layer which installed systemd console, you could make it depend on device-base so that locale etc was set up before the systemd console service was started and your layer was built. That way, you'd boot into a system with the keyboard layout of choice configured from boot. Does that help?

[cmahnke]

Thanks for your reply.

I need to check my own example /problem with mmdebstrap...
But anyways I won't mind a good example.

The last one is certainly also a possible requirement / feature request in terms of validation of configs or at least generate warnings. I'm just thinking out loud here, and using the input language example from the initial post. Just to make it clear this is a complete made up example.
I see two ways:

• Almost no global config variables, at least none that require additional packages to be in place in the generated images. This would avoid the need for a defined dependency between a config variable and the inclusion of the required package. The result for the example would be no general / global variable for console language. It would be namespaced to the layer responsible for implementing the console locale setup and handling.
• Layers can register global variables (in the builder on startup and layer indexing time, indexing layers is certainly a requirement for validation later on) when those are set a implicit layer dependency is defined. This is easy to set up but might be harder to debug due to the implicit nature.

[learmj]

Not sure I follow - variables are already namespaced to layers. If they weren't already they certainly are now. It's only layers that can declare variables once #79 lands. Can you perhaps provide an example for me to try and figure out how to address the problem you're experiencing?

[cmahnke]

Oh, it was just meant to solve the (currently) possible problem of having variables that don't enforce their implied action.
That's the reason why I called it a possible feature request, im other words and the example above that would be:
Defining KEYBOARD_KEYMAP should either make sure console-setup is setup (without having to the user handle dependencies) or throw an error.
I was just thinking about ways to implement this. If this is already solved with the new approach even better.

[learmj]

That's the reason why I called it a possible feature request, im other words and the example above that would be:
Defining KEYBOARD_KEYMAP should either make sure console-setup is setup (without having to the user handle dependencies) or throw an error.
I was just thinking about ways to implement this. If this is already solved with the new approach even better.

Ok. I think I see. You make a good point. However, it would seem to have a hard dependency on what should be (in terms of a modular build system) an optional way to init the device, ie what services to build into the image and start, have full-fat systemd init vs a thin busybox based init, while having the desired keymap.

Variables like device keymap aren't expected to enforce hard policy decisions about what packages to install - that's why their values are just pre-seeded in the debconf database. However, on the flip side, if you define a keymap you may expect the build system to do the needful, but that may involve it making hard decisions and those likely wouldn't suit all users.

It sounds like you're suggesting that systemd
systemd-vconsole-setup.service should always be installed and started to configure the virtual terminals in order to honour the device keymap variable. I don't agree with that because a user may not care about the keymap variable at all and just keep it at the default, but definitely does wants a busybox system without systemd. To that end, by forcibly installing systemd just to honour this variable I'd be hardwiring systemd into everything. I'm not going to do that because a modular build system is about choice.

Maybe the best thing here (for this particular case) is to ensure that the systemd console service is started by any layer that installs systemd as the boot 'scheme' (eg layer systemd-net-min).

More generally, if there is a layer responsible for installing a 'sysv boot' it should try its best to honour variables like device keymap.

Have you tried just starting the vconsole service in an image with systemd boot to see if it does what you need?

[cmahnke]

Ok, I guess I see your point: Those variables have more the one implication I suspect? In this case it is not only are responsible for the setup of the console keyboard. The most obvious solution might be the namespaced duplication / specification of certain variables. Sticking with the example there might be something like CONSOLE_KEYMAP which would add as described above - that is systemd setup, including installation of required dependencies, but also sets the needed env vars. But due to the duplication (that is multiple specialised variables can set on global / general variable) that would require a validation since its' theoretically possible that those specialised variables result in conflicting values gor the global variable.

In terms of actual solution with possible (user) problems with the current code base, I guess we still aren't completely on the same page here: These are just suggestions. They are base on my experiences, but I don't expect solutions as part of this issue, actually I already solved this (for me) - I'm just a bit unhappy how it's currently solved...
For me it's just an example since I expect that this "trap" can also happen with others of the (currently) global variables - that's the reason why I argue to try to avoid them where they

[cmahnke]

Just another idea for an example:

• The kiosk mode including the ability to use a on-screen keyboard of a touch screen

[daybyter]

Thanks a lot for your proposals!

I am trying to build a similar application as you, it seems?

You know the videos of MacMind? Like:

https://www.youtube.com/watch?v=RnehTqAVqQQ

They helped me a lot, although it seems they are somewhat broken for me at moment with the pulled rpi-image-gen version.

[cmahnke]

@daybyter That's definitelly possible...
I know @jonnymacs videos.

These proposals for examples are hopefully the starting point for more: As I see it, the existing examples, new ones and configurations like the MacMind repositories on rpi-image-gen can be a starting point of a library of building blocks for a custom Raspberry Pie Image.

This issue will hopefully gather some more attention for ideas and maybe even implementations.

[daybyter]

Before I forget: seems like the hint with executable customize* files was the solution in my case. Now the web-kiosk example seems to build including the kiosk service etc.

I wonder if it would be possible for rpi-image-gen to issue a warning if such a file was ignored? At best write it to a logfile... ;-)

[jonnymacs]

@daybyter I think the bin/runner could theoretically warn when files with a matching prefix lack executable permissions.

https://github.com/raspberrypi/rpi-image-gen/blob/master/bin/runner

if [ -x "$script" ] ; then

matching scripts that fail this condition could be echo'd out. You might make a PR and see what @learmj thinks.

[learmj]

@daybyter I think the bin/runner could theoretically warn when files with a matching prefix lack executable permissions.

https://github.com/raspberrypi/rpi-image-gen/blob/master/bin/runner

if [ -x "$script" ] ; then

matching scripts that fail this condition could be echo'd out. You might make a PR and see what @learmj thinks.

It's an easy and helpful thing to add. I'll roll it into the current changes.