wrong disk size used when building AMI's

[T0MASD]

bootc-image-builder Issue: Fixed 10GB Disk Images

Summary

bootc-image-builder creates fixed 10GB disk images regardless of actual container size, leading to:

• Longer upload times: 10GB upload vs 4.5GB in our case
• Wasted bandwidth: 5.5GB unnecessary data transfer
• Build time impact: 100 minutes vs 30 minutes for our use case

Expected Behavior

Images should be sized to fit the actual container content exactly:

• Calculate actual filesystem usage
• Create partition sized to content (no margin needed)
• Let cloud-init/growpart expand to requested volume size at instance launch

Current Behavior

$ sudo podman run ... bootc-image-builder:latest --type ami localhost/myapp:latest

# Creates fixed 10GB image:
Uploading myapp-bootc to S3
90 MiB / 10 GiB [->___] 0.88% 1.58 MiB p/s  # ~100 minutes for a 2GB container!

Actual image sizes (qemu-img info):

$ qemu-img info output/image/disk.raw
file format: raw
virtual size: 4.49 GiB (4817879040 bytes)  # Our shrunk image
file length: 4.49 GiB (4817879040 bytes)   # Upload size

# bootc-image-builder AMI output:
virtual size: 10 GiB (10737418240 bytes)   # Fixed at 10GB
file length: 10 GiB (10737418240 bytes)    # Upload size

Problem: Raw images upload the full file length to S3

• Our shrunk image: 4.49 GB upload (4817879040 bytes)
• bootc-image-builder: 10 GB upload (10737418240 bytes)
• Waste: 5.51 GB unnecessary upload (2.23x larger than needed)

Real-World Impact

Fedora bootc container example:

• Current: 10 GB upload (10737418240 bytes) - 100 minutes
• Should be: 4.5 GB upload (4817879040 bytes) - 30 minutes
• Wasted: 5.5 GB and 70 minutes per build

Cloud platforms handle expansion:

1. Image sized to content -> uploaded to S3/storage
2. User launches with desired volume size (e.g., 20GB)
3. cloud-init/growpart expands partition at boot
4. Filesystem grows to fill partition

Proposed Solution

Option 1: Auto-size to content

bootc-image-builder --type ami --auto-size localhost/myapp:latest

Option 2: User-specified size

bootc-image-builder --type ami --disk-size 2500M localhost/myapp:latest

Option 3: Make 10GB opt-in

bootc-image-builder --type ami localhost/myapp:latest  # Auto-sizes by default
bootc-image-builder --type ami --disk-size 10G localhost/myapp:latest  # If needed

Workaround

Until this is fixed, we use virt-resize to shrink images post-build:

#!/bin/bash

RAW_IMAGE="output/image/disk.raw"

# Setup loop device
LOOP_DEV=$(sudo losetup -f --show --partscan $RAW_IMAGE)

# Shrink root filesystem to minimum
sudo e2fsck -fy ${LOOP_DEV}p4
sudo resize2fs -M ${LOOP_DEV}p4

# Get actual filesystem size
FS_BLOCKS=$(sudo tune2fs -l ${LOOP_DEV}p4 | grep "^Block count:" | awk '{print $3}')
FS_BLOCKSIZE=$(sudo tune2fs -l ${LOOP_DEV}p4 | grep "^Block size:" | awk '{print $3}')
FS_SIZE_MB=$((FS_BLOCKS * FS_BLOCKSIZE / 1024 / 1024))

# Calculate target with safety margin
ROOT_PART_MB=$((FS_SIZE_MB + 10))
TOTAL_SIZE_MB=$((1 + 501 + 1024 + ROOT_PART_MB))  # BIOS + EFI + boot + root

echo "Shrinking from 10GB to ${TOTAL_SIZE_MB}MB"

# Cleanup and resize
sudo losetup -d $LOOP_DEV
sudo virt-resize --align-first auto \
  --resize /dev/sda4=${ROOT_PART_MB}M \
  $RAW_IMAGE output/image/disk_shrunk.raw

# Result: 2.5GB image instead of 10GB

Results: 10 GB -> 4.5 GB (100 min -> 30 min upload)

Environment

$ sudo podman run --rm -it quay.io/centos-bootc/bootc-image-builder:latest version
bootc-image-builder v0.X.Y (or latest)

Container: Fedora 43 bootc with ~2GB of packages
Build host: Fedora 43
Target: AWS AMI (eu-west-3)

[achilleas-k]

Hi. Thanks for the detailed report. We're aware of this issue and have already started working towards removing the 10 GiB minimum restriction (osbuild/images#2018).

The plan right now is to make it so that the hard-minimum is 2x the size of the container. Does that make sense for you, or do you need to be able to go smaller than that even?

[achilleas-k]

I created https://issues.redhat.com/browse/HMS-9766 for out internal tracking.

[T0MASD]

at least for --type ami there's no need to have any padding in the last partition by default (user would handle growing of the last partition using cloud-init or custom growfs oneshot systemd service), if for some reason user wants padding (not sure why tho) then they can specify that during the build (which is not yet implemented)

[achilleas-k]

Well the 2x rule isn't just for padding, it's generally because we don't expect the container size to accurately represent the size of the deployed content and because we will need room for the runtime data. We consider 2x to be a sane default, but maybe that's too mush and we could scale it down a bit.

[T0MASD]

I think since built os is immutable (and as demonstrated with shrinking of the partition after the build and before boot) in case of ami it's fairly safe keep partition size to the minimum and let tooling expand the partition during the firt boot, I'd be happy to test and confirm if fix works (will just need link to quay with dev version of builder image)