BlueBuild Blog

default-flatpaks v2

Sat, 26 Jul 2025 00:00:00 GMT

The default-flatpaks module is one of the earliest BlueBuild modules. As such, the module contained a few issues that made it imperfect and prevented it from being useful for some use cases. Additionally, the module was pretty complicated, and the bash shell code was quite hard to read and maintain. As such, the module has been rewritten using Nushell and a new configuration format.

Changes

It is now possible to enable multiple Flatpak repositories.
- Previously setting the system or user repository in the module overrode the last one set.
The ability to do local modifications to the Flatpak install list was removed.
- This niche feature made the module more complex and mixed up the purpose of modules, which is to set up the image's default configuration and not to provide configuration to end-users.
The ability to configure automatic removal of Flatpaks was removed.
- This is considered an anti-feature, as setting up automatic removal of a Flatpak that an end-user relies on would cause unexpected nuisance to the user. There is no way to automatically differentiate between Flatpaks that were manually installed by the user and are part of their workflow, and Flatpaks that were installed by the maintainer of the image that are now unneeded.
A new CLI tool called bluebuild-flatpak-manager has been added to let end-users easily turn off and on the automatic Flatpak setup and to manually execute Flatpak setup.
The new configuration format is based on a list of configurations: which are appended to /usr/share/bluebuild/default-flatpaks/configuration.yaml.
- This means that all Flatpak configuration done in the base image of the image and in previous module calls are retained, and each module call can do their own configuration without being affected by others.

Usage

Most users will probably only need one Flatpak setup configuration. The most common use case is to install Flatpaks from the Flathub repository. To do this, you can adapt from the following configuration:

type: default-flatpaks
configurations:
  - install:
      - org.mozilla.firefox
      - com.github.tchx84.Flatseal
      - io.github.flattool.Warehouse
      - io.missioncenter.MissionCenter
      - com.github.rafostar.Clapper
      - org.gnome.Loupe

By default, these will be user Flatpaks, and the installer will notify the user on each boot. You can change this by setting the configuration options as follows:

type: default-flatpaks
configurations:
  - notify: false
    scope: system
    install:
      - org.mozilla.firefox
      - com.github.tchx84.Flatseal
      - io.github.flattool.Warehouse
      - io.missioncenter.MissionCenter
      - com.github.rafostar.Clapper
      - org.gnome.Loupe

To specify custom repo details, use the repo: key in the configuration:

type: default-flatpaks
configurations:
  - notify: false
    scope: system
    repo:
      url: https://flathub.org/beta-repo/flathub-beta.flatpakrepo
      name: flathub-beta
      title: Flathub Beta
    install:
      - org.mozilla.firefox
      - com.github.tchx84.Flatseal
      - io.github.flattool.Warehouse
      - io.missioncenter.MissionCenter
      - com.github.rafostar.Clapper
      - org.gnome.Loupe

How to Migrate

To migrate to the new configuration format, simply create the top-level configurations: key and copy the configurations from under user: and system: to be under it under it. Remove the user: and system: keys, and set the scope: key to user or system as appropriate. Then, if you have configured repositories, move the repo- keys under the repo: section and remove the repo- prefix from them.

How to not Migrate

Your current configuration is likely set to use the latest version of the module with type: default-flatpaks. To switch to the now-deprecated older version of the module, set type: default-flatpaks@v1.

Introducing: the dnf module

Sun, 27 Apr 2025 00:00:00 GMT

When BlueBuild was created, Fedora Atomic only used rpm-ostree to build images, install packages, and manage installed systems. rpm-ostree is basically a mash-up of libostree and libdnf, which was made to handle composing OSTree images and managing OSTree-based systems, while existing in the RPM ecosystem.

However, focus slowly shifted to enabling bootable native container images based on the OCI standard, allowing the usage of standard tools such as docker or buildah to build images. This lessened the need for rpm-ostree as a separate build tool for the images, since it was now just called from the OCI container build process, just like any other package manager. Additionally, the lackluster UX and potentially confusing architecture were starting to show, and for example, dependency resolution had issues not present in dnf.

This made the Fedora project shift towards replacing rpm-ostree with dnf5, which would be updated to handle package management on OSTree-based systems, and bootc, which would handle deploying the bootable container images alongside system administration tasks such as updating and switching base images. Since F41, bootc and dnf5 have been included in all official images and are now the main way to build and manage Atomic systems, replacing rpm-ostree.

The `dnf` module

For the past few months, the BlueBuild team has been working on a new module called dnf that works as a repository and package management wrapper for dnf5 (and dnf4 if dnf5 isn't available). The configuration structure for the module has been designed from the ground up to be intuitive and featureful. New features compared to the rpm-ostree module are, for example, package group installation, installing packages from specific repositories, and managing COPR repositories.

The module is a part of our new batch of modules that are written in Nushell, a modern shell that is built on top of the Rust programming language. This approach makes the modules more maintainable and the source code more readable. You can read more about this change and decision process in the GitHub Issue and possibly a future blog post.

And yes, it can install Steam (this was a headache with rpm-ostree because of dependency resolution issues).

type: dnf
repos:
  files:
    - https://negativo17.org/repos/fedora-steam.repo
install:
  packages:
    - install-weak-deps: false
      packages:
        - steam
        - kernel-modules-extra

Switching from `rpm-ostree` to `dnf`

To switch to the dnf module, switch the module type from rpm-ostree to dnf. Make sure to check out the module reference page to see how to use the new module. Below is a short list of the main differences between the two modules.

Changes to the repos: section
- The rpm-ostree module used to have a repos: section that was a list of repo files.
- The dnf module's repos: section supports adding repositories from files and COPR directly, adding repository keys, and choosing a nonfree repository (rpmfusion or negativo17).
- The section repos: files: in the dnf module is equivalent to the repos: section in the rpm-ostree module.
- The proper way to add COPR repositories is to use the repos: copr: section in the dnf module.
Changes to the install: and remove: sections
- The rpm-ostree module used to have install: and remove: sections that were lists of packages to install/remove.
- The dnf module's install: and remove: sections support setting flags for the package manager.
- The section install: packages: in the dnf module is equivalent to the install: section in the rpm-ostree module.
- The list of packages also supports including sublists of packages to install from a specific repository or with specific flags.
Changes to the replace: section
- While the syntax remains the same, the replace: section in the dnf module supports setting flags for the package manager.
- The from-repo: parameter now takes the name of the repository to replace from, and the repository has to be added earlier in the repos: section.
New sections
- The group-install: and group-remove: sections are new to the dnf module.

New CLI Features & Breaking Changes — v0.9.0

Fri, 06 Dec 2024 00:00:00 GMT

:::caution[TLDR] The BlueBuild CLI v0.9.0 update introduces breaking changes. Show me what you broke! :::

The BlueBuild CLI is the tool that builds all BlueBuild custom images. It is also installed on every image built with BlueBuild and can be used to build and tinker with images locally. Lately we (gmpinder) have been hard at work on a new CLI version to automate many administrative tasks you have to do as a custom image creator. But enough of the fluff, let's get right to it!

How do I update?

:::tip[Wait a moment...] It's recommended that you read the rest of the article before updating, but if you're busy, here's the instructions. :::

Updates to the BlueBuild GitHub Action automatically update the CLI version as well. Your repository should also get an update PR for the new Action version thanks to Dependabot. If the Dependabot PR hasn't landed yet, though, you can manually delve into build.yml and make the following update.

jobs:
  bluebuild:
    steps:
      - name: Build Custom Image
-       uses: blue-build/github-action@v1.7
+       uses: blue-build/github-action@v1.8

Voilà! That was easy.

And if you're not using GitHub, just update wherever you might be specifying the CLI version to the latest one. With local builds, a simple rebuild should be enough to get the latest CLI version baked into your image at /usr/bin/bluebuild.

New Features

Generate ISOs for a fresh install

You can now generate offline ISO installation images as shown in the how-to guide. Under the hood this is a wrapper of JasonN3's build-container-installer. Feel free to report issues to the CLI repository first, and to the upstream project only if the issue is deemed to be caused by them.

Quickstart a new BlueBuild project from the command line

You can now use the bluebuild new and bluebuild init commands to start a new BlueBuild project from the command line. This will set up a new local git repository for you based on the template and some short configuration questions. (new creates a project in a new directory, init in the current empty one)

Here's a sneak peek:

❯ bluebuild new ./weird-os
✔ What would you like to name your image? · weird-os
✔ What is the registry for the image? (e.g. ghcr.io or registry.gitlab.com) · ghcr.io
✔ What is the name of your org/username? · octocat
✔ Write a short description of your image: · this is my weird custom OS :3
✔ Are you building on Github or Gitlab? · Github
Private key written to cosign.key
Public key written to cosign.pub
[main (root-commit) 994d18c] chore: Initial Commit
 11 files changed, 355 insertions(+)
 create mode 100644 .github/dependabot.yml
 create mode 100644 .github/workflows/build.yml
 create mode 100644 .gitignore
 create mode 100644 LICENSE
 create mode 100644 README.md
 create mode 100644 cosign.pub
 create mode 100644 files/scripts/example.sh
 create mode 100644 files/system/etc/.gitkeep
 create mode 100644 files/system/usr/.gitkeep
 create mode 100644 modules/.gitkeep
 create mode 100644 recipes/recipe.yml
INFO  => Created new BlueBuild project in ./weird-os

Image rechunking for more efficient diffs and updates

You can now enable rechunking for all your ostree-based custom images to make your resulting images more efficient at the cost of some build time. Rechunking is done using github.com/hhd-dev/rechunk, thanks to all the contributors for this enhancement!

To enable rechunk on GitHub, add rechunk: true to the parameters given to the BlueBuild GitHub Action. Locally and elsewhere, it's just a --rechunk flag for the bluebuild build command. Read more on the reference page.

If you're regularly making changes to your image and waiting to see if the build is successful, enabling rechunking will make the wait time slower. Feel free to not enable rechunk if you'd rather optimize build time than user bandwidth.

Build stages for compiling programs from source while keeping your image clean

You can define stages with a new top-level stages: key in the recipe before the modules:. Stages use the same module system as the rest of the build, but you'll probably find the script module most useful.

After building something in a stage, you can use the newly stable copy module to copy files from the stage. This maps directly to a COPY statement in the Containerfile.

Miscellaneous

A new login command for logging into registries for all tools like docker, skopeo, cosign, etc.
A new validate command and check before building that will tell you where you have errors in your recipes
A new prune command to easily clean build space for the build drivers
The sigstore driver can now handle signing if you don't have cosign installed
A new switch command consolidates upgrade/rebase commands
The ability to build multiple recipes at the same time

Breaking Changes

Related to custom modules

Internal changes and refactors always might affect your custom modules, but we try to be transparent when making such changes and announce them, so that you aren't left with broken builds without any apparent reason.

Removal of `yq`

We no longer include yq by default in images built with BlueBuild, and have refactored all modules to use jq instead (as such, modules using get_json_array are now also broken on older CLI versions). This comes in tandem with our experimentation with using Nushell to build some new modules, and a concern of yq being included unconditionally regardless of if it's actually needed in an image, increasing attack surface. Read more about this from the related GitHub issue.

What this means for you:

Reading module configuration with yq and get_yaml_array will break with the new update
You now have to use jq and get_json_array instead

Here's how you should refactor your module code:

#!/usr/bin/env bash

# read a single variable from the configuration
- VAR=$(echo "$1" | yq -I=0 '.var')
+ VAR=$(echo "$1" | jq -r 'try .["var"]')
echo "$VAR"

# read an array from the configuration
- get_yaml_array ARRAY '.array[]' "$1"
+ get_json_array ARRAY 'try .["array"][]' "$1"

Read more on the updated custom module how-to guide.

New way to use a local modules

Local modules must now have the source property set to local. Local modules are those that live inside the same repository as your custom image, inside the ./modules/ folder. This change allows us to validate recipes properly while not causing errors when custom modules are used.

modules:
    # use the module `./modules/custom-module/`
    - type: custom-module
      source: local
      input:
          - value 1
          - value 2

Phasing out official support for blue-build/legacy-template

With the switch to jq and get_json_array as talked about above, the 'legacy template', aka the old 'startingpoint', is starting to become ever more broken. As such, we will no longer be updating the template to accommodate for new changes. The repository is now archived, and a message is left in the README.

If you're one of the few people who still run their image builds from a repository based on it, and you are still not considering migration to the new system, you shall be crowned the new absolute ruler of your repository, in charge of keeping the builds running without someone telling you what to do.

Some things that might help you on your quest are our guide to running BlueBuild modules from a Containerfile directly and plain-old reading the source code of BlueBuild's CLI, especially the file containing the bash exports for modules and the Containerfile template used by BlueBuild images.

Recipe validation strictness

Recipe validation, which runs by default on every build, might break your build if you include the key of an object or array in your recipe, but do not populate it with anything. This is because the recipe parser fails to see the type of the parameter as an object or an array, and fails with an incorrect type error. In this case, you either need to delete the whole configuration key, or alternatively declare the value to either be {} (an empty object) or [] (an empty array).

For example:

modules:
    # This will fail validation
    - type: default-flatpaks
      user:
          install:
              - org.mozilla.firefox
          remove:
      system:

    # This will validate
    # (no Flatpaks will be removed, but the system Flathub repo will be set up)
    - type: default-flatpaks
      user:
          install:
              - org.mozilla.firefox
          remove: []
      system: {}

    # But you can also just leave out the `remove:` intirely
    # without changing what the module does
    - type: default-flatpaks
      user:
          install:
              - org.mozilla.firefox
          remove:
      system: {}

Miscellaneous

Minimum supported version of buildah is now 1.29, up from 1.24.

Final words

Happy holidays, everyone! Have fun with the new features, and enjoy your free time whenever you have the chance. Stay tuned for the eventual v1.0.0!

Discuss this post:

Place your system configuration files in /etc/

Sun, 11 Aug 2024 00:00:00 GMT

Hello BlueBuilders!

In this blog post, I will explain why the usage of /etc/ is now recommended over /usr/etc/ when building custom images of atomic Fedora.

We, and many others in the community have recommended previously that the system configuration files in custom images should be placed in /usr/etc/, while files in /etc/ are reserved for local-users. While it is true that in run-time the /usr/etc/ directory contains the original configuration of the image, this is technically undefined behavior build-time.

Technically, files written to /etc/ in the custom image are what populates /usr/etc/ on a system rebased to that image.

It is thanks to a recent discovery by the Universal Blue folks & the Rechunk developer Antheas that ostree has different behavior regarding /usr/etc/ & /etc/ in build-time vs in run-time.
Thanks to the RedHat employee & ostree maintainer Colin Walters who directly notified Universal Blue folks of this.

Why to transition

When files are placed to /usr/etc/ in build-time, currently there are no issues & everything works as expected for now.

However, it is still not the ideal thing to do, since we are relying on undefined rpm-ostree behavior, which may stop working some time in the future. Quoting Antheas:

When OSTree commits get converted to OCI, /usr/etc/ becomes /etc/. Then during deployment, new /etc/ files are silently moved to /usr/etc/.

Relying on /usr/etc/ to place modified files in customized packages is a hack and relies on undefined behavior in the rpm-ostree source code that performs the path rewriting.

(Take a note that we are utilizing an OCI image, so this part can be ignored: When OSTree commits get converted to OCI, /usr/etc/ becomes /etc/.)

As can be seen from the 1st paragraph of the note above, this is how ostree functions & this is why it's desired to transition to /etc/. /etc/ files in the image are copied to /usr/etc/ during deployment, so local-users will experience no changes.

How to transition

It is highly recommended that you follow & complete The Caching Update & Updating the directory structure blog post updates
If you're working with a 1-to-1 filetree mapping structure, where you copy files & folders to the root (/) folder, then do the following (source-folder is the folder defined in source key inside files module recipe entry):

Move files from files/system/usr/etc/ to files/system/etc/ (or replace system with the name of the specific directory)

If you copy files to the image in other ways, then any files that go to /usr/etc/ should be copied to go to /etc/ according, including files copied by custom scripts.
You're good to go! Commit your changes and wait to see that your custom image builds correctly.

Updating the directory structure

Sun, 21 Jul 2024 00:00:00 GMT

A previous update split the original config/ directory into two parts, config/ and the newly created recipes/, because we realized that it did not make sense to include files that will be copied into the image and configuration files used for generating the Containerfile (the lower-level build-instructions) in the same directory anymore.

This update left us at a weird state, though, because the directory that no longer contained image configuration, just files to be copied into the image, was called config/. We didn't want to just rename config/ to files/, though, because that would have created the weirdly named directory files/files/ used by the files module (very confusing).

What we opted to in addition to renaming config/ to files/ is giving the files module access to the entire files/ directory. This keeps the convention of each module reading files from a folder named after itself, while unlocking better ways to organize files when using the files module.

This is a small and non-critical change, and existing configurations will continue to be supported for the time being. Our small team of volunteers is constantly working on improving the UX of BlueBuild and slowly integrating new features. We still have multiple similar refactors planned to increase the general cohesion of the whole system, so stay tuned to see us evolve.

And now the part you're here for...

Migration

Make sure you've done the caching update migration that introduced the recipes/ folder.
Rename your config/ directory to files/.
If you're already working with a 1-to-1 filetree mapping structure, where you have root folders with names such as usr or etc that are copied into the image (as is the default), do the following:
1. Move all such folders into the directory files/system/
2. Change your files module configuration (or just the part that copied those folders) to below:
```
type: files
files:
    - source: system
      destination: /
```
  (notice that this is also using the new explicit source/destination syntax added in the same update)
If you're using the files module to copy other kinds of folders outside of the 1-to-1 filetree mapping for example using a copying a folder called wallpapers into the directory containing the default wallpapers in your environment, you may add those folders directly inside the files/ directory and copy them to the correct place similarly to the snippet above.
You're good to go! Commit your changes and wait to see that your custom image builds correctly.

The Caching Update

Wed, 01 May 2024 00:00:00 GMT

:::tip[TLDR] A recent BlueBuild CLI update makes builds and updates much faster. What do I need to do to take advantage? :::

Hello! I'm Jerry and I'm the creator and maintainer of the Rust-based bluebuild CLI tool. I started creating this tool as a way to extend the work that XYNY did with the recipe standard, but make it so that I didn't have to manage the Containerfile directly. The tool is an integral part of the project that handles Containerfile generation from the recipe, the building of container images, and pushing them to the registry.

For the past month or so, we've been working on trying to make the building experience better for our users. One of the biggest concerns brought up was the size of updates when using rpm-ostree upgrade to pull your new changes. The size of the updates would end up being bigger than the original single RUN build.sh instruction that was used. While this was really great for the time, more advanced uses of the legacy startingpoint repo required users to know how to manage a Containerfile. This can be really intimidating to less-technical users. One of the bluebuild CLI's goals is to perform most of the optimizations for the user so they don't have to worry about setting that up.

Migration

I'm happy to announce that we have now released v0.8.4 of bluebuild CLI (and the accompanying v1.4.0 of the GitHub Action) which completes the last set of changes needed to allow users to quickly iterate on changes to their build to try new changes without having to wait for long build times or large downloads. The change requires:

Moving all of your recipe files into the ./recipes/ directory in the root of your repo instead of ./config/.
- This does not include all the other files/directories in your ./config/ like ./config/scripts/ or ./config/files/
- Any .yml/.yaml file that contains information for the modules to build your image are your recipe files.
- Be sure to update any use of from-file: to account for any changes in your directory structure
  - For example if the recipes were in ./config/recipes/ and extended recipe files in ./config/recipes/image_configs/.
  - One of the recipes calls from-file: recipes/image_configs/common.yml.
```
config
└─recipes
  ├── image_configs
  │   ├── common.yml
  │   ├── gnome.yml
  │   └── kde.yml
  ├── recipe-gnome-nvidia.yml
  ├── recipe-gnome.yml
  ├── recipe-kde-nvidia.yml
  └── recipe-kde.yml
```
  - You could move recipes up a level to be:
```
config
recipes
├── image_configs
│   ├── common.yml
│   ├── gnome.yml
│   └── kde.yml
├── recipe-gnome-nvidia.yml
├── recipe-gnome.yml
├── recipe-kde-nvidia.yml
└── recipe-kde.yml
```
  - Then from-file: can have the path set to image_configs/common.yml
- Be sure to update your workflow file too
  - If your recipe.yml was located in ./config/ and you moved it to ./recipes/, the GHA will already take that into account
  - The thing to look out for is the recipe's relative location to the "root" of recipe locations
It's also suggested to move the ./config/containerfiles/ directory into the root of your repo too.
- You can see the docs for the containerfile module here.
Update your GHA to be at least v1.4

Wait, what's wrong with how it is now?

When you build an image with docker, buildah, or podman the layers in your Containerfile are kept on your system to be re-used by other images or when you rebuild the image so that you don't have to rebuild layers that did not change. BlueBuild interfaces with these tools to build your images. BlueBuild takes in your recipe file and translates that into Containerfile instructions. The instructions are created in the order you define in your recipe. Assuming your base image hasn't pushed an update, when you make a change to an instruction in your Containerfile, the image will only be built from the changed instruction forward. If you're using COPY to copy files into your image, any changes to those COPY'd files in your repo will cause the build to start re-building from that point.

The problem was that the recipe files were being stored in ./config/ which also happened to contain all of your other build files which may or may not have changed. This means that when you made a change to your recipe file, it would end up marking ./config/ (which is copied into a stage and mounted on each RUN instruction) as changed as well as updating the corresponding RUN instructions in the Containerfile. The mount indicates to the builder that it is dependant on the new changes found in ./config/ and so every single instruction is run again on that change.

How does this update solve the cache problem?

The recipe file should only be seen as a way to define how to create the Containerfile and the parameters that are passed into the module. It shouldn't be COPY'd into the build. By moving the recipe files into ./recipes/ it removes the build's dependency on file changes in ./config/ from recipe updates and will now only change the Containerfile itself allowing the builder to only run the instructions that changed.

How does this cache persist in a GitHub Action runner?

Docker has an experimental feature that allows builds to interface directly with the GitHub Action cache system. We've setup our docker driver in BlueBuild and our GHA to make use of this feature. This means that BlueBuild will handle this for you.

This sounds great, but I need to see numbers

During my testing and work on this feature in the CLI, I attempted testing this out by making a change on the second to last module I defined in my recipe. Theoretically, this would mean the only layers that would need to be built would be those last ones.

Here's a snippet of my recipe before the change.

name: jp-laptop
description: The image of Wunker OS for JP's Laptop.
base_image: ghcr.io/ublue-os/bazzite
image_version: "39"
modules:
    # ...
    - type: script
      scripts:
          - setup-selinux-dockersock.sh
    - type: script
      scripts:
          - setup-kubectl.sh
    - type: rpm-ostree
      repos:
          - https://pkg.earthly.dev/earthly.repo
          - https://cli.github.com/packages/rpm/gh-cli.repo
      install:
          - earthly
          - neovim
          - helix
          - alacritty
          - gh
    - type: script
      scripts:
          - install-mkcert.sh
          - install-codelldb.sh

And here's the change. I'm removing neovim cause I'm a filthy helix user.

name: jp-laptop
description: The image of Wunker OS for JP's Laptop.
base_image: ghcr.io/ublue-os/bazzite
image_version: "39"
modules:
    # ...
    - type: script
      scripts:
          - setup-selinux-dockersock.sh
    - type: script
      scripts:
          - setup-kubectl.sh
    - type: rpm-ostree
      repos:
          - https://pkg.earthly.dev/earthly.repo
          - https://cli.github.com/packages/rpm/gh-cli.repo
      install:
          - earthly
          - helix
          - alacritty
          - gh
    - type: script
      scripts:
          - install-mkcert.sh
          - install-codelldb.sh

So we re-run the build.

 => CACHED [stage-config 1/1] COPY ./config /config                                                                                                       0.0s
 => CACHED [stage-modules 1/2] COPY --from=ghcr.io/blue-build/modules:latest /modules /modules                                                            0.0s
 => CACHED [stage-modules 2/2] COPY ./modules /modules                                                                                                    0.0s
 => CACHED [stage-keys 1/1] COPY cosign.pub /keys/jp-desktop-gaming.pub                                                                                   0.0s
 => CACHED [stage-4  2/16] RUN --mount=type=bind,from=stage-keys,src=/keys,dst=/tmp/keys   mkdir -p /usr/etc/pki/containers/   && cp /tmp/keys/* /usr/et  0.0s
 => CACHED [stage-bins 1/3] COPY --from=ghcr.io/sigstore/cosign/cosign /ko-app/cosign /bins/cosign                                                         0.0s
 => CACHED [stage-bins 3/3] COPY --from=ghcr.io/blue-build/cli:main-installer /out/bluebuild /bins/bluebuild                                              0.0s
 => CACHED [stage-4  3/16] RUN --mount=type=bind,from=stage-bins,src=/bins,dst=/tmp/bins   mkdir -p /usr/bin/   && cp /tmp/bins/* /usr/bin/   && ostree   0.0s
 => CACHED [stage-4  4/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4  5/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4  6/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4  7/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4  8/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4  9/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4 10/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4 11/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4 12/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4 13/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => CACHED [stage-4 14/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind  0.0s
 => [stage-4 15/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind,from=  33.4s
 => [stage-4 16/16] RUN   --mount=type=tmpfs,target=/var   --mount=type=bind,from=stage-config,src=/config,dst=/tmp/config,rw   --mount=type=bind,from=s  0.7s

As you can see above, all preceding layers are cached and the last 2 layers are re-built. Performing an upgrade for this ends up only pulling these last layers.

$> rpm-ostree upgrade
Pulling manifest: ostree-image-signed:docker://registry.gitlab.com/wunker-bunker/wunker-os/jp-laptop
Importing: ostree-image-signed:docker://registry.gitlab.com/wunker-bunker/wunker-os/jp-laptop (digest: sha256:01073d98adf4041f3b91840af1135b15aee97db90de6bc3a3846d67c07345e6a)
ostree chunk layers already present: 65
custom layers already present: 16
custom layers needed: 2 (255.0 MB)

So all I need to do is move my recipe files?

Yes. The vast majority of the work to get these benefits are done on our end. All you need to do is move those files.

Do I HAVE to make this change right now?

No you don't. We will continue to support the legacy method for the time being.

Final words

Hopefully I was able to explain how these changes will help you in your building ventures. We'll be working more on creating useful features as well as optimizing your building experience.

Happy building!

Introducing BlueBuild

Sun, 25 Feb 2024 00:00:00 GMT

:::note If you're not totally sure what this is about, check out the front page. :::

Per a mutual decision, Universal Blue's old custom image tooling has now been transferred to the BlueBuild org and development will be continuing under the BlueBuild project with basically the same team of maintainers and developers as before. The issue was discussed extensively in ublue-os/startingpoint#223 and eventually voted for in ublue-os/main#476.

We've been working on BlueBuild for a month now to provide you a smooth migration and exciting new features, so don't worry, this change is positive.

To briefly summarize, this desire to split stemmed from a difference in philosophy and scope between the main Universal Blue maintainers and the developers of 'startingpoint'. Since most of the Universal Blue project's build systems use classic cloud methodologies like Containerfiles and GitHub Actions directly to build their images, the abstraction introduced with recipes in 'startingpoint' might have seemed unnecessary. Additionally, I felt that as a subproject of Universal Blue, this project couldn't really achieve its full potential.

The repositories affected by this transition are:

ublue-os/startingpoint (transferred to blue-build/legacy-template)
ublue-os/bling (transferred to blue-build/modules)
ublue-os/images-website (transferred to blue-build/images-website)

:::note BlueBuild is based on the idea of recipes as abstractions for the image build. Universal Blue might in the future provide official documentation for custom image creation using Containerfiles directly. See the FAQ to learn more about the difference. :::

History lesson...

This section gives a bigger picture of the developments in the history of this project from my perspective. Jump to the What's new? section to jump straight to the important stuff.

When the Universal Blue organization was starting out, the base was just a single repository: ublue-os/base. This repository never built more than a single Silverblue image, and was easily forked into a new image by a tinkerer wishing to use a different base image or set of packages. The repository also had a script for installing Flatpaks after the first boot, and in my first PR I added in a recipe.yml file for easier customization of the Flatpaks and RPM-packages by people tinkering the images. This was in February 2023, so a few months before the first commit to VanillaOS' Vib, which is a similar project that uses a slightly different recipe.yml format.

A while after, ublue-os/base was deprecated in favor of ublue-os/main. I got to keep working on the recipe.yml-based repository as ublue-os/startingpoint, which would now only serve the part of a template. Eventually it became possible to build a full image only by editing the recipe, with no need to delve into Containerfiles or GitHub Actions.

The recipe.yml format was still quite different than today, but after the rewrite in September 2023, the current modular system was established, and we started utilizing the ublue-os/bling repository as a source of modules instead of including them all in the template. This change made the project way more maintainable and extensible, and more abstracted.

The BlueBuild project is a direct continuation of this lineage, aiming to be an accessible way of building custom images of Linux distributions.

What's new?

The shiny new brand and website is probably what you'll notice first, but that isn't the only new thing to be excited about!

Recipe compiler

The recipe compiler turns your recipe into a Containerfile, freeing us from the restrictions that having the entire build system being based on a shell script ran during the image build caused. For example, it will now be possible to include Containerfile snippets from the recipe directly. The compiler lives as part of the blue-build/cli program, which is written in Rust, and also offers other features including building images and rebasing to them locally.

Our new re-usable GitHub Action and template repository use the compiler by default, so new users will get the benefits instantly. The new template requires less maintenance, as important parts of the build system have been moved to other repositories.

Keyless signing with OIDC and Sigstore

:::note This feature has currently been delayed due to lacking verification support in upstream projects. See blue-build/cli#84. :::

The keyless signing support with OIDC will make new repository setup easier and more straightforward by making the generation of a cosign / sigstore keypair optional. When signing without keys, the verification will be based on ephemeral keys bound to an OIDC identity from GitHub, making it possible to verify that the image you are using came from the specific GitHub Action, without specifying a public key. This will also shorten the first-time rebase process to one command, going directly onto the signed image, and eliminating the need to trust the registry on the first rebase.

New users and existing users alike should be able to take advantage of this soon. Once this feature is fully ready, a guide will be provided.

How to migrate?

With the important repositories having been moved, it is unfortunately required to do at least some migration. It is also possible to migrate to your old custom image to be built using the new recipe compiler, allowing you to make use of new and upcoming features. From the list below, choose your preferred migration path.

Upgrade your current repository (recommended)

Pros/cons

+ Keep git history and repository
+ Keep your existing configuration
- Requires some copy-pasting of workflow files

Instructions

Temporarily clone the new template repository to your computer.
Overwrite your old .github/ folder with the .github/ folder from the new template.
- If you're building multiple images, remember to copy your matrix of recipes from the old build.yml to the new build.yml.
- If you've made changes to build.yml, those have to be ported manually.
Remove irrelevant files from your repository.
- The Containerfile and build.sh are not used by the recipe compiler -based build system. If you've made any changes to those files, they have to be manually ported over.
- You might also want to add "/Containerfile" to the .gitignore file, because the recipe compiler generates a Containerfile when run locally.
- It is also recommended to clean out files that are related to Universal Blue community management (CODE_OF_CONDUCT.md and CONTRIBUTING.md), if you haven't changed them to contain information relevant to your custom image project.
Clean up your configuration.
- Some changes to the template defaults have been made in preparation for the migration, which make the template more minimal, and should be adopted by both old and new users.
- Remove the old signing.sh in favor of the signing module, which is essentially the same, but requires less maintenance from custom image creators. signing.sh is deprecated, and won't be supported any longer.
- If you use just, remove 100-bling.just and the import for it from 60-custom.just. The bling justfiles have been removed due to lack of maintenance, and all their features being covered by the justfiles included by default with Universal Blue images.
- If you don't use just, you can remove the whole config/files/usr/share/ublue-os/just/ folder from your repository. This is unneeded and no longer included in the templates by default.
Detach your custom image repository, which is most likely a fork of the upstream template.
- While this step isn't strictly required, it should still be done because your repository is no longer related to its old upstream template, and it now functions as though it was generated from the new template.
- This can be done through the GitHub Virtual Support Assistant.
Clean up the git branch structure by removing the template branch and renaming live to main.
- This step is optional, but recommended. The existing branch structure will continue to work as expected, but migration will make it more straightforward and syncing with the upstream template easier.
Move your recipe files into ./recipes/.
- Doing so will allow your builds to cache layers with the new system.
- The ./recipes/ directory will never be copied into the build context which allows changes to your recipe to only be reflected in the changed Containerfile. This way docker will only build starting at the layer that changed.
- Cached builds will only work if your base image hasn't updated yet, otherwise it will have to rebuild from the start.
- This is great for making tweaks to your recipe and testing it as your computer will only have to download the new layers.
You should be all set now! Try starting your builds if you haven't already.

Create a new repository (not recommended)

Pros/cons

+ Accidental breakage unlikely
+ Keep your existing configuration
- Lose git history
- Will change cosign keys
- You'll have two repositories

Instructions

Set up a new repository using the guide.
Make sure both your old and new repositories are cloned locally.
Copy relevant parts from your old repository to the new one.
- Everyone should copy at least the config/ and recipes/ directories, but you can also copy README.md and modules/, if you've made any changes to those.
- If you're building multiple images, remember to copy your matrix of recipes from the old build.yml to the new build.yml.
- If you've made changes to build.sh, build.yml, or the Containerfile, those have to be ported manually.
Clean up your configuration in the new repository.
- During the last month some changes to the template defaults have been made in preparation for the migration, which make the template more minimal, and should be adopted by both old and new users.
- Remove the old signing.sh in favor of the signing module, which is essentially the same, but requires less maintenance from custom image creators. signing.sh is deprecated, and won't be supported any longer.
- If you use just, remove 100-bling.just and the import for it from 60-custom.just. The bling justfiles have been removed due to lack of maintenance, and all their features being covered by the justfiles included by default with Universal Blue images.
- If you don't use just, you can remove the whole config/files/usr/share/ublue-os/just/ folder from your repository. This is unneeded and no longer included in the templates by default.
Add a message to the README of your old repository pointing to the new one, disable the workflows (either in the GitHub UI or by removing build.yml), and archive it.
Fix ghcr.io: denied: permission_denied: write_package errors in your new repo.
- See the troubleshooting page.
You should be all set now! Try starting your builds if you haven't already.

Do nothing

It is also possible to keep building a custom image based on the legacy template / build system as it will be supported for the foreseeable future. A quick migration process is still needed to prevent your custom image builds from breaking after 90 days (from 2024-02-25), when GitHub cleans up the old module repository image from the ublue-os GitHub organization.

Pros/cons

+ Keep your existing configuration and whatever customizations you might have made to the build system
+ Only a quick migration process needed to keep receiving updates to modules
- You won't get new features that depend on the recipe compiler

Instructions

Either sync up with the upstream template or manually change the occurrences of ublue-os/bling in the Containerfile to blue-build/modules.
- On the latest version of the template, there should be just one on line 31.
Clean up your configuration (optional, recommended).
- During the last month some changes to the template defaults have been made in preparation for the migration, which make the template more minimal, and should be adopted by both old and new users.
- Remove the old signing.sh in favor of the signing module, which is essentially the same, but requires less maintenance from custom image creators. signing.sh is deprecated, and won't be supported any longer.
- If you use just, remove 100-bling.just and the import for it from 60-custom.just. The bling justfiles have been removed due to lack of maintenance, and all their features being covered by the justfiles included by default with Universal Blue images.
- If you don't use just, you can remove the whole config/files/usr/share/ublue-os/just/ folder from your repository. This is unneeded and no longer included in the templates by default.
Migration complete! You can now resume chilling.

Discussion

This post can be discussed on GitHub Discussions: https://github.com/orgs/blue-build/discussions/10

This change was also announced over on the Universal Blue forums: https://universal-blue.discourse.group/t/universal-blues-old-custom-image-tooling-has-been-migrated-to-bluebuild/669

Be sure to join our community spaces and interact in a healthy way with others: https://blue-build.org/community/

BlueBuild Blog

default-flatpaks v2

Changes

Usage

How to Migrate

How to not Migrate

Introducing: the dnf module

The dnf module

Switching from rpm-ostree to dnf

New CLI Features & Breaking Changes — v0.9.0

How do I update?

New Features

Generate ISOs for a fresh install

Quickstart a new BlueBuild project from the command line

Image rechunking for more efficient diffs and updates

Build stages for compiling programs from source while keeping your image clean

Miscellaneous

Breaking Changes

Related to custom modules

Removal of yq

New way to use a local modules

Phasing out official support for blue-build/legacy-template

Recipe validation strictness

Miscellaneous

Final words

Place your system configuration files in /etc/

Why to transition

How to transition

Updating the directory structure

Migration

The Caching Update

Migration

Wait, what's wrong with how it is now?

How does this update solve the cache problem?

How does this cache persist in a GitHub Action runner?

This sounds great, but I need to see numbers

So all I need to do is move my recipe files?

Do I HAVE to make this change right now?

Final words

Introducing BlueBuild

History lesson...

What's new?

Recipe compiler

Keyless signing with OIDC and Sigstore

How to migrate?

Upgrade your current repository (recommended)

Pros/cons

Instructions

Create a new repository (not recommended)

Pros/cons

Instructions

Do nothing

Pros/cons

Instructions

Discussion

The `dnf` module

Switching from `rpm-ostree` to `dnf`

Removal of `yq`