NixOS (5): The Lifecycle of a Change — Evaluate, Build, Activate and Verify

Motivation

#

The earlier posts mentioned nix flake check, nixos-rebuild build, test and switch. They all seem to be related to updating the system, but they do different things. Some check the configuration, some produce a system build, and some immediately change the running system.

Running sudo nixos-rebuild switch every time may work well enough for ordinary changes. When a build fails, a service will not start, or you are unsure whether an update needs a reboot, it helps to understand the steps separately.

Here I will work through a small example from editing the configuration to applying it. It should also make it easier to understand how far an AI assistant has progressed when helping with the system.

A Simple Example

#

Suppose there is already a working NixOS flake, with a host output named laptop and a module at hosts/laptop/configuration.nix. These are example names and paths; replace them with your own.

We will make one change: have NixOS create /etc/nixos-lifecycle-demo with the contents version=1. Assuming that file does not already exist, checking it will show when the configuration actually takes effect.

Editing the Configuration

#

Enter the repository and look for existing uncommitted changes:

1
2
3
4

cd /path/to/nixos-config
git status --short
git diff
git diff --cached

Add this setting inside the configuration attribute set—the configuration’s braces—in the existing host module:

1

environment.etc."nixos-lifecycle-demo".text = "version=1\n";

environment.etc declares files that NixOS manages under /etc. Here, nixos-lifecycle-demo is the filename, text supplies the contents, and \n adds a newline.

Saving the file does not create anything under /etc yet. The configuration has been edited, but has not been applied.

There is no need to update flake.lock for this example. Updating all dependencies while adding a setting is possible, but if something breaks, it becomes harder to tell which change caused it.

A local Git flake can read modifications to tracked files before they are committed. If you create a new module instead, import it and add it to Git’s index first, or Nix may not see the new file. The flake documentation explains the details.

Checking the Configuration

#

After editing, run:

1
2

git diff --check
nix flake check --no-build --show-trace --no-update-lock-file

The first command checks some formatting problems in the diff. The second reads the flake, combines modules and options, and checks evaluation. An incorrect option name or conflicting definitions may produce an error here.

The options mean:

• --no-build: do not build the flake’s declared checks;
• --show-trace: show tracing information on errors to help find their source;
• --no-update-lock-file: fail if this operation would require changing the lock file.

Without --no-build, the command also builds the checks; see the nix flake check reference. Evaluation may need to download missing inputs, so it is not necessarily offline or immediate.

A successful check means Nix can evaluate the checked configuration. Whether compilation succeeds and whether the service works are still to be determined.

Building the System

#

Build the laptop system:

1

nixos-rebuild build --flake .#laptop --no-update-lock-file

The . selects the current repository, and #laptop selects its host output. build produces the system without applying it, so activation privileges are not needed for this step.

After the build, a result symlink appears in the current directory. Inspect its target with:

1

readlink -f result

It points to a system directory in /nix/store. The system and all its dependencies make up its closure.

“Building the system” sounds as though it might compile the entire operating system again. Usually it does much less: existing store paths are reused, available cached outputs are downloaded, and only missing outputs need building. Generating a configuration file in this example is quite different from updating a kernel.

You can compare the package and closure changes with the current system:

1

nix store diff-closures /run/current-system ./result

This helps inspect package versions and closure sizes. It does not show a line-by-line diff of configuration files; for our example, the Git diff explains the change most clearly.

The build now exists, but /etc/nixos-lifecycle-demo still has not appeared on the running system. We have not activated it yet.

When a build takes a long time

#

The time needed depends on what changed. One kernel rebuild after the migration took over four hours and about 45 GiB at peak. An attempt failed near the end because it ran out of space. Ordinary configuration edits do not require that budget, but input updates can trigger more building than expected.

For a kernel or large dependency update, check free space first and allow enough uninterrupted time. AI can help read logs and monitor progress, but the compilation still needs time and storage.

Applying the Configuration

#

Trying it with test

#

Use test to try the configuration. First save the current system path in a variable, which we will use later if we need to restore it:

1
2

previous_system=$(readlink -f /run/current-system)
sudo nixos-rebuild test --flake .#laptop --no-update-lock-file

Now read the file:

1

cat /etc/nixos-lifecycle-demo

The expected output is:

1

version=1

The configuration is now active. NixOS manages the file, usually through a link to content in the store. Future changes should be made in the Nix configuration and applied again.

test activates immediately without changing the boot default. If you only try a configuration this way, rebooting normally returns to the previously configured boot default.

However, trying it still changes the running system. Services may restart, network changes may disconnect you, and programs may write data. There is no automatic undo after a few minutes, so remote network changes still need a way to recover access. An activation error can also occur after some changes have taken effect; inspect the error and the actual state first.

Keeping it with switch

#

Once the file has the expected contents:

1
2

sudo nixos-rebuild switch --flake .#laptop --no-update-lock-file
cat /etc/nixos-lifecycle-demo

switch applies the configuration and makes it the boot default. Future boots will use this setting too. Adding this file does not require a reboot before you can use it.

Keep the source and lock file unchanged between build, test and switch. The later commands evaluate and prepare the system again; they do not simply execute the earlier result. Unchanged inputs normally allow the outputs to be reused.

What boot does

#

To prepare a configuration for the next boot without applying it immediately:

1

sudo nixos-rebuild boot --flake .#laptop --no-update-lock-file

This sets the boot default but does not activate it. The four commands can be compared like this:

Preparing the system may involve reuse, cache downloads or building; it does not always mean compilation. The nixos-rebuild documentation also explains these commands.

When to Reboot

#

Our file can be used as soon as activation completes. A kernel or early-boot configuration change needs a reboot before it can be checked.

These two paths are useful:

1
2

readlink -f /run/current-system
readlink -f /run/booted-system

current-system is the currently activated configuration; booted-system is the configuration used for this boot. After switch in our example, they may differ until the next reboot. That is normal, and different paths do not always mean a reboot is necessary.

To see whether the difference involves the kernel:

1
2
3

readlink -f /run/current-system/kernel
readlink -f /run/booted-system/kernel
uname -r

Different system configurations can share a kernel, while different kernel builds can report the same version string. That is why uname -r alone does not tell the whole story.

After rebooting, check those paths again and look for failed services:

1
2

systemctl --failed --no-pager
systemctl --user --failed --no-pager

Run the second command in the intended user’s session. Then try the changed feature: read the file again for our example, connect to a network service, or use the device after a driver change. Having no failed services does not establish that every feature has been tested.

Recovering from a Bad Configuration

#

After test only

#

If the saved previous_system still points to an existing system, run this in the same shell:

1

sudo "$previous_system/bin/switch-to-configuration" test

It reactivates the saved system without changing the boot default. Check that the variable names the version you want, and do not collect the old system paths while testing.

Rebooting is another way to return to the boot default, although that may differ from the system saved in the variable.

After switch

#

List the retained generations:

1

sudo nixos-rebuild list-generations

If the previous profile generation is the version you want, roll back:

1

sudo nixos-rebuild switch --rollback

One detail is easy to mix up: test does not advance the system profile, so switch --rollback is not necessarily an undo of the last test. It may select an earlier generation. Choose the recovery action according to what you ran before.

If a new system cannot boot, select a retained older generation in the boot menu. Returning to an earlier kernel also requires booting that kernel.

Rollback does not edit the source files in Git. Fix or revert the offending configuration too, or the next rebuild may apply it again. Databases and user files are not restored by system rollback; use the appropriate data backups for those.

Cleaning Up Old Generations

#

After many updates, older systems can take up space. The generation count alone does not tell you how much: generations share dependencies, and other links such as result can retain store paths too.

Nix uses garbage-collection roots to determine which paths need to stay. Old generations are one source of references; after a generation is removed, paths are eligible for collection only when nothing else retains them. Removing a generation therefore may not free as much space as expected. See the garbage-collection documentation for details.

I suggest checking that the new system works before deciding which older generations to remove. Having the configuration in Git does not guarantee that rebuilding the old system later will be quick: downloads or compilation may be needed, and the original sources might no longer be available.

When AI helps maintain the system, I ask it to say which steps it performed, especially whether it activated the configuration or rebooted. That makes a message such as “the build succeeded” easier to interpret and leaves a useful starting point for the next session.

The next post will cover NixOS modules, options and overrides, and how the configuration files are combined.