Warewulf Restart Required When Editing Configuration Files.
Introduction
In the Warewulf 4.x series and higher, whenever any changes are made to nodes.conf or warewulf.conf, the warewulfd systemd service has to be restarted in order to properly reflect the changes.
Problem
Nodes may not boot with the expected container image, despite following standard configuration and deployment steps.
Attempts to set the container through the wwctl node set command and rebuild overlays do not push the new image to the node.
Even deleting and re-adding the node using the correct container configuration does not resolve the issue.
A node will continue to boot with an old or default container image.
Verification using wwctl node list -a shows the correct container value set, however the new image is not actually being provisioned to the node.
The issue persists even after rebuilding the overlays and restarting the node.
Resolution
Direct edits to nodes.conf do not trigger a configuration reload by warewulfd. As a result, the daemon continues using the old cached configuration until it is explicitly restarted.
Any restart to warewulfd forces it to reload nodes.conf and warewulf.conf, after which the correct container will be applied.
Root cause
The root cause is due to the nodes.conf and warewulf.conf files being edited directly, instead of using the wwctl node set command.
If you run wwctl node set, this sends a signal to warewulfd to reload the configuration, but editing the file manually bypasses this mechanism, so it is recommended to use wwctl node set instead.
Additional troubleshooting steps for configuration changes not taking effect
-
Confirm container is set for the node:
sudo wwctl node set <NODE_NAME> --container=<CONTAINER_NAME> -
Verify the container assignment:
sudo wwctl node list -a <NODE_NAME> -
Rebuild the node configuration:
sudo wwctl configure node <NODE_NAME> -
Rebuild the overlays:
sudo wwctl overlay build -a -
Reboot the node:
sudo wwctl node power off <NODE_NAME>
sudo wwctl node power on <NODE_NAME> -
If unsuccessful, verify container is present and built:
sudo wwctl container list
sudo wwctl container show <CONTAINER_NAME> -
Optionally, delete and re-add the node:
sudo wwctl node delete <NODE_NAME>
sudo wwctl node add <NODE_NAME> --container=<CONTAINER_NAME> --ipaddr=<IP> -
Reconfigure and rebuild:
sudo wwctl configure node <NODE_NAME>
sudo wwctl overlay build -a -
Set profile and container explicitly if needed:
sudo wwctl node set <NODE_NAME> --profile=<PROFILE_NAME> --container=<CONTAINER_NAME> -
Final node reboot:
sudo wwctl node power off <NODE_NAME>
sudo wwctl node power on <NODE_NAME>