Error When Symlinking Subuid And Subgid

Stephen
Customer Support Engineer

Nov 13, 2024

Introduction

In certain environments, managing user namespace mappings across multiple hosts is essential for consistent access control and permissions. A common approach is to symlink /etc/subuid and /etc/subgid files to a shared storage location to easily distribute these mappings. However, this approach can lead to issues with tools like Apptainer that utilize user namespaces, as they rely on newuidmap and newgidmap utilities that do not support symlinks. This article outlines the problem, details the root cause, and offers alternative solutions to achieve synchronization across hosts without symlinking.

Problem

To simplify the management of /etc/subuid and /etc/subgid, you may attempt to symlink these files to a shared storage. This approach is intended to ensure consistent mapping across multiple hosts.

Symptoms

When attempting to build with Apptainer in fakeroot mode while using symlinked subuid and subgid files, you may encounter the following error:

# apptainer build --fakeroot rockylinux_fakeroot.sif docker://rockylinux:9
ERROR  : 'newgidmap' execution failed. Check that 'newgidmap' is setuid root or has setcap cap_setgid+eip.
ERROR  : Error while waiting event for user namespace mappings: no event received

Root Cause

This issue occurs because newuidmap and newgidmap utilities do not support symbolic links. When these programs attempt to read the subuid and subgid files, they fail due to the O_NOFOLLOW flag in the openat system call, which prevents following symbolic links. The following strace output illustrates this behavior:

openat(AT_FDCWD, "/etc/subgid", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_NOFOLLOW) = -1 ELOOP (Too many levels of symbolic links)

The O_NOFOLLOW flag, as specified in the open man page, causes the call to fail with an ELOOP error if the final path component is a symbolic link, which is what happens here.

Resolution

To synchronize subuid and subgid files across hosts without using symlinks, consider alternative methods such as:

Rsync and Cron Jobs: Use rsync in conjunction with a cron job to periodically sync these files across hosts. For details, refer to the Rocky Linux Rsync Guide and Rocky Linux Cron Guide.
Ansible Automation: Automate file synchronization with Ansible for a more flexible and scalable solution. Ascender, an orchestration tool from CIQ, can assist in managing these automation tasks. For more details, refer to the Rocky Linux Ansible Guide and Ascender Automation.

References & related articles

Rocky Linux Rsync Guide [https://docs.rockylinux.org/guides/backup/rsync_ssh/]
Rocky Linux Cron Guide [https://docs.rockylinux.org/guides/automation/cron_jobs_howto/]
Rocky Linux Ansible Guide [https://docs.rockylinux.org/books/learning_ansible/02-advanced/]
Ascender Automation [https://ciq.com/products/ascender/]