How to Create a Local Mirror Using Reposync with CIQ Depot

Introduction

This guide explains how to synchronize repositories locally using reposync. It covers syncing both publicly available repositories and private repositories, such as those within CIQ Depot. This guide will provide you with two options for downloading packages depending on the complexity of your needs.

Problem

You need to create a local copy of a repository for an air-gapped environment, a network with limited or slow internet access, or to provide faster access to a local set of servers.

Prerequisites

A web server is required to serve the downloaded packages. You can follow our guide on how to set up a web server. When syncing repositories, you will also need to consider the storage requirements.

For smaller repositories that only feature updates such as our Long Term Support (LTS) repository, the overall size can be somewhere in the range of 1GB to 15GB or more in size.

The full repository for Rocky Linux or CentOS can potentially exceed 100GB in total size.

In this guide, the web server will have an IP address of 192.168.1.30.

Option 1: Sync on an existing server

If you are using a Rocky Linux server or another server using CIQ's CentOS Bridge service that is already configured for the repository you are looking to sync, you can leverage its existing setup to download the repository.

Identify the repository you want to download by running dnf repolist. The first column will provide you with the repo id that you will use in your reposync command.

In this example, CIQ is syncing using the repoid of rocky-lts-9.2.x86_64 attached to our CIQ LTS for Rocky Linux 9.2 repository.

dnf reposync -p repodata --download-metadata \
  --repoid=rocky-lts-9.2.x86_64 --download-path=/var/www/html/rocky-lts-9.2.x86_64/

If you are trying to sync our CentOS Bridge repository (requires an active subscription) from a CentOS 7 server, the command is different:

reposync -p repodata --download-metadata \
   --repoid=ciq-bridge.x86_64 --download_path=/var/www/html/ciq-bridge/

The above commands will store the repository files in the local web server path /var/www/html/. You can also store these files on shared storage for centralized access or copy them to another server.

Option 2: Sync on a separate server

If you want to run this on a dedicated server or need to sync multiple repositories, consider creating dedicated repo files to manage the information.

Step 1: Create a repository configuration file

For CIQ repositories, the depot client will write the repo file for you. On a mirror host that already has the client installed and logged in with depot login, start by listing what your organization has access to:

depot list
depot list rlc-pro-9

Then generate the config for the product you want:

depot enable rlc-pro-9 --print > /root/repos/rlc-pro-9.repo

Use --print. Without it, the file goes straight into /etc/yum.repos.d/ and the product is marked enabled, so your mirror host starts installing from the repos you only meant to copy. With --print, the config goes to stdout instead. The filename it would have used goes to stderr, so a plain redirect gives you a clean file.

To take part of a product rather than all of it, name the repos with --repo:

depot enable rlc-pro-9 --print --repo=rlc-9-core.x86_64 --repo=rocky-9-appstream.x86_64 > /root/repos/rlc-pro-9.repo

The same config is in the Customer Portal if you would rather not use the CLI. Pick the product, installation method, version, and architecture, then open the Reposync tab under Installation Instructions for a .repo file and a matching dnf reposync command.

Two things to know about the file before you sync it.

Most of the repositories in it are disabled. A generated RLC Pro 8 file holds 37 repositories and enables 6: BaseOS, AppStream, Extras, and the CIQ Core, Pro and Supplemental repos. Devel, PowerTools or CRB, high availability, resilient storage, NFV, Plus and RT are all off, as is a .source and -debug pair for each. reposync mirrors the enabled ones only, which is a sensible baseline. Add any others at sync time with --enablerepo=rocky-8-powertools.x86_64.

The file also holds your organization's credentials. chmod 600 it and keep it out of git.

For our example, we will use the publicly available Rocky Linux 9 BaseOS repository. We will create a new repository file named rocky-baseos-9.repo in our root directory. Feel free to save this in a more permanent location.

[rocky-baseos-9]
name=Rocky Linux 9 - BaseOS
baseurl=http://dl.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/
gpgcheck=1
enabled=1
countme=1
metadata_expire=6h
gpgkey=https://dl.rockylinux.org/pub/rocky/RPM-GPG-KEY-Rocky-9

The repo id is specified at the top of the file between the [] characters. In this case, [rocky-baseos-9].

Repeat the above step to create separate files for each repository you plan to sync.

Alternatively, you can add multiple repositories to a single file, ensuring each has a unique repo id.

Step 2: Run reposync

Once the configuration file is created, you can run the reposync command as before. Only now you will specify the repo file like in the below example:

dnf -c ./rocky-baseos-9.repo reposync -v -p repodata \
--download-metadata --repoid=rocky-baseos-9 \
--download-path=/var/www/html/rocky-baseos-9/

Repeat this step for each file and/or repository you intend to sync ensuring that you change the file, repo id, and folder as needed.

Step 3: Verify and configure clients

After all downloads are complete, confirm the repository is accessible via your web browser.

In the below example, our repository is located at http://192.168.1.30/rocky-baseos-9/rocky-baseos-9/.

Once you’ve verified that the packages are being served correctly, configure Rocky Linux to use the repository by updating the appropriate file in /etc/yum.repos.d/.

As illustrated below, since we are serving the BaseOS repository, we will modify /etc/yum.repos.d/rocky.repo.

[baseos]
name=Rocky Linux $releasever - BaseOS
#mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=BaseOS-$releasever$rltype
baseurl=http://192.168.1.30/rocky-baseos-9/rocky-baseos-9/
gpgcheck=1
enabled=1
countme=1
metadata_expire=6h
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-9

Additional Information

Mirroring more than one major version from a single host

If you are building a patching baseline for 8, 9, and 10, one mirror server can hold all three. reposync only cares about the baseurl in the repo file, so a Rocky 9 host has no trouble pulling 8 and 10 content. Generate a repo file per major version:

/root/repos/rlc-pro-8.repo
/root/repos/rlc-pro-9.repo
/root/repos/rlc-pro-10.repo

Generating them with the CLI has one catch. depot enable checks a product's repositories against the OS ID, OS version, and architecture of the host you run it on, and if none of them match you get a 404 rather than a config file:

# depot enable rlc-pro-8
ERROR: failed to enable product: could not obtain dnf repos: failed to get dnf configuration:
bad status code from server: 404: Not Found: no compatible dnf repositories found,
skipped 63 incompatible repos

That is the version check failing. The mirror host's own version (9, here) goes into the request, so every repo in the 8 product gets skipped. Override it with --osversion:

depot enable rlc-pro-8  --osid=rocky --osversion=8  --print > /root/repos/rlc-pro-8.repo
depot enable rlc-pro-9  --osid=rocky --osversion=9  --print > /root/repos/rlc-pro-9.repo
depot enable rlc-pro-10 --osid=rocky --osversion=10 --print > /root/repos/rlc-pro-10.repo

Architecture works the same way. Add --arch when the content is not for the architecture the mirror host runs on:

depot enable rlc-pro-8  --osid=rocky --osversion=8  --arch=aarch64 --print > /root/repos/rlc-pro-8-aarch64.repo
depot enable rlc-pro-9  --osid=rocky --osversion=9  --arch=aarch64 --print > /root/repos/rlc-pro-9-aarch64.repo
depot enable rlc-pro-10 --osid=rocky --osversion=10 --arch=aarch64 --print > /root/repos/rlc-pro-10-aarch64.repo

From there one reposync per version does the work:

#!/bin/bash
set -uo pipefail
rc=0

for ver in 8 9 10; do
  conf=/root/repos/rlc-pro-${ver}.repo
  dnf -c "${conf}" --setopt=reposdir= reposync --download-metadata \
    --download-path="/var/www/html/rocky/${ver}/x86_64/" \
    || { echo "sync failed for ${ver}" >&2; rc=1; }
done

exit "${rc}"

--setopt=reposdir= is the part not to drop. dnf -c replaces the main config but leaves reposdir pointing at /etc/yum.repos.d/, so without it the mirror host's own repos join the sync and you get Rocky 9 packages in your 8 tree.

Leave --repoid off. reposync already limits itself to the repos the file enables, and --repoid is a shortcut for --disablerepo="*" --enablerepo=<repoid>, which brings back whatever you name regardless of the enabled setting. --enablerepo is the flag for adding one.

reposync writes a subdirectory per repo id, so the served path lands at /var/www/html/rocky/10/x86_64/rlc-10-core.x86_64/. Keep the versions in separate trees and clients using $releasever resolve correctly.

Rocky 10 ships dnf5, where --download-path is --destdir. Check dnf reposync --help on a 10 host before reusing the command above.

Confirm your subscription covers all three versions first. If it does not, you get an authentication failure rather than the 404 above, which looks similar and is a different problem.

Avoid --newest-only with --download-metadata

--newest-only is the obvious way to keep a mirror small, and the RHEL 7 era scripts most people are migrating from used it. Paired with --download-metadata it produces a repository that does not work.

The metadata comes from upstream unchanged, so it still lists the older packages --newest-only skipped. Clients that resolve to one of those get a 404 for a package the metadata promised. The man page says as much: "When the option is used with --newest-only, only latest packages will be downloaded, but metadata will still contain older packages."

Either sync everything and leave the metadata alone, or use --newest-only without --download-metadata and build the metadata yourself with createrepo_c afterwards. For a patching baseline, sync everything. The disk is cheaper than the upkeep.

Migrating from a reposync and createrepo script

RHEL 7 era scripts usually ran createrepo after reposync, because reposync did not fetch the upstream metadata:

sudo reposync --newest-only --gpgcheck -l --repoid=${repo} --download_path=${rootpath}
sudo createrepo --update -v ${rootpath}/${repo}

On Rocky 8 and later, --download-metadata handles that, so the createrepo line goes away for a straight mirror. Drop --newest-only at the same time, for the reason covered above. You will still want createrepo_c if you change what is in the local repository after syncing, such as keeping older RPMs that upstream has dropped (see Delete old packages on sync).

Automate reposync

There are several ways to automate fetching the latest packages via reposync.

For a complete, centralized solution, we recommend running a playbook within Ascender.

Alternatively, you can set up a simple cronjob to pull the latest packages automatically. For example, you might create /etc/cron.d/reposync in the example provided:

# Run the hourly jobs
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
@daily root dnf -c /root/rocky-baseos-9.repo reposync -v -p repodata --download-metadata --repoid=rocky-baseos-9 --download-path=/var/www/html/rocky-baseos-9/

This will run the cronjob daily at midnight. Add a new line for each repository you intend to sync.

For more granular control of cronjob timing, please consult this documentation on cron jobs.

Delete old packages on sync

When running reposync, older packages will not be deleted by default.

This means your local repository will contain all versions of packages, as newer versions are released.

This is helpful if you desire having older packages available that are not retained in the upstream repository. However, this can take up significant storage space.

To delete packages locally that are no longer present on the remote server, append --delete to the reposync command like so:

dnf -c ./rocky-baseos-9.repo reposync -v -p repodata \
--download-metadata --repoid=rocky-baseos-9 \
--download-path=/var/www/html/rocky-baseos-9/ \
--delete

For a baseline you freeze per patch cycle, leave --delete off. The package you are most likely to need mid-cycle is the previous build of something that just shipped broken, and once upstream drops it your mirror is the only copy left. --delete removes it on the next sync.

Although the old RPM files remain in the repository folders, they won’t be accessible via DNF on remote systems.

This is because the metadata is copied from upstream.

To retain these preexisting RPMs, update the metadata using a tool like createrepo.

Notes

• reposync keeps every package version by default. Pass --delete to prune packages no longer upstream, but leave it off for a baseline you freeze per patch cycle.
• Client-visible metadata is copied from upstream, so RPMs you keep locally are not served over DNF until you rebuild the metadata with createrepo_c.
• Pair --newest-only with --download-metadata and the mirror breaks: the metadata still lists the older packages that were skipped.

References & related articles

How to Set Up a Web Server in Rocky Linux
Rocky Linux Cronjob Guide
DNF RepoSync Options
DNF RepoManage Options