How to Create a Local Mirror Using Reposync with CIQ Depot

Introduction

This guide explains how to synchronize repositories locally using reposync. It covers syncing both publicly available repositories and private repositories, such as those within CIQ Depot. This guide will provide you with two options for downloading packages depending on the complexity of your needs.

Problem

You need to create a local copy of a repository for an air-gapped environment, a network with limited or slow internet access, or to provide faster access to a local set of servers.

Prerequisites

A web server is required to serve the downloaded packages. You can follow our guide on how to set up a web server. When syncing repositories, you will also need to consider the storage requirements.

For smaller repositories that only feature updates such as our Long Term Support (LTS) repository, the overall size can be somewhere in the range of 1GB to 15GB or more in size.

The full repository for Rocky Linux or CentOS can potentially exceed 100GB in total size.

In this guide, the web server will have an IP address of 192.168.1.30.

Option 1: Sync on an existing server

If you are using a Rocky Linux server or another server using CIQ's CentOS Bridge service that is already configured for the repository you are looking to sync, you can leverage its existing setup to download the repository.

Identify the repository you want to download by running dnf repolist. The first column will provide you with the repo id that you will use in your reposync command.

In this example, CIQ is syncing using the repoid of rocky-lts-9.2.x86_64 attached to our CIQ LTS for Rocky Linux 9.2 repository.

dnf reposync -p repodata --download-metadata \
  --repoid=rocky-lts-9.2.x86_64 --download-path=/var/www/html/rocky-lts-9.2.x86_64/

If you are trying to sync our CentOS Bridge repository (requires a subscription to CIQ Depot) from a CentOS 7 server, the command is different:

reposync -p repodata --download-metadata \
   --repoid=ciq-bridge.x86_64 --download_path=/var/www/html/ciq-bridge/

The above commands will store the repository files in the local web server path /var/www/html/. You can also store these files on shared storage for centralized access or copy them to another server.

Option 2: Sync on a separate server

If you want to run this on a dedicated server or need to sync multiple repositories, consider creating a dedicated repo file(s) to manage the information.

Step 1: Create a repository configuration file

If you plan to sync CIQ Depot repositories, you can obtain the necessary repo information from the following location:

1. Navigate to My Products.

2. Select the repository you would like to sync.

3. Click the button with the three dots on the right-hand side of the repository name.

4. Select "DNF Repo Config".

For our example, we will use the publicly available Rocky Linux 9 BaseOS repository. We will create a new repository file named rocky-baseos-9.repo in our root directory. Feel free to save this in a more permanent location.

[rocky-baseos-9]
name=Rocky Linux 9 - BaseOS
baseurl=http://dl.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/
gpgcheck=1
enabled=1
countme=1
metadata_expire=6h
gpgkey=https://dl.rockylinux.org/pub/rocky/RPM-GPG-KEY-Rocky-9

The repo id is specified at the top of the file between the [] characters. In this case, [rocky-baseos-9].

Repeat the above step to create separate files for each repository you plan to sync.

Alternatively, you can add multiple repositories to a single file, ensuring each has a unique repo id.

Step 2: Run reposync

Once the configuration file is created, you can run the reposync command as before. Only now you will specify the repo file like in the below example:

dnf -c ./rocky-baseos-9.repo reposync -v -p repodata \
--download-metadata --repoid=rocky-baseos-9 \
--download-path=/var/www/html/rocky-baseos-9/

Repeat this step for each file and/or repository you intend to sync ensuring that you change the file, repo id, and folder as needed.

Step 3: Verify and configure clients

After all downloads are complete, confirm the repository is accessible via your web browser.

In the below example, our repository is located at http://192.168.1.30/rocky-baseos-9/rocky-baseos-9/.

Once you’ve verified that the packages are being served correctly, configure Rocky Linux to use the repository by updating the appropriate file in /etc/yum.repos.d/.

As illustrated below, since we are serving the BaseOS repository, we will modify /etc/yum.repos.d/rocky.repo.

[baseos]
name=Rocky Linux $releasever - BaseOS
#mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=BaseOS-$releasever$rltype
baseurl=http://192.168.1.30/rocky-baseos-9/rocky-baseos-9/
gpgcheck=1
enabled=1
countme=1
metadata_expire=6h
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Rocky-9

Notes

Automate reposync

There are several ways to automate fetching the latest packages via reposync.

For a complete, centralized solution, we recommend running a playbook within Ascender.

Alternatively, you can set up a simple cronjob to pull the latest packages automatically. For example, you might create /etc/cron.d/reposync in the example provided:

# Run the hourly jobs
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
@daily root dnf -c /root/rocky-baseos-9.repo reposync -v -p repodata --download-metadata --repoid=rocky-baseos-9 --download-path=/var/www/html/rocky-baseos-9/

This will run the cronjob daily at midnight. Add a new line for each repository you intend to sync.

For more granular control of cronjob timing, please consult this documentation on cron jobs.

Delete old packages on sync

When running reposync, older packages will not be deleted by default.

This means your local repository will contain all versions of packages, as newer versions are released.

This is helpful if you desire having older packages available that are not retained in the upstream repository. However, this can take up significant storage space.

To delete packages locally that are no longer present on the remote server, append --delete to the reposync command like so:

dnf -c ./rocky-baseos-9.repo reposync -v -p repodata \
--download-metadata --repoid=rocky-baseos-9 \
--download-path=/var/www/html/rocky-baseos-9/ \
--delete

Although the old RPM files remain in the repository folders, they won’t be accessible via DNF on remote systems.

This is because the metadata is copied from upstream.

To retain these preexisting RPMs, update the metadata using a tool like createrepo.

References & related articles

How to Set Up a Web Server in Rocky Linux
Rocky Linux Cronjob Guide
DNF RepoSync Options
DNF RepoManage Options