ArticlesRocky Linux

Setting Up Rsyslog with Encrypted Transport on Rocky Linux

Introduction

This guide focuses on troubleshooting issues encountered during or after setting up rsyslog for encrypted log transport on Rocky Linux. It addresses common problems such as missing modules and misconfigured parameters that can prevent proper decryption of incoming logs. Whether you’re building a centralized logging infrastructure or enhancing your lab environment’s security, this guide provides hands-on insights into troubleshooting and correct configuration.

Problem

When an rsyslog collector is configured to receive logs from multiple systems and intended to secure the log transport using encryption, errors indicating required modules are missing or misconfigured can be encountered. This can cause scenarios where logs are received on the collector, but are not being decrypted correctly, halting proper secure communication.

Symptoms

The rsyslog configuration can produce several notable errors:

Module loading issue

could not load module 'gtls', errors: trying to load module /usr/lib64/rsyslog/gtls.so: /usr/lib64/rsyslog/gtls.so: cannot open shared object file: No such file or directory

Parsing errors in the configuration file

error during parsing file /etc/rsyslog.d/01-collector.conf, on or before line 9: parameter 'StreamDriver.AuthMode' not known -- typo in config file? 
error during parsing file /etc/rsyslog.d/01-collector.conf, on or before line 9: parameter 'StreamDriver.Mode' not known -- typo in config file? error during parsing file /etc/rsyslog.d/01-collector.conf, on or before line 9: parameter 'StreamDriver.Name' not known -- typo in config file?

Even though logs are being received, they remain encrypted and unprocessed, indicating a configuration or module issue.

Resolution

Below are the prerequisites and detailed steps to resolve the issue:

Testing

  • Utilize the root account or a user with sudo privileges for testing.

  • Set up two Rocky Linux 8.10 machines: one designated as the server and the other as the client.

  • Set the hostname for each node with an FQDN:

    • Server: hostnamectl set-hostname rocky-linux810-server.rsyslog.test

    • Client: hostnamectl set-hostname rocky-linux810-client.rsyslog.test

  • Update the /etc/hosts file on both nodes with the respective IP addresses and hostnames. An example is below (please replace the IPs and hostnames with your own):

cat << "EOF" | tee /etc/hosts
10.0.0.4    rocky-linux810-server.rsyslog.test  rocky-linux810-server
10.0.0.3    rocky-linux810-client.rsyslog.test  rocky-linux810-client
EOF

Package installation

On both the server and client, install the necessary packages using the following command:

dnf install -y rsyslog rsyslog-gnutls rsyslog-openssl logrotate

Firewall and SELinux adjustments

  • Open port 6514/tcp on both nodes:
firewall-cmd --add-port=6514/tcp --permanent
firewall-cmd --reload
  • On the server, allow port 6514 through SELinux:
semanage port -m -t syslogd_port_t -p tcp 6514

Certificate generation

  • Create a directory for certificates:
mkdir -p /etc/rsyslog/keys
  • Generate a self-signed certificate and private key:
openssl req -new -x509 -days 365 -nodes \
  -out /etc/rsyslog/keys/server-cert.pem \
  -keyout /etc/rsyslog/keys/server-key.pem

Directory setup for log storage

  • Create directories for logs on both the server and client:
# Server
mkdir -p /data/logs/rocky-linux810-server

# Client
mkdir -p /data/logs/rocky-linux810-client
  • Set appropriate permissions for each directory:
# Server
chmod 644 /data/logs/rocky-linux810-server

# Client
chmod 644 /data/logs/rocky-linux810-client
  • Configure SELinux contexts:
# Server
semanage fcontext -a -t var_log_t "/data/logs/rocky-linux810-server(/.*)?"
restorecon -Rv /data/logs/rocky-linux810-server

# Client
semanage fcontext -a -t var_log_t "/data/logs/rocky-linux810-client(/.*)?"
restorecon -Rv /data/logs/rocky-linux810-client

Correcting the module issue

  • The key resolution is to ensure the gtls module is correctly referenced. Since the file /usr/lib64/rsyslog/gtls.so module is missing, a symbolic link needs to be created:
ln -s /usr/lib64/rsyslog/lmnsd_gtls.so /usr/lib64/rsyslog/gtls.so

Server configuration (/etc/rsyslog.d/01-collector.conf)

  • Create/update the configuration file with the following content:
cat << "EOF" | tee /etc/rsyslog.d/01-collector.conf
# /etc/rsyslog.d/01-collector.conf

module(load="imtcp")
module(load="gtls")

$DefaultNetstreamDriver gtls

$DefaultNetstreamDriverCAFile   /etc/pki/tls/certs/ca-bundle.crt
$DefaultNetstreamDriverCertFile /etc/rsyslog/keys/server-cert.pem
$DefaultNetstreamDriverKeyFile  /etc/rsyslog/keys/server-key.pem

$InputTCPServerRun 6514

$template RemoteLogs,"/data/logs/%HOSTNAME%/%PROGRAMNAME%.log"
*.* ?RemoteLogs
& ~
EOF

Client configuration (/etc/rsyslog.d/01-sender.conf)

  • Create/update the configuration file on the client with:
cat << "EOF" | tee /etc/rsyslog.d/01-sender.conf
module(load="omfwd")
module(load="gtls")

$DefaultNetstreamDriverCAFile /etc/pki/tls/certs/ca-bundle.crt

$DefaultNetstreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode anon

*.* action(type="omfwd"
           target="10.0.0.4"
           port="6514"
           protocol="tcp")
EOF

Service restart and testing

  • Restart rsyslog on both the server and client:
systemctl restart rsyslog
  • Test by sending an encrypted message from the client:
logger "THIS IS A TEST ENCRYPTED MESSAGE"
  • On the server, verify the decrypted log messages:
tail -f /data/logs/rocky-linux810-client/root.log
  • An example output:
tail -f /data/logs/rocky-linux810-client/root.log
[root@rocky-linux810-server ~]# tail -f /data/logs/rocky-linux810-client/root.log
Apr 30 07:26:07 rocky-linux810-client root[53589]: THIS IS A TEST ENCRYPTED MESSAGE
Apr 30 07:26:07 rocky-linux810-client root[53589]: THIS IS A TEST ENCRYPTED MESSAGE
Apr 30 07:26:08 rocky-linux810-client root[53590]: THIS IS A TEST ENCRYPTED MESSAGE

Root Cause

The core issue stems from a missing symbolic link for the gtls module. The rsyslog configuration is attempting to load the module from /usr/lib64/rsyslog/gtls.so, which does not exist. Instead, the system provides the library as /usr/lib64/rsyslog/lmnsd_gtls.so. Creating the symbolic link resolves the module-not-found error and allows the proper configuration directives—such as StreamDriver.Name, StreamDriver.Mode, and StreamDriver.AuthMode—to be recognized and processed, thereby enabling successful decryption of incoming logs.

Notes

  • SELinux and Firewall: Always ensure that SELinux contexts and firewall rules are appropriately configured, as they can block the secure transport on non-standard ports like 6514.

  • Module Naming and Configuration: Module names and parameters can vary between rsyslog releases. It’s advisable to consult the official rsyslog documentation when encountering similar issues.

  • Certificate Management: For production environments, replace self-signed certificates with those issued by a trusted CA and ensure proper certificate management policies are in place.

References & Related Articles

Rsyslog Error 2066 - Missing gtls Module
Rsyslog Error 2207 - Unknown Parameter Issues
Rsyslog gtls Network Stream Driver