Enabling Local User Login with Active Directory Authentication using Kerberos
Introduction
CIQ had a customer issue, where the customer was attempting to set up local users and then authenticate using Kerberos
to their Active Directory
server. The following guide will explain how this was resolved.
Problem
The customer was unable to authenticate via ssh
to their Rocky Linux 9.2 LTS node using a local user account stored on the node. No passwords were locally saved in the /etc/shadow
file and the passwords were only available on the Active Directory
server.
The customer's configuration of sssd
, Kerberos
and Active Directory
had worked without issue on CentOS 7.9, CentOS 8.1 and Rocky Linux 8.5. It was only on Rocky Linux 9.2 LTS where the customer was facing issues.
Symptoms
When attempting to log in to the server via ssh
, the following authentication messages were observed under /var/log/secure
:
Dec 9 11:07:31 <server_name> sshd[1155]: Received signal 15; terminating.
Dec 9 11:07:31 <server_name> sshd[79752]: Server listening on 0.0.0.0 port 22.
Dec 9 11:07:31 <server_name> sshd[79752]: Server listening on :: port 22.
Dec 9 11:10:04 <server_name> unix_chkpwd[79934]: check pass; user unknown
Dec 9 11:10:04 <server_name> unix_chkpwd[79934]: password check failed for user (<username>)
Dec 9 11:10:04 <server_name> sshd[79919]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=<ip_address> user=<username>
Dec 9 11:10:44 <server_name> sshd[79919]: pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=<ip_address> user=<username>
Dec 9 11:10:44 <server_name> sshd[79919]: pam_sss(sshd:auth): received for user <username>: 4 (System error)
Account retrieval errors could also be seen via the /var/log/sssd/sssd_pam.log
and /var/log/sssd/sssd_nss.log
files:
[root@<server_name> ~]# cat /var/log/sssd/sssd_pam.log
(2024-12-16 13:45:58): [pam] [server_setup] (0x3f7c0): Starting with debug level = 0x0070
(2024-12-16 13:47:20): [pam] [cache_req_common_process_dp_reply] (0x3f7c0): [CID#1] CR #3: Could not get account info [<account_number>]: SSSD is offline
[root@<server_name> ~]# cat /var/log/sssd/sssd_nss.log
(2024-12-16 13:45:58): [nss] [server_setup] (0x3f7c0): Starting with debug level = 0x0070
(2024-12-16 13:47:20): [nss] [cache_req_common_process_dp_reply] (0x3f7c0): [CID#5] CR #17: Could not get account info [<account_number>]: SSSD is offline
Resolution
The customer was utilizing id_provider = files
in their /etc/sssd/sssd.conf
file. Since the Rocky Linux 9.x series, this is a deprecated option in sssd.
Instead, they added the following settings into their sssd.conf
file:
id_provider = proxy
krb5_server = example.com:88
proxy_lib_name = files
Then restarted the sssd
service:
systemctl restart sssd`
The customer's sssd.conf
file ultimately looked like the following (all sensitive customer information has been obfuscated):
[sssd]
services = nss, pam
domains = EXAMPLE.COM
default_domain_suffix = example.com
[domain/EXAMPLE.COM]
id_provider = proxy
proxy_lib_name = files
auth_provider = krb5
krb5_realm = EXAMPLE.COM
krb5_server = example.com:88
krb5_validate = false
krb5_ccachedir = /var/tmp
krb5_keytab = /etc/krb5.keytab
[nss]
filter_groups = root,<local user>
filter_users = root,<local user>
In addition, the customer's /etc/kr5b.conf
file:
includedir /etc/krb5.conf.d/
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm=EXAMPLE.COM
dns_lookup_realm = true
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
default_ccache_name = KEYRING:persistent:%{uid}
[realms]
EXAMPLE.COM = {
kdc = example.com:88
admin_server = example.com:749
}
example.com = {
kdc = example.com:88
admin_server = example.com:749
}
[domain_realm]
example.com = EXAMPLE.COM
example.com = EXAMPLE.COM
[appdefaults]
pam = {
debug = true
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}
Furthermore, the customer's /etc/nsswitch.conf
file:
# In order of likelihood of use to accelerate lookup.
shadow: files
hosts: files dns myhostname
aliases: files
ethers: files
gshadow: files
# Allow initgroups to default to the setting for group.
# initgroups: files
networks: files dns
protocols: files
publickey: files
rpc: files
Troubleshooting notes
Adding debug_level = 9
to each section of the sssd.conf
file, yields more verbose logs under /var/log/sssd/*
When troubleshooting the issue, the following logs are essential:
/var/log/sssd/*
/var/log/secure
In addition, these files are also important to check:
/etc/kr5b.conf
/etc/nsswitch.conf
/etc/pam.d/password-auth
/etc/pam.d/system-auth
/etc/sssd/sssd.conf
Next are these command outputs to run:
getent passwd <username>
klist
kinit <username>
ssh -o PubkeyAuthentication=no -vvv -k <username>@<server_name>
Finally, generating an sosreport
from the node.