SELinux/Debugging

How to debug SELinux issues

This page is currently a draft.

Install packages useful for debugging

$ sudo dnf -y install setools-console selinux-policy-devel policycoreutils-newrole strace initscripts-service bzip2

Enable full auditing

For performance reasons, full auditing is not enabled by default. Instructions how to enable it:

1. Open the /etc/audit/rules.d/audit.rules file in an editor.

2. Remove the following line if it exists:

-a task,never

3. Add the following line to the end of the file:

-w /etc/shadow -p w

4. Restart the audit daemon using the legacy service command, or reboot the system:

$ sudo service auditd restart

5. Run the scenario which effects in SELinux denials.

6. Collect AVC denials:

$ sudo ausearch -i -m avc,user_avc,selinux_err,user_selinux_err -ts today

Setting up confined users

Create new users assigned to particular SELinux users

Create 4 new Linux users assigned to a corresponding SELinux user:

PWD=${PWD-"my_p4ss-w0rd"}
for username in guest xguest user staff
do
  adduser -Z ${username}_u ${username}
  echo "${PWD}" | passwd --stdin "${username}"
done

Assign a SELinux user to an existing Linux user

Change SELinux user for a Linux user:

$ sudo semanage login -a -s staff_u existinguser
$ sudo semanage login -l
Login Name           SELinux User         MLS/MCS Range        Service
...
existinguser         staff_u              s0-s0:c0.c1023       *

Assign a SELinux user an additional role

Allow the staff user to access the dbadm role which is not allowed to access by default.

$ sudo semanage user -l
                Labeling   MLS/       MLS/
SELinux User    Prefix     MCS Level  MCS Range                      SELinux Roles
...
staff_u         user       s0         s0-s0:c0.c1023                 staff_r sysadm_r system_r unconfined_r
$ sudo semanage user -m -R "staff_r sysadm_r system_r unconfined_r dbadm_r" staff_u
$ sudo semanage user -l
                Labeling   MLS/       MLS/
SELinux User    Prefix     MCS Level  MCS Range                      SELinux Roles
...
staff_u         user       s0         s0-s0:c0.c1023                 staff_r sysadm_r system_r unconfined_r dbadm_r

Assign admin roles to Linux users when they use sudo

For running commands using sudo, sudo can be configured so that the user id changes as well as the SELinux role and the corresponding type.

$ sudo cat > /etc/sudoers.d/admin-roles << EOF
# staff can become sysadm for all commands and shell
staff        ALL=(ALL)       ROLE=sysadm_r TYPE=sysadm_t NOPASSWD: ALL
# staff2 can only run networking commands
#Cmnd_Alias NETWORKING = /sbin/route, /sbin/ifconfig, /bin/ping, /sbin/dhclient, /usr/bin/net, /sbin/iptables, /usr/bin/rfcomm, /usr/bin/wvdial, /sbin/iwconfig, /sbin/mii-tool
#staff2        ALL=(ALL)       ROLE=sysadm_r TYPE=sysadm_t NOPASSWD: NETWORKING
# staff3 can become dbadm for databases administration
#CMND_Alias DATABASES = /usr/bin/mariadb-admin /usr/bin/mysqladmin /usr/bin/psql
#staff3        ALL=(ALL)       ROLE=dbadm_r TYPE=dbadm_t NOPASSWD: DATABASES
EOF

Switch the system to SELinux permissive mode

For testing purposes and to gather as many denials as possible, the permissive mode is useful not to be blocked in actual work.

Open the /etc/selinux/config file in an editor, change the SELINUX=enforcing line to

SELINUX=permissive

and reboot the system. After tests finish, switch the system back to enforcing.

For a one-time change to permissive, execute

$ sudo setenforce 0

This setting will be valid till the next reboot, or till setenforce 1 was executed.

Advanced debugging

In many cases it is helpful to track the failing syscalls or the kernel code path through which the denial occurs. For that, use the strace and perf commands, or use the kernel's tracing support to get the kernel (or even userspace) backtraces for SELinux denials.

Install additional tools and debugging information for affected packages (systemd in this example).

$ sudo dnf -y install dnf-utils strace perf
$ debuginfo-install "systemd*"

Using perf to trace denials from a command

Install perf and debugging information for the involved package and its libraries. Execute a command through perf:

$ sudo perf record -o perf.data -a -g --call-graph dwarf -e avc:selinux_audited -- /path/command -options

The perf.data file with information about audited denials caused by the command is stored.

Using perf to trace all system denials

Install perf and debugging information for the involved package and its libraries. Execute perf and terminate it when the scenario is finished:

$ sudo perf record -o perf.data -a -g --call-graph dwarf -e avc:selinux_audited
^C

The perf.data file with information about all denials audited during the perf command run is stored.

Interpret data gathered by perf

Run perf script to read stored data and display stack trace:

$ sudo perf script -i perf.data

Run perf report to display call chains:

$ sudo perf report -g

Refer to: perf-record(1)

Using tracefs

1. Run the following commands as root:

# echo stacktrace >/sys/kernel/tracing/trace_options
# echo 1 >/sys/kernel/tracing/events/avc/selinux_audited/enable

2. Run the scenario which triggers the SELinux denials.

3. Dump the backtraces of captured AVC events to report them:

# cat /sys/kernel/tracing/trace
[...]

4. (Optional) Reset the tracing settings by running the following commands (or just rebooting the machine):

# echo nostacktrace >/sys/kernel/tracing/trace_options
# echo 0 >/sys/kernel/tracing/events/avc/selinux_audited/enable