Features/Securecontainers

Feature Name

Create a tool virt-sandbox-server that will allow an administrator to easily create a LXC Container to run the an application server, with libvirt and SELinux locking it down. This tool is part of the virt-sandbox package introduced in Fedora 17.

Summary

This change allows an administrator to run multiple services on the same machine each service in a secure Linux Container. virt-sandbox-service takes an systemd unit file and sets up all of the mountpoints and libvirt container information to run the service unit within the container. Then libvirt will be able to launch the container with a SELinux context that will prevent the containers process from interacting with other process on the system including other containers. This could allow an administrator to run multiple web servers simultaneously each with their own data and sharing the system data, even running as root, but prevent them from breaking the host machine or other processes on the system.

Owner

• Name: Daniel J Walsh
• Name: User:berrange: Daniel Berrange

• Email: dwalsh@redhat.com
• Email: berrange@redhat.com

Current status

• Targeted release: [Fedora 18]
• Last updated: Thu Aug 1 2012
• Percentage of completion: 80%
  • libvirt changes going into Rawhide end of week, currently in libvirt upstream
  • libvirt-sandbox-0.0.3-3.fc18 currently in Fedora/Rawhide
    • Continuing to be developed and tested.
  • kernel fixes for connecting applications to a container still being developed.
  • Policy changes ongoing.
    • Continuing to be developed and tested.

Detailed Description

libvirt as of Fedora 16 has the ability to run processes within a container, the problem with this is the processes can run as root and can easily break out of the container. Also it is fairly difficult to setup one of these containers to run a particular service, for example apache.

We want to make the processes of setting up these containers easier and to use SELinux and containers together to lock down the processes running within the container, meaning you could run multiple customers on a host and prevent them from attacking each other or the host.

Benefit to Fedora

Linux Containers is a low level way of doing virtualization, and allows you to run multiple copies of the same service at the same time on a system. It has some advantages over full virtualization as it does not have to wait for an entire system to boot, it can use less memory and can use the base OS, in a read/only manner.

Scope

This change effects

• SELinux Policy writers
  • New policy has to be written to define what a confined application container is allowed to do
• Kernel and SELinux toolchain changes
  • One problem we have with the way the kernel currently labels files, is that MCS portion of the label is applied based off the process level not the level of the directory that the file was create in. This means when a process running as unconfined_t:s0 creates a file in a directory labeled svirt_lxc_file_t:s0:c1,c2, the file will be created as svirt_lxc_file_t:s0 and the container will not be able to use the file, until the administrator changed the label.

This feature requires that the kernel support new policy construct to allow the policy writer to specify whether the MCS Label is inherited from the parent directory MCS Label or from the creating process MCS Label. kernel-3.3 should have this feature. (Completed in Current Kernel

• libvirt-sandbox package introduced in Fedora 17
  • virt-sandbox-service script used for creating/starting/stopping Secure Containers
    • Create
      • Create an image file or a directory on disk and populate the image/directory with the default content based on the rpm data associated with the systemd unit file.
      • It will also create the virt-sandbox database used by libvirt-sandbox to create a libvirt-lxc container.
      • Setup systemd content within the container so that systemd will start the correct services
      • Also create a systemd unit file so the administrator of the host machine can start and stop the service containers just like any other servers.
    • Start
      • Convert the Linux-sandbox configuration into libvirt-lxc configuration and tell libvirt to start container.
      • Container will execute systemd which will then execut
        • dhcpd to connect the network
        • systemd-journal to handle syslog messages
        • Unit file configured to start within the container.
      • Container will container a console that the host administrator can connect to and run a shell to debug what is going on within the container.
    • Execute
      • Allow host administrator to execute commands within the container.
    • Stop
      • Stop the container, and remove it from libvirt
    • Delete
      • Delete all content related to the container
    • Connect
      • Connect to the console and run a administrative shell with container

How To Test

Use the virt-sandbox-service command to create a container.

virt-sandbox-service create -C -l s0:c1,c2 -u httpd.service container1
Created sandbox container dir /var/lib/libvirt/filesystems/container1
Created sandbox config /etc/libvirt-sandbox/services/container1.sandbox
Created unit file /etc/systemd/system/httpd@container1.service

Manipulate the data within the container while running outside of the container.

cd /var/lib/libvirt/filesystems/container1/var/log
touch content
ls -lZ content
# Make sure the content gets created with the correct MCS label.
# Content should be labeled with s0:c1,c2 : Not s0

Now create a file with a bad label for the container.
cat "Secret" > badcontent
chcon -l s0:c3,c4 badcontent
 
 

Start the container

virt-sandbox-service start container1

In another window

Make sure the processes are running with the proper SELinux label. ps -eZ | grep svirt_lxc You should see processes like systemd, systemd-journal, dhclient and httpd running within the container with the MCS label of s0:c1,c2

Connect to the container

virt-sandbox-service connect container1
id 
getenforce   # Should tell you SELinux is disabled.
setenforce 1 # Should be denied
touch /file  # Should deny you creating this file
touch /var/www/html/content  # Should be allowed
cat /var/www/html/badcontent # Should be denied
Configure the apache server any way you would like, and manipulate html pages
ifconfig eth0  # Grap IP Address for use on next test
# Use the shell running with in the container to attempt to break out of the container. 
^] 

Connect to container using Firefox

firefox $IP # Using IP address from container, make sure you see the content.

Shut down the container

virt-sandbox-service stop container1

Now lets try to do the same but starting and stoping the container using systemctl commands

systemctl start httpd@container1.service
systemctl enable httpd@container1.service # Check on reboot if the container is running

Make sure the container is running.

virt-sandbox-service connect container1
ps -eZ
^]

Connect to container again

virt-sandbox-service connect container1
ps -eZ
^]

Stop container

systemctl stop httpd@container1.service

Delete the container

virt-sandbox-service delete container1
# Make sure it is deleted
ls /var/lib/libvirt/filesystems/container1
ls /etc/libvirt-sandbox/services/container1.sandbox
ls /etc/systemd/system/httpd@container1.service

Now lets setup hundreds of containers

cat /usr/bin/containers
#!/bin/bash
create() {
   virt-sandbox-service create -C -l s0:c$2 -u httpd.service $1
}

delete() {
    virt-sandbox-service delete $1
}

start() {
    systemctl start httpd@$1.service
}

stop() {
    systemctl stop httpd@$1.service
}

command=$1
name=$2
repeat=$3
for i in $(seq 1 $repeat)
do
    eval $command $name$i $i
done

Now lets use this script to test scalability:

containers create containers 100
containers start containers 100

Make sure containers start.

ps -eZ | grep systemd-journald | wc -l

Should be close to 101.

Make sure you can connect to a couple of container. Run Top to check CPU usage and see if any processes in container are using substantial CPU. Check for memory.

containers stop containers 100
containers delete containers 100

User Experience

Users should be able to easily establish containers and be able to run multiple containers simultaneously.

Comments and Discussion

• See Talk:Features/YourFeatureName