Secure Containers
Create a tool virt-sandbox-server that will allow an administrator to easily create a LXC Container to run the an application server, with libvirt and SELinux locking it down. This tool is part of the virt-sandbox package introduced in Fedora 17.
Summary
This change allows an administrator to run multiple services on the same machine each service in a secure Linux Container. virt-sandbox-service takes an systemd unit file and sets up all of the mountpoints and libvirt container information to run the service unit within the container. Then libvirt will be able to launch the container with a SELinux context that will prevent the containers process from interacting with other process on the system including other containers. This could allow an administrator to run multiple web servers simultaneously each with their own data and sharing the system data, even running as root, but prevent them from breaking the host machine or other processes on the system.
Owner
- Name: Daniel J Walsh
- Name: Daniel Berrange
- Email: dwalsh@redhat.com
- Email: berrange@redhat.com
Current status
- Targeted release: [Fedora 18]
- Last updated: Thu Aug 9 2012
- Percentage of completion: 90%
- libvirt-0.10.0-0rc0.fc18.x86_64 changes in Rawhide
- selinux-policy-3.11.1-4.fc18.noarch changes in Rawhide
- libvirt-sandbox-0.0.3-3.fc18 currently in Rawhide update required.
- Continuing to be developed and tested.
- kernel fixes for connecting applications to a container still being developed.
- Continuing to be developed and tested.
Detailed Description
libvirt as of Fedora 16 has the ability to run processes within a container, the problem with this is the processes can run as root and can easily break out of the container. Also it is fairly difficult to setup one of these containers to run a particular service, for example apache.
We want to make the processes of setting up these containers easier and to use SELinux and containers together to lock down the processes running within the container, meaning you could run multiple customers on a host and prevent them from attacking each other or the host.
Benefit to Fedora
Linux Containers is a low level way of doing virtualization, and allows you to run multiple copies of the same service at the same time on a system. It has some advantages over full virtualization as it does not have to wait for an entire system to boot, it can use less memory and can use the base OS, in a read-only manner.
Scope
This change effects
- SELinux Policy writers
- New policy has to be written to define what a confined application container is allowed to do
- Kernel and SELinux toolchain changes
- One problem we have with the way the kernel currently labels files, is that MCS portion of the label is applied based off the process level not the level of the directory that the file was create in. This means when a process running as unconfined_t:s0 creates a file in a directory labeled svirt_lxc_file_t:s0:c1,c2, the file will be created as svirt_lxc_file_t:s0 and the container will not be able to use the file, until the administrator changed the label.
This feature requires that the kernel support new policy construct to allow the policy writer to specify whether the MCS Label is inherited from the parent directory MCS Label or from the creating process MCS Label. kernel-3.3 should have this feature. (Completed in Current Kernel
- libvirt-sandbox package introduced in Fedora 17
- virt-sandbox-service script used for creating/starting/stopping Secure Containers
- Create
- Create an image file or a directory on disk and populate the image/directory with the default content based on the rpm data associated with the systemd unit file.
- It will also create the virt-sandbox database used by libvirt-sandbox to create a libvirt-lxc container.
- Setup systemd content within the container so that systemd will start the correct services
- Also create a systemd unit file so the administrator of the host machine can start and stop the service containers just like any other servers.
- Start
- Convert the Linux-sandbox configuration into libvirt-lxc configuration and tell libvirt to start container.
- Container will execute systemd which will then execut
- dhcpd to connect the network
- systemd-journal to handle syslog messages
- Unit file configured to start within the container.
- Container will container a console that the host administrator can connect to and run a shell to debug what is going on within the container.
- Execute
- Allow host administrator to execute commands within the container.
- Stop
- Stop the container, and remove it from libvirt
- Delete
- Delete all content related to the container
- Connect
- Connect to the console and run a administrative shell with container
- Create
- virt-sandbox-service script used for creating/starting/stopping Secure Containers
How To Test
Use the virt-sandbox-service command to create a container.
virt-sandbox-service create -C -l s0:c1,c2 -u httpd.service container1 Created sandbox container dir /var/lib/libvirt/filesystems/container1 Created sandbox config /etc/libvirt-sandbox/services/container1.sandbox Created unit file /etc/systemd/system/httpd@container1.service
Manipulate the data within the container while running outside of the container.
cd /var/lib/libvirt/filesystems/container1/var/log touch content ls -lZ content # Make sure the content gets created with the correct MCS label. # Content should be labeled with s0:c1,c2 : Not s0
Now create a file with a bad label for the container. cat "Secret" > badcontent chcon -l s0:c3,c4 badcontent
Start the container
virt-sandbox-service start container1
In another window
Make sure the processes are running with the proper SELinux label. ps -eZ | grep svirt_lxc You should see processes like systemd, systemd-journal, dhclient and httpd running within the container with the MCS label of s0:c1,c2
Connect to the container
virt-sandbox-service connect container1 id getenforce # Should tell you SELinux is disabled. setenforce 1 # Should be denied touch /file # Should deny you creating this file touch /var/www/html/content # Should be allowed cat /var/www/html/badcontent # Should be denied Configure the apache server any way you would like, and manipulate html pages ifconfig eth0 # Grap IP Address for use on next test # Use the shell running with in the container to attempt to break out of the container. ^]
Connect to container using Firefox
firefox $IP # Using IP address from container, make sure you see the content.
Shut down the container
virt-sandbox-service stop container1
Now lets try to do the same but starting and stoping the container using systemctl commands
systemctl start httpd@container1.service systemctl enable httpd@container1.service # Check on reboot if the container is running
Make sure the container is running.
virt-sandbox-service connect container1 ps -eZ ^]
Connect to container again
virt-sandbox-service connect container1 ps -eZ ^]
Stop container
systemctl stop httpd@container1.service
Delete the container
virt-sandbox-service delete container1 # Make sure it is deleted ls /var/lib/libvirt/filesystems/container1 ls /etc/libvirt-sandbox/services/container1.sandbox ls /etc/systemd/system/httpd@container1.service
Now lets setup hundreds of containers
cat /usr/bin/containers
#!/bin/bash
create() {
virt-sandbox-service create -C -l s0:c$2 -u httpd.service $1
}
delete() {
virt-sandbox-service delete $1
}
start() {
systemctl start httpd@$1.service
}
stop() {
systemctl stop httpd@$1.service
}
command=$1
name=$2
repeat=$3
for i in $(seq 1 $repeat)
do
eval $command $name$i $i
done
Now lets use this script to test scalability:
containers create containers 100 containers start containers 100
Make sure containers start.
ps -eZ | grep systemd-journald | wc -l
Should be close to 101.
Make sure you can connect to a couple of container. Run Top to check CPU usage and see if any processes in container are using substantial CPU. Check for memory.
containers stop containers 100 containers delete containers 100
User Experience
Users should be able to easily establish containers and be able to run multiple containers simultaneously.
See Test Information for use cases.
Documentation
The virt-sandbox-service command line tool comes with a manpage:
# man virt-sandbox-service
This provides complete command line help and examples of usage
Release Notes
- The 'libvirt-sandbox' RPM introduces a new command 'virt-sandbox-service' for configurtin/running system service in a confined environment.
- The confined environment may have a custom view of the filesystem and custom networking configuration.