From Fedora Project Wiki

m (Minor wiki-fication)
(Note this is being run as part of CI now)
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{QA/Test_Case
{{QA/Test_Case
|description=Test that compression is reversible for {{package|squashfs-tools}}.  The compression types tested will be those expected to work with the version of squashfs-tools you have installed and the kernel you are running.   
|description=Test that compression is reversible for {{package|squashfs-tools}}.  The compression types tested will be those expected to work with the version of squashfs-tools you have installed and the kernel you are running.  These tests are used to make sure squashfs-tools will work as expected for live images.


A test script is available to provide a consistent test result.  The test script needs to be run as the ''root'' user so that squashfs images can be mounted and to prevent warnings about setting selinux extended attributes when running unsquashfs.  The test script will also create a directory in the current working directory named {{filename|test-squashfs}}.  When run, the script will check if {{filename|test-squashf/data}} is already created and if so won't recreate the test data.
This test case should be being run as part of continuous integration testing. That currently will only will cover x86_64 and will only get checked against one kernel version. So there can be value in running this manually still.
 
A test script is available to provide a consistent test result.  The test script needs to be run as the ''root'' user so that squashfs images can be mounted and to prevent warnings about setting selinux extended attributes when running unsquashfs.  The test script will also create a directory in the current working directory named {{filename|test-squashfs}}.  When run, the script will check if {{filename|test-squashfs/data}} is already created and if so won't recreate the test data.
|setup=
|setup=
# Install the required {{package|squashfs-tools}} and {{package|rpmdevtools}} packages: <pre>yum install squashfs-tools rpmdevtools</pre>
# Install the required {{package|squashfs-tools}} and {{package|rpmdevtools}} packages: <pre>yum install squashfs-tools rpmdevtools</pre>
|actions=
|actions=
# Download the test script [[:File:squashfs-compression-test.sh|squashfs-compression-test.sh]]
# Download the test script [https://fedoraproject.org/w/uploads/2/25/Squashfs-compression-test.sh Squashfs-compression-test.sh]
# As root, execute the test script using the command: {{command|bash squashfs-compression-test.sh}}.  Note any failure messages.
# As root, execute the test script using the command: {{command|bash Squashfs-compression-test.sh}}.  Note any failure messages.
{{admon/important|FIXME|Convert the following into an uploaded wiki file (e.g. [[:File:squashfs-compression-test.sh|squashfs-compression-test.sh]])}}
|results=
<ol>
<li>The test script should exit cleanly and not abort early because it was unable to create some file or free disc space running out. There should be no messages with the string <code>failed</code> in them. Typically you might run this using script and then grep <code>typescript</code> for <code>failed</code>. Successful test output is included below:
<pre><nowiki>
<pre><nowiki>
#!/bin/bash
Building data directory.
 
Building squashfs image using gzip compression.
# We need rpmdev-vercmp
Parallel mksquashfs: Using 2 processors
rpm -q rpmdevtools > /dev/null 2>&1 || (echo 'rpmdev-vercmp from rpmdevtools is needed to run this script, aborting.'; exit 1)
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
 
[=============================================================|] 2316/2316 100%
# Define block sizes
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
blocks=(4K 1M)
compressed data, compressed metadata, compressed fragments, compressed xattrs
 
duplicates are removed
# Define fill files
Filesystem size 107770.89 Kbytes (105.25 Mbytes)
fill=(/dev/zero /dev/urandom)
38.79% of uncompressed filesystem size (277839.40 Kbytes)
 
Inode table size 2185 bytes (2.13 Kbytes)
# Define number of iterations
11.16% of uncompressed inode table size (19586 bytes)
iter=5
Directory table size 1442 bytes (1.41 Kbytes)
 
21.57% of uncompressed directory table size (6684 bytes)
# Define fragment sizes
Xattr table size 54 bytes (0.05 Kbytes)
frags=(0 1 2047 4095)
100.00% of uncompressed xattr table size (54 bytes)
 
Number of duplicate files found 42
# Define test directory
Number of inodes 189
testdir=./test-squashfs
Number of files 188
 
Number of fragments 6
# Define mount point
Number of symbolic links  0
mp=${testdir}/mnt
Number of device nodes 0
 
Number of fifo nodes 0
# Define data directory
Number of socket nodes 0
datadir=${testdir}/data
Number of directories 1
 
Number of ids (unique uids + gids) 1
# Check for squashfs-tools version and set compression types to test
Number of uids 1
sqfsver=`rpm -q --qf '%{EVR}' squashfs-tools`
root (0)
if rpmdev-vercmp 4.1 ${sqfsver} > /dev/null; [ $? == '11' ]; then
Number of gids 1
  ucomp=('gzip')
root (0)
elif rpmdev-vercmp 4.2 ${sqfsver} > /dev/null; [ $? == '11' ]; then
Testing unmounted extract using gzip compression.
  ucomp=(gzip lzo lzma)
Parallel unsquashfs: Using 2 processors
else
188 inodes (2316 blocks) to write
  ucomp=(gzip lzo lzma xz)
fi
 
# Check for kernel verion and set mount test compression types
kernel=`uname -r`
if rpmdev-vercmp 2.6.36 ${kernel} > /dev/null; [ $? == '11' ]; then
  mcomp=('gzip')
elif rpmdev-vercmp 2.6.38 ${kernel} > /dev/null; [ $? == '11' ]; then
  mcomp=(gzip lzo)
else
  mcomp=(gzip lzo xz)
fi
 
# Check for uid 0 and print a warning if not
[ ${UID} -ne 0 ] && echo 'Mount tests will not be performed when not running as root'


# Check if test directory exists and make if not
[=============================================================-] 2316/2316 100%
[ -d ${testdir} ] || mkdir ${testdir}
created 188 files
[ -d ${testdir} ] || (echo "Unable to make '${testdir}', aborting."; exit 1)
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using gzip compression.
Building squashfs image using lzo compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzo compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 107764.62 Kbytes (105.24 Mbytes)
38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2553 bytes (2.49 Kbytes)
13.03% of uncompressed inode table size (19586 bytes)
Directory table size 1732 bytes (1.69 Kbytes)
25.91% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using lzo compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


# Check if mount point directory exists and make if not
[=============================================================/] 2316/2316 100%
[ -d ${mp} ] || mkdir ${mp}
created 188 files
[ -d ${mp} ] || (echo "Unable to make '${mp}', aborting."; exit 1)
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using lzo compression.
Building squashfs image using lzma compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzma compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 107750.75 Kbytes (105.23 Mbytes)
38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1747 bytes (1.71 Kbytes)
8.92% of uncompressed inode table size (19586 bytes)
Directory table size 1209 bytes (1.18 Kbytes)
18.09% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using lzma compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


# Check if data directory exists and make if not
[=============================================================-] 2316/2316 100%
if [ -d ${datadir} ]; then
created 188 files
  echo "Using existing data directory."
created 1 directories
else
created 0 symlinks
  echo "Building data directory."
created 0 devices
  mkdir ${datadir}
created 0 fifos
  [ -d ${datadir} ] || (echo "Unable to make '${datadir}', aborting."; exit 1)
Building squashfs image using xz compression.
  for size in ${frags[*]}; do
Parallel mksquashfs: Using 2 processors
    for file in ${fill[*]}; do
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
      dd if=${file} of=${datadir}/frag-`basename ${file}`-${size} bs=1 count=${size} > /dev/null 2>&1
[=============================================================/] 2316/2316 100%
    done
Exportable Squashfs 4.0 filesystem, xz compressed, data block size 131072
  done
compressed data, compressed metadata, compressed fragments, compressed xattrs
  for size in ${blocks[*]}; do
duplicates are removed
    for ((count=1;${count}<=${iter};count++)); do
Filesystem size 107751.25 Kbytes (105.23 Mbytes)
      for file in ${fill[*]}; do
38.78% of uncompressed filesystem size (277839.40 Kbytes)
        dd if=${file} of=${datadir}/file-`basename ${file}`-${size}-${count} bs=${size} count=${count} > /dev/null 2>&1
Inode table size 1874 bytes (1.83 Kbytes)
      done
9.57% of uncompressed inode table size (19586 bytes)
    done
Directory table size 1262 bytes (1.23 Kbytes)
  done
18.88% of uncompressed directory table size (6684 bytes)
  for size1 in ${frags[*]}; do
Xattr table size 54 bytes (0.05 Kbytes)
    for file1 in ${fill[*]}; do
100.00% of uncompressed xattr table size (54 bytes)
      for size2 in ${blocks[*]}; do
Number of duplicate files found 42
        for ((count=1;${count}<=${iter};count++)); do
Number of inodes 189
          for file2 in ${fill[*]}; do
Number of files 188
            cat ${datadir}/file-`basename ${file2}`-${size2}-${count} ${datadir}/frag-`basename ${file1}`-${size1} > ${datadir}/combined-`basename ${file2}`-${size2}-${count}-`basename ${file1}`-${size1}
Number of fragments 6
          done
Number of symbolic links  0
        done
Number of device nodes 0
      done
Number of fifo nodes 0
    done
Number of socket nodes 0
  done
Number of directories 1
fi
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using xz compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


# Run unmounted tests
[=============================================================/] 2316/2316 100%
for comp in ${ucomp[*]}; do
created 188 files
  echo "Building squashfs image using ${comp} compression."
created 1 directories
  if [ "${comp}" == gzip ]; then
created 0 symlinks
    mksquashfs ${datadir} ${testdir}/sq.img || (echo "mksquashfs failed for ${comp} compression."; continue)
created 0 devices
  else
created 0 fifos
    mksquashfs ${datadir} ${testdir}/sq.img -comp ${comp} || (echo "mksquashfs failed for ${comp} compression."; continue)
  fi
  echo "Testing unmounted extract using ${comp} compression."
  unsquashfs -d ${testdir}/sq ${testdir}/sq.img  || echo "unsquashfs failed for ${comp} compression."
  diff -r -q ${testdir}/sq ${datadir} || echo "Extract test failed for ${comp} compression."
  rm -rf ${testdir}/sq
  if [ ${UID} == 0 ]; then
    for kern in ${mcomp[*]}; do
      if [ ${kern} == ${comp} ]; then
        echo "Testing mounted image using ${comp} compression."
        mount -r -o loop ${testdir}/sq.img ${mp} || echo "Mount failed.";
        diff -r -q ${mp} ${datadir} || echo "Mounted test failed for ${comp} compression."
        umount ${mp}
        break
      fi
    done
  fi
  rm -f ${testdir}/sq ${testdir}/sq.img
done
</nowiki></pre>
</nowiki></pre>
 
</li></ol>
|results=
# The test script should exit cleanly and not abort early because it was unable to create some file or free disc space running out. There should be no messages with the string <code>failed</code> in them. Typically you might run this using script and then grep <code>typescript</code> for <code>failed</code>.  Successful test output is included below
 
}}
}}
[[Category:Critical_path_test_cases]]
[[Category:Critical_path_test_cases]]
[[Category:Package_squashfs-tools_test_cases]]

Latest revision as of 11:34, 23 May 2019

Description

Test that compression is reversible for squashfs-tools. The compression types tested will be those expected to work with the version of squashfs-tools you have installed and the kernel you are running. These tests are used to make sure squashfs-tools will work as expected for live images.

This test case should be being run as part of continuous integration testing. That currently will only will cover x86_64 and will only get checked against one kernel version. So there can be value in running this manually still.

A test script is available to provide a consistent test result. The test script needs to be run as the root user so that squashfs images can be mounted and to prevent warnings about setting selinux extended attributes when running unsquashfs. The test script will also create a directory in the current working directory named test-squashfs. When run, the script will check if test-squashfs/data is already created and if so won't recreate the test data.

Setup

  1. Install the required squashfs-tools and rpmdevtools packages: 
    yum install squashfs-tools rpmdevtools

How to test

  1. Download the test script Squashfs-compression-test.sh
  2. As root, execute the test script using the command: bash Squashfs-compression-test.sh. Note any failure messages.

Expected Results

  1. The test script should exit cleanly and not abort early because it was unable to create some file or free disc space running out. There should be no messages with the string failed in them. Typically you might run this using script and then grep typescript for failed. Successful test output is included below: 
    Building data directory.
Building squashfs image using gzip compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107770.89 Kbytes (105.25 Mbytes)
	38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2185 bytes (2.13 Kbytes)
	11.16% of uncompressed inode table size (19586 bytes)
Directory table size 1442 bytes (1.41 Kbytes)
	21.57% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using gzip compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================-] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using gzip compression.
Building squashfs image using lzo compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzo compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107764.62 Kbytes (105.24 Mbytes)
	38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2553 bytes (2.49 Kbytes)
	13.03% of uncompressed inode table size (19586 bytes)
Directory table size 1732 bytes (1.69 Kbytes)
	25.91% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using lzo compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================/] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using lzo compression.
Building squashfs image using lzma compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzma compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107750.75 Kbytes (105.23 Mbytes)
	38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1747 bytes (1.71 Kbytes)
	8.92% of uncompressed inode table size (19586 bytes)
Directory table size 1209 bytes (1.18 Kbytes)
	18.09% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using lzma compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================-] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Building squashfs image using xz compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, xz compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107751.25 Kbytes (105.23 Mbytes)
	38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1874 bytes (1.83 Kbytes)
	9.57% of uncompressed inode table size (19586 bytes)
Directory table size 1262 bytes (1.23 Kbytes)
	18.88% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using xz compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================/] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Retrieved from "https://fedoraproject.org/w/index.php?title=QA:Testcase_squashfs-tools_compression&oldid=544007"