From Fedora Project Wiki

(Initial shot at squashfs-tools test case)
 
(Note this is being run as part of CI now)
 
(22 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= Description =
{{QA/Test_Case
{{{description|Test that compression is reversible for squashfs-tools.}}}
|description=Test that compression is reversible for {{package|squashfs-tools}}.  The compression types tested will be those expected to work with the version of squashfs-tools you have installed and the kernel you are running. These tests are used to make sure squashfs-tools will work as expected for live images.
{{#if: {{{setup|}}}|= Setup =
{{{setup}}}|}}
= How to test =
Run the provided script as root and note any failure messages. The script needs to be run as root so that squashfs images can be mounted and to prevent warnings about setting selinux extended attributes when running unsquashfs.


You need to have squashfs-tools and rpmdevtools installed.
This test case should be being run as part of continuous integration testing. That currently will only will cover x86_64 and will only get checked against one kernel version. So there can be value in running this manually still.


The compression types tested will be those expected to work with the version of squashfs-tools you have installed and the kernel you are running.
A test script is available to provide a consistent test result.  The test script needs to be run as the ''root'' user so that squashfs images can be mounted and to prevent warnings about setting selinux extended attributes when running unsquashfs.  The test script will also create a directory in the current working directory named {{filename|test-squashfs}}.  When run, the script will check if {{filename|test-squashfs/data}} is already created and if so won't recreate the test data.
|setup=
# Install the required {{package|squashfs-tools}} and {{package|rpmdevtools}} packages: <pre>yum install squashfs-tools rpmdevtools</pre>
|actions=
# Download the test script [https://fedoraproject.org/w/uploads/2/25/Squashfs-compression-test.sh Squashfs-compression-test.sh]
# As root, execute the test script using the command: {{command|bash Squashfs-compression-test.sh}}.  Note any failure messages.
|results=
<ol>
<li>The test script should exit cleanly and not abort early because it was unable to create some file or free disc space running out. There should be no messages with the string <code>failed</code> in them. Typically you might run this using script and then grep <code>typescript</code> for <code>failed</code>.  Successful test output is included below:
<pre><nowiki>
Building data directory.
Building squashfs image using gzip compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 107770.89 Kbytes (105.25 Mbytes)
38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2185 bytes (2.13 Kbytes)
11.16% of uncompressed inode table size (19586 bytes)
Directory table size 1442 bytes (1.41 Kbytes)
21.57% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using gzip compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


{{{actions|Run the following script:
[=============================================================-] 2316/2316 100%
<code>
created 188 files
#!/bin/bash
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using gzip compression.
Building squashfs image using lzo compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzo compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 107764.62 Kbytes (105.24 Mbytes)
38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2553 bytes (2.49 Kbytes)
13.03% of uncompressed inode table size (19586 bytes)
Directory table size 1732 bytes (1.69 Kbytes)
25.91% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using lzo compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


# We need rpmdev-vercmp
[=============================================================/] 2316/2316 100%
rpm -q rpmdevtools > /dev/null 2>&1 || (echo 'rpmdev-vercmp from rpmdevtools is needed to run this script, aborting.'; exit 1)
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using lzo compression.
Building squashfs image using lzma compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzma compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 107750.75 Kbytes (105.23 Mbytes)
38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1747 bytes (1.71 Kbytes)
8.92% of uncompressed inode table size (19586 bytes)
Directory table size 1209 bytes (1.18 Kbytes)
18.09% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using lzma compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


# Define block sizes
[=============================================================-] 2316/2316 100%
blocks=(4K 1M)
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Building squashfs image using xz compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, xz compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 107751.25 Kbytes (105.23 Mbytes)
38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1874 bytes (1.83 Kbytes)
9.57% of uncompressed inode table size (19586 bytes)
Directory table size 1262 bytes (1.23 Kbytes)
18.88% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using xz compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


# Define fill files
[=============================================================/] 2316/2316 100%
fill=(/dev/zero /dev/urandom)
created 188 files
 
created 1 directories
# Define number of iterations
created 0 symlinks
iter=5
created 0 devices
 
created 0 fifos
# Define fragment sizes
</nowiki></pre>
frags=(0 1 2047 4095)
</li></ol>
 
}}
# Define test directory
[[Category:Critical_path_test_cases]]
testdir=./test-squashfs
[[Category:Package_squashfs-tools_test_cases]]
 
# Define mount point
mp=${testdir}/mnt
 
# Define data directory
datadir=${testdir}/data
 
# Check for squashfs-tools version and set compression types to test
sqfsver=`rpm -q --qf '%{EVR}' squashfs-tools`
if rpmdev-vercmp 4.1 ${sqfsver} > /dev/null; [ $? == '11' ]; then
  ucomp=('gzip')
elif rpmdev-vercmp 4.2 ${sqfsver} > /dev/null; [ $? == '11' ]; then
  ucomp=(gzip lzo lzma)
else
  ucomp=(gzip lzo lzma xz)
fi
 
# Check for kernel verion and set mount test compression types
kernel=`uname -r`
if rpmdev-vercmp 2.6.36 ${kernel} > /dev/null; [ $? == '11' ]; then
  mcomp=('gzip')
elif rpmdev-vercmp 2.6.38 ${kernel} > /dev/null; [ $? == '11' ]; then
  mcomp=(gzip lzo)
else
  mcomp=(gzip lzo xz)
fi
 
# Check for uid 0 and print a warning if not
[ ${UID} -ne 0 ] && echo 'Mount tests will not be performed when not running as root'
 
# Check if test directory exists and make if not
[ -d ${testdir} ] || mkdir ${testdir}
[ -d ${testdir} ] || (echo "Unable to make '${testdir}', aborting."; exit 1)
 
# Check if mount point directory exists and make if not
[ -d ${mp} ] || mkdir ${mp}
[ -d ${mp} ] || (echo "Unable to make '${mp}', aborting."; exit 1)
 
# Check if data directory exists and make if not
if [ -d ${datadir} ]; then
  echo "Using existing data directory."
else
  echo "Building data directory."
  mkdir ${datadir}
  [ -d ${datadir} ] || (echo "Unable to make '${datadir}', aborting."; exit 1)
  for size in ${frags[*]}; do
    for file in ${fill[*]}; do
      dd if=${file} of=${datadir}/frag-`basename ${file}`-${size} bs=1 count=${size} > /dev/null 2>&1
    done
  done
  for size in ${blocks[*]}; do
    for ((count=1;${count}<=${iter};count++)); do
      for file in ${fill[*]}; do
        dd if=${file} of=${datadir}/file-`basename ${file}`-${size}-${count} bs=${size} count=${count} > /dev/null 2>&1
      done
    done
  done
  for size1 in ${frags[*]}; do
    for file1 in ${fill[*]}; do
      for size2 in ${blocks[*]}; do
        for ((count=1;${count}<=${iter};count++)); do
          for file2 in ${fill[*]}; do
            cat ${datadir}/file-`basename ${file2}`-${size2}-${count} ${datadir}/frag-`basename ${file1}`-${size1} > ${datadir}/combined-`basename ${file2}`-${size2}-${count}-`basename ${file1}`-${size1}
          done
        done
      done
    done
  done
fi
 
# Run unmounted tests
for comp in ${ucomp[*]}; do
  echo "Building squashfs image using ${comp} compression."
  if [ "${comp}" == gzip ]; then
    mksquashfs ${datadir} ${testdir}/sq.img
  else
    mksquashfs ${datadir} ${testdir}/sq.img -comp ${comp}
  fi
  echo "Testing unmounted extract using ${comp} compression."
  unsquashfs -d ${testdir}/sq ${testdir}/sq.img
  diff -r -q ${testdir}/sq ${datadir} || echo "Extract test failed for ${comp} compression."
  rm -rf ${testdir}/sq
  if [ ${UID} == 0 ]; then
    for kern in ${mcomp[*]}; do
      if [ ${kern} == ${comp} ]; then
        echo "Testing mounted image using ${comp} compression."
        mount -o loop ${testdir}/sq.img ${mp} || echo "Mount failed.";
        diff -r -q ${testdir}/sq ${datadir} || echo "Mounted test failed for ${comp} compression."
        umount ${mp}
        break
      fi
    done
  fi
  rm -f ${testdir}/sq ${testdir}/sq.img
done
</code>
}}}
 
= Expected Results =
 
The test should not abort early because it was unable to create some file or free disc space running out. There should be no messages starting with either "Mounted test failed" or "Extract test failed". None of mksquashfs, unsquashfs or mount commands should report failure.

Latest revision as of 11:34, 23 May 2019

Description

Test that compression is reversible for squashfs-tools. The compression types tested will be those expected to work with the version of squashfs-tools you have installed and the kernel you are running. These tests are used to make sure squashfs-tools will work as expected for live images.

This test case should be being run as part of continuous integration testing. That currently will only will cover x86_64 and will only get checked against one kernel version. So there can be value in running this manually still.

A test script is available to provide a consistent test result. The test script needs to be run as the root user so that squashfs images can be mounted and to prevent warnings about setting selinux extended attributes when running unsquashfs. The test script will also create a directory in the current working directory named test-squashfs. When run, the script will check if test-squashfs/data is already created and if so won't recreate the test data.

Setup

  1. Install the required squashfs-tools and rpmdevtools packages: 
    yum install squashfs-tools rpmdevtools

How to test

  1. Download the test script Squashfs-compression-test.sh
  2. As root, execute the test script using the command: bash Squashfs-compression-test.sh. Note any failure messages.

Expected Results

  1. The test script should exit cleanly and not abort early because it was unable to create some file or free disc space running out. There should be no messages with the string failed in them. Typically you might run this using script and then grep typescript for failed. Successful test output is included below: 
    Building data directory.
Building squashfs image using gzip compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107770.89 Kbytes (105.25 Mbytes)
	38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2185 bytes (2.13 Kbytes)
	11.16% of uncompressed inode table size (19586 bytes)
Directory table size 1442 bytes (1.41 Kbytes)
	21.57% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using gzip compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================-] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using gzip compression.
Building squashfs image using lzo compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzo compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107764.62 Kbytes (105.24 Mbytes)
	38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2553 bytes (2.49 Kbytes)
	13.03% of uncompressed inode table size (19586 bytes)
Directory table size 1732 bytes (1.69 Kbytes)
	25.91% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using lzo compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================/] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using lzo compression.
Building squashfs image using lzma compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzma compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107750.75 Kbytes (105.23 Mbytes)
	38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1747 bytes (1.71 Kbytes)
	8.92% of uncompressed inode table size (19586 bytes)
Directory table size 1209 bytes (1.18 Kbytes)
	18.09% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using lzma compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================-] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Building squashfs image using xz compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, xz compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107751.25 Kbytes (105.23 Mbytes)
	38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1874 bytes (1.83 Kbytes)
	9.57% of uncompressed inode table size (19586 bytes)
Directory table size 1262 bytes (1.23 Kbytes)
	18.88% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using xz compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================/] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Retrieved from "https://fedoraproject.org/w/index.php?title=QA:Testcase_squashfs-tools_compression&oldid=544007"