From Fedora Project Wiki
(drop direct test day category membership (should be via release))
 
(74 intermediate revisions by 23 users not shown)
Line 7: Line 7:
|}
|}


{{admon/note | Can't make the date? | If you come to this page before or after the test day is completed, your testing is still valuable, and you can use the information on this page to test, file any bugs you find at [http://bugzilla.redhat.com Bugzilla], and add your results to the results section. If this page is more than a month old when you arrive here, please check the current schedule - which should be linked at [[QA/Test_Days]] - and see if a similar but more recent Test Day is planned or has already happened.}}
{{admon/note | Can't make the date? | If you come to this page before or after the test day is completed, your testing is still valuable, and you can use the information on this page to test, file any bugs you find at [http://bugzilla.redhat.com Bugzilla], and add your results to the results section. If this page is more than a month old when you arrive here, please check the [[QA/Test_Days|current schedule]] and see if a similar but more recent Test Day is planned or has already happened.}}


== What to test? ==
== What to test? ==


Today's installment of Fedora Test Day will focus on '''<<FIXME>>'''
Have you ever used any security '''scanning application'''? Does the '''security configuration''' of your box matters? Do you want to keep you system in '''consistent state'''? If you have positive answer to any of these questions then it's probably worth to joint this Fedora Test Day that will focus on [https://fedoraproject.org/wiki/Features/OpenSCAP OpenSCAP] feature.
 
What is '''SCAP?''' It is a line of standards managed by [http://scap.nist.gov/ NIST]. It was created to provide a standardized approach to '''maintaining the security of systems''', such as automatically verifying the presence of patches, checking system security configuration settings, and examining systems for signs of compromise.
 
The SCAP suite contains multiple complex data exchange formats that are to be used to transmit important vulnerability, configuration, and other security data. Historically, there have been few tools that provide a way to query this data in the needed format. This lack of tools makes the barrier to entry very high and discourages adoption of these protocols by the community. It's a goal of '''OpenSCAP''' [http://www.open-scap.org/page/Main_Page project] to create a framework of libraries to improve the accessibility of SCAP and enhance the usability of the information it represents.


== Who's available ==
== Who's available ==


The following cast of characters will be available testing, workarounds, bug fixes, and general discussion:
The following cast of characters will be available testing, workarounds, bug fixes, and general discussion:
* Development - [[User:pvrabec|Peter Vrabec]] (wrabco), '''<<FIXME>>'''
* Development - [[User:pvrabec|Peter Vrabec]] (wrabco), Tomas Heinrich (theinric), Maros Barabas (mbarabas), Daniel Kopecek (dkopecek), Lukas Kuklinek (lkukline)
* Quality Assurance - [[User:kparal|Kamil Páral]] (kparal)
* FirstAidKit development - [[User:msivak|Martin Sivák]] (msivak)
* Quality Assurance - [[User:kparal|Kamil Páral]] (kparal), Ondrej Moris (omoris)


== Prerequisite for Test Day ==  
== Prerequisite for Test Day ==  


List any prerequisite needs for the test event.  A fresh system, virtualized guest, a blank DVD ... a desire to break software?
* A fully updated Fedora 13 or 14.
 
** This must be a real installation, live CDs are unfortunately not suitable for this test day.
* Usb key
** We are interested in different software setups, so if possible please use your real workstation, rather than clean install of F13 or F14. You don't have to be afraid, this software is not destructive in any way.
* Usb externally connected HD IDE/SATA
* At least 2 GB of RAM is recommended for the system, otherwise the tool may work very slow.
* Empty HD IDE/SATA/SCSI
* Free space on HD
 
Here's a chunk which is commonly used for most Test Days. Replace XX with whatever Fedora release is pending:
 
* An updated [http://fedoraproject.org/get-prerelease Fedora XX pre-release], [[Releases/Rawhide|Rawhide]] (tips on installing Rawhide below), or a [http://alt.fedoraproject.org/pub/alt/nightly-composes/desktop/ nightly live image]
* Your hardware profile uploaded to [http://www.smolts.org Smolt] according to [http://smolts.org/smolt-wiki/Main_Page#Usage these instructions]


== How to test? ==
== How to test? ==


High level details on how a contributor can get involved.  This can include (but not limited to):
<ol>
 
<li>Fully update your '''Fedora 13''' or '''Fedora 14'''.</li>
* Areas to target with exploratory testing
<li>Install '''openscap, openscap-utils''' and '''openscap-python''' packages version '''0.6.1-testday5'''. Download them from: http://people.redhat.com/pvrabec/openscap/
* A list of pre-defined test cases to execute
{{admon/important|Packages updated|Packages have been updated to fix numerous errors. Please update if you've downloaded the old ones.}}
* How to report back results
</li>
 
<li>Download required SCAP content: http://people.redhat.com/pvrabec/openscap/content
Here's another common chunk (again, replace XX as above):
{{admon/important|Files updated|SCAP content was updated during this test day. Please update if you've downloaded the old one.}}
 
</li>
=== '''Update your machine''' ===
<li>Follow the test cases below.</li>
 
<li>Write your results to the result matrix.</li>
If you're running Fedora XX, make sure you have all the current updates for it installed, using the update manager. If you want to try Rawhide, see the instructions on the [[Releases/Rawhide|Rawhide]] page on the various ways in which you can install or update to Rawhide. Or:
</ol>
 
=== '''Live image''' ===
 
Optionally, you may download a non-destructive Rawhide live image for your architecture. Tips on using a live image are available at [[FedoraLiveCD]]. Live images can be found [http://alt.fedoraproject.org/pub/alt/nightly-composes/desktop/ here].


== Test Cases ==
== Test Cases ==


Provide a list of test areas or test cases that you'd like contributors to execute.  For other examples, see [[:Category:Test_Cases]].
Please execute as many test cases from the following list of [[:Category:OpenSCAP Test Cases|OpenSCAP Test Cases]] as possible:
* [[QA:TestCase OpenSCAP Fedora default settings|Fedora default settings]]
* [[QA:TestCase OpenSCAP Fedora adjusted settings|Fedora adjusted settings]]
* [[QA:TestCase OpenSCAP secstate|secstate tool]]
* [[QA:TestCase_OpenSCAP_Fedora_FirstAidKit|FirstAidKit plugin for OpenSCAP]]


== Test Results ==
== Test Results ==
Line 61: Line 59:
{|
{|
! User
! User
! [[QA:Testcase_sample_1|Sample test 1]]
! [[QA:TestCase OpenSCAP Fedora default settings|Fedora default settings]]
! [[QA:Testcase_sample_2|Sample test 2]]
! [[QA:TestCase OpenSCAP Fedora adjusted settings|Fedora adjusted settings]]
! [[QA:Testcase_sample_3|Sample test 3]]
! [[QA:TestCase OpenSCAP secstate|secstate tool]]
! [[QA:Testcase_sample_4|Sample test 4]]
! [[QA:TestCase_OpenSCAP_Fedora_FirstAidKit|FAK plugin]]
! References
! References
|-
|-
| [[User:SampleUser|Sample User]]
| [[User:SampleUser|Sample User]]
| {{result|none}}
| {{result|pass}}
| {{result|pass}}
| {{result|warn}} <ref>Test pass, but also encountered {{bz|54321}}</ref>
| {{result|warn}} <ref>Test pass, but also encountered {{bz|54321}}</ref>
| {{result|fail}} <ref>{{bz|12345}}</ref>
| {{result|fail}} <ref>{{bz|12345}}</ref>
| {{result|none}}
| <references/>
|-
| [[User:Newgle1|newgle1]]
| {{result|fail|newgle1}}<ref name=bug />
| {{result|fail|newgle1}} <ref name=bug>err:*** buffer overflow detected ***: oscap terminated</ref>
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:Rhe|He Rui]]
| {{result|fail|rhe}}<ref>buffer overflowed and some rules failed: http://fpaste.org/wSvq/</ref>
| {{result|fail|rhe}}<ref group="long">tested the rule-2.2.2.3.a (Disable the Automounter if Possible), when I stopped the autofs service as the rules suggested, the result was still 'fail'.(Yum remove autofs can get a 'pass' result) </ref>
| {{result|none}}
| {{result|none}}
| <references/>
|-
|-
| [[User:jkaluza|Jan Kaluza]]
| {{result|fail|jkaluza}}<ref>buffer overflowed - {{bz|627488}}</ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:ppisar|Petr Pisar]]
| {{result|fail|ppisar}} <ref group="long">Tests checking file permissions (rule-2.2.3.3.a, rule-2.2.3.4.a, rule-2.2.3.4.b, rule-2.2.3.5.a, rule-2.2.3.5.b, rule-2.2.3.6.a) eats all memory (4 GiB) and are terminated by kernel – {{bz|565691}}</ref>
  {{result|fail|ppisar}} <ref group="long">Test rule-2.1.2.3.4.a (Ensure Package Signature Checking is Not Disabled For Any Repos) fails because I have defined rawhide repositories with disabled signature checking and disabled for installation. I think disabled repositories should not be considered in this test.</ref>
  {{result|fail|ppisar}} <ref group="long">Test rule-2.5.1.2.b (Set net.ipv4.conf.all.accept_redirects for Hosts and Routers) fails. This is default value for F13. F13 should be fixed (/etc/sysctl.conf) or the test removed as far as it can be useful in some scenarios (link with more routers, link with more IP networks).</ref>
  {{result|pass|ppisar}} <ref>Other tests passed</ref>
| {{result|fail|ppisar}} <ref group="long">Test rule-3.6.1.1.a (Disable X Windows at System Boot) fails if enabled despite my inittab has default runlevel 3. Test is defined as equality to number 5 in oval file. More ever `X Windows' is nonsense. Correct name is `X Window' without the `s' suffix. See X(7) manual page. You are breaking trade mark ;)</ref>
  {{result|fail|ppisar}} <ref>Test rule-3.7.1.1.a (Disable Avahi Server Software) fails even if avahi-deamon is disabled in all runlevels and none is running</ref>
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:jgorig|Jan Gorig]]
| {{result|fail|jgorig}}<ref>same problem - buffer overflowed on x86_64 F13 - {{bz|627488}}</ref>
  {{result|pass|jgorig}}<ref>bug fixed</ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:kushal|Kushal Das]]
| {{result|fail|kushal}}<ref>same problem - buffer overflowed on x86 F13 - {{bz|627488}}</ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:dramsey|David Ramsey]]
| {{result|fail}}<ref>Same problem with buffer overflowed on x86 F14</ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:mgrepl|Miroslav Grepl]]
| {{result|pass|mgrepl}}<ref>Test finished (fixed pkgs from koji)</ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:omoris|Ondrej Moriš]]
| {{result|fail|omoris}}<ref>test finished (fixed pkgs from koji) with several fails: http://fpaste.org/Sgys/</ref>
| {{result|none}}
| {{result|none}}
| {{result|warn|omoris}}<ref>getting error while changing some variable values (HTTP reply/request), gui is mostly not updated during evaluation</ref>
| <references/>
|-
| [[User:masami|Masami Ichikawa]]
| {{result|fail|masami}}<ref>same problem - buffer overflowed on x86 F14 - {{bz|627488}}</ref> {{result|fail|masami}}<ref>testday5 fails rule-2.5.1.2.b (Set net.ipv4.conf.all.accept_redirects for Hosts and Routers). same as {{bz|627600}}</ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:kparal|Kamil Páral]]
| {{result|warn|kparal}}<ref>''Set net.ipv4.conf.all.accept_redirects for Hosts and Routers'' fails {{bz|627600}}</ref> {{result|fail|kparal||627674}}
| {{result|none}}
| {{result|fail|kparal}}<ref>''Not Selected:  0'' in http://fpaste.org/4Okv/</ref> {{result|pass|kparal}}<ref>openscap-0.6.1-testday4.fc14</ref>
| {{result|fail|kparal||627633}}<ref>Values in test and policy selection allows "0.5" and "enforcingaaa"</ref>
| <references/>
|-
| [[User:David.Paige|David Paige]]
| {{result|pass|David.Paige}}<ref>'No errors, five failed individual tests.</ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
| <references/>
|-
|-
|}
|}


[[Category:Test Days]]
== Long comments ==
<references group="long" />
 
[[Category:Fedora 14 Test Days]]

Latest revision as of 23:54, 18 June 2015

DATE TIME WHERE
2010-08-26 From 9:00 to 17:00 UTC #fedora-test-day (webirc)
Can't make the date?
If you come to this page before or after the test day is completed, your testing is still valuable, and you can use the information on this page to test, file any bugs you find at Bugzilla, and add your results to the results section. If this page is more than a month old when you arrive here, please check the current schedule and see if a similar but more recent Test Day is planned or has already happened.

What to test?[edit]

Have you ever used any security scanning application? Does the security configuration of your box matters? Do you want to keep you system in consistent state? If you have positive answer to any of these questions then it's probably worth to joint this Fedora Test Day that will focus on OpenSCAP feature.

What is SCAP? It is a line of standards managed by NIST. It was created to provide a standardized approach to maintaining the security of systems, such as automatically verifying the presence of patches, checking system security configuration settings, and examining systems for signs of compromise.

The SCAP suite contains multiple complex data exchange formats that are to be used to transmit important vulnerability, configuration, and other security data. Historically, there have been few tools that provide a way to query this data in the needed format. This lack of tools makes the barrier to entry very high and discourages adoption of these protocols by the community. It's a goal of OpenSCAP project to create a framework of libraries to improve the accessibility of SCAP and enhance the usability of the information it represents.

Who's available[edit]

The following cast of characters will be available testing, workarounds, bug fixes, and general discussion:

  • Development - Peter Vrabec (wrabco), Tomas Heinrich (theinric), Maros Barabas (mbarabas), Daniel Kopecek (dkopecek), Lukas Kuklinek (lkukline)
  • FirstAidKit development - Martin Sivák (msivak)
  • Quality Assurance - Kamil Páral (kparal), Ondrej Moris (omoris)

Prerequisite for Test Day[edit]

  • A fully updated Fedora 13 or 14.
    • This must be a real installation, live CDs are unfortunately not suitable for this test day.
    • We are interested in different software setups, so if possible please use your real workstation, rather than clean install of F13 or F14. You don't have to be afraid, this software is not destructive in any way.
  • At least 2 GB of RAM is recommended for the system, otherwise the tool may work very slow.

How to test?[edit]

  1. Fully update your Fedora 13 or Fedora 14.
  2. Install openscap, openscap-utils and openscap-python packages version 0.6.1-testday5. Download them from: http://people.redhat.com/pvrabec/openscap/
    Packages updated
    Packages have been updated to fix numerous errors. Please update if you've downloaded the old ones.
  3. Download required SCAP content: http://people.redhat.com/pvrabec/openscap/content
    Files updated
    SCAP content was updated during this test day. Please update if you've downloaded the old one.
  4. Follow the test cases below.
  5. Write your results to the result matrix.

Test Cases[edit]

Please execute as many test cases from the following list of OpenSCAP Test Cases as possible:

Test Results[edit]

If you have problems with any of the tests, report a bug to Bugzilla for the openscap component. If you are unsure about exactly how to file the report or what other information to include, just ask on IRC and we will help you. Once you have completed the tests, add your results to the Results table below, following the example results from the first line as a template.

User Fedora default settings Fedora adjusted settings secstate tool FAK plugin References
Sample User
Pass pass
Warning warn
[1]
Fail fail
[2]
none
  1. ↑ Test pass, but also encountered RHBZ #54321
  2. ↑ RHBZ #12345
newgle1
Fail fail newgle1
[1]
Fail fail newgle1
[1]
none
none
  1. ↑ 1.0 1.1 err:*** buffer overflow detected ***: oscap terminated
He Rui
Fail fail rhe
[1]
Fail fail rhe
[long 1]
none
none
  1. ↑ buffer overflowed and some rules failed: http://fpaste.org/wSvq/
Jan Kaluza
Fail fail jkaluza
[1]
none
none
none
  1. ↑ buffer overflowed - RHBZ #627488
Petr Pisar
Fail fail ppisar
[long 2]
Fail fail ppisar
[long 3]
Fail fail ppisar
[long 4]
Pass pass ppisar
[1]
Fail fail ppisar
[long 5]
Fail fail ppisar
[2]
none
none
  1. ↑ Other tests passed
  2. ↑ Test rule-3.7.1.1.a (Disable Avahi Server Software) fails even if avahi-deamon is disabled in all runlevels and none is running
Jan Gorig
Fail fail jgorig
[1]
Pass pass jgorig
[2]
none
none
none
  1. ↑ same problem - buffer overflowed on x86_64 F13 - RHBZ #627488
  2. ↑ bug fixed
Kushal Das
Fail fail kushal
[1]
none
none
none
  1. ↑ same problem - buffer overflowed on x86 F13 - RHBZ #627488
David Ramsey
Fail fail
[1]
none
none
none
  1. ↑ Same problem with buffer overflowed on x86 F14
Miroslav Grepl
Pass pass mgrepl
[1]
none
none
none
  1. ↑ Test finished (fixed pkgs from koji)
Ondrej Moriš
Fail fail omoris
[1]
none
none
Warning warn omoris
[2]
  1. ↑ test finished (fixed pkgs from koji) with several fails: http://fpaste.org/Sgys/
  2. ↑ getting error while changing some variable values (HTTP reply/request), gui is mostly not updated during evaluation
Masami Ichikawa
Fail fail masami
[1]
Fail fail masami
[2]
none
none
none
  1. ↑ same problem - buffer overflowed on x86 F14 - RHBZ #627488
  2. ↑ testday5 fails rule-2.5.1.2.b (Set net.ipv4.conf.all.accept_redirects for Hosts and Routers). same as RHBZ #627600
Kamil Páral
Warning warn kparal
[1]
Fail fail kparal
none
Fail fail kparal
[2]
Pass pass kparal
[3]
Fail fail kparal
[4]
  1. ↑ Set net.ipv4.conf.all.accept_redirects for Hosts and Routers fails RHBZ #627600
  2. ↑ Not Selected: 0 in http://fpaste.org/4Okv/
  3. ↑ openscap-0.6.1-testday4.fc14
  4. ↑ Values in test and policy selection allows "0.5" and "enforcingaaa"
David Paige
Pass pass David.Paige
[1]
none
none
none
  1. ↑ 'No errors, five failed individual tests.

Long comments[edit]

  1. ↑ tested the rule-2.2.2.3.a (Disable the Automounter if Possible), when I stopped the autofs service as the rules suggested, the result was still 'fail'.(Yum remove autofs can get a 'pass' result)
  2. ↑ Tests checking file permissions (rule-2.2.3.3.a, rule-2.2.3.4.a, rule-2.2.3.4.b, rule-2.2.3.5.a, rule-2.2.3.5.b, rule-2.2.3.6.a) eats all memory (4 GiB) and are terminated by kernel – RHBZ #565691
  3. ↑ Test rule-2.1.2.3.4.a (Ensure Package Signature Checking is Not Disabled For Any Repos) fails because I have defined rawhide repositories with disabled signature checking and disabled for installation. I think disabled repositories should not be considered in this test.
  4. ↑ Test rule-2.5.1.2.b (Set net.ipv4.conf.all.accept_redirects for Hosts and Routers) fails. This is default value for F13. F13 should be fixed (/etc/sysctl.conf) or the test removed as far as it can be useful in some scenarios (link with more routers, link with more IP networks).
  5. ↑ Test rule-3.6.1.1.a (Disable X Windows at System Boot) fails if enabled despite my inittab has default runlevel 3. Test is defined as equality to number 5 in oval file. More ever X Windows' is nonsense. Correct name is X Window' without the `s' suffix. See X(7) manual page. You are breaking trade mark ;)
Retrieved from "https://fedoraproject.org/w/index.php?title=Test_Day:2010-08-26_OpenSCAP&oldid=416091"