https://www.opencompute.org/w/api.php?action=feedcontributions&user=Jlane&feedformat=atomOpenCompute - User contributions [en]2024-03-28T15:46:46ZUser contributionsMediaWiki 1.39.2https://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=1079C&I Wiki Portal/OCP Ready2015-04-08T18:39:54Z<p>Jlane: /* Build your own */</p>
<hr />
<div>== OCP Ready Test Suite Packages ==<br />
<br />
OCP Ready Test Suite is a different, more generalized test to ensure your hardware works and could be ready for full OCP Certification. <br />
<br />
=== Installing ===<br />
<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
<br />
The OCP Ready packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-ready</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-ready<br />
<br />
and follow the prompts.<br />
<br />
=== Obtaining the Source Code ===<br />
The tools are fully open source under a GPL v3 license and freely available for download here:<br />
<br />
http://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
OR via bazaar:<br />
<br />
bzr branch lp:opencompute/opencompute-testing<br />
<br />
<br />
<br />
== OCP Ready Thumbdrive ==<br />
<br />
'''NOTE: The Thumbdrive is being deprecated and likely will not be maintained. Eventually, this will be updated with instructions for creating your own'''<br />
<br />
The files for the thumbdrive will be available here:<br />
http://people.canonical.com/~jlane/ocp-thumb-drive<br />
<br />
Note that this is a rather large download (> 1GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.<br />
<br />
Additionally, the OCP Ready Thumb Drive is based on old code and is not really considered reliable. It's a nice way to demo and explore, but not really something you should be using to pre-test your hardware.<br />
<br />
=== Build your own ===<br />
Building your own is not terribly difficult. The following will allow you to build your own USB Thumbdrive and assumes you are using Ubuntu to do the building. ''Note: This is not the same process we used to create the original USB images, that process is far more involved, but this will suffice for getting you a bootable device.''<br />
<br />
You will need the following:<br />
* A USB thumb drive that is at least 2GB in size<br />
* A Ubuntu Desktop or Server 14.04 LTS ISO image. (Later versions will also work)<br />
** We use the Desktop image to provide a more useful system. Please note that the actual test tools are console based so using Desktop or Server here is completely at your discretion.<br />
* A computer running Ubuntu (Desktop here, we need the GTK Tools for the following, there is likely a way to do this purely from console, this is just easier and faster to describe).<br />
<br />
These next steps will create the bootable USB stick initially.<br />
# Insert the USB thumb drive into your computer running Ubuntu Desktop.<br />
# Open the Dash (Hit the Windows key or click on the top most icon on the Unity Panel).<br />
# Type USB and when you see '''Startup Disk Creator''' click on that to launch the tool.<br />
# Set the following items in USB Creator GTK (Also known as Startup Disk Creator).<br />
## Under '''Source disc image''', click '''Other''' and locate your Ubuntu ISO image and click '''Open'''. It should now appear in the upper display.<br />
## In '''Disk to use''', select the USB Thumb Drive you wish to build.<br />
## Click '''Erase Disk''' to clear the disk of all data. You will need to provide your password to do this.<br />
## Select '''Stored in reserved extra space''' and move the slider to at least 1GB, preferably more if your Thumb Drive has the space.<br />
# Click '''Make startup disk'''.<br />
## This will take a few moments. You will see a password prompt that you must use before the bootloader can be written to the thumb drive.<br />
# When complete, exit the Startup Disk Creator.<br />
<br />
Once the above steps are complete, you should now have a bootable Ubuntu USB Thumb drive with persistent storage of between 1 - 4 GB. Now you need to boot something from that and complete the next steps to have a bootable USB drive with OCP Ready on it.<br />
# Insert the USB thumb drive into a machine that has internet access.<br />
# Boot the machine and select the USB thumb drive from the boot menu if that machine doesn't automatically boot from USB.<br />
## In the boot menu, you may see two entries, one for EFI and one for Legacy BIOS. Use the Legacy BIOS entry to load the bootloader.<br />
# At the boot menu, select '''Try Ubuntu without installing'''. This is the default selection.<br />
# After Ubuntu has booted, open a console by opening the Dash and typing "Terminal" into the search box and clicking on the Terminal icon.<br />
# In the Terminal, perform the following steps:<br />
## <code>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa</code><br />
### Hit Enter to add the PPA when prompted.<br />
## <code>$ sudo add-apt-repository ppa:hardware-certification/public</code><br />
### Again, hit Enter to add the PPA when prompted.<br />
## <code>$ sudo apt update</code><br />
## <code>$ sudo apt upgrade>/code><br />
## <code>$ sudo apt install opencompute-ready</code><br />
### Note this may fail because the Live USB environment lacks the Universe repository by default. If this does fail for you, the following will resolve dependency problems:<br />
#### <code>$ sudo add-apt-repository universe</code><br />
#### <code>$ sudo apt update</code><br />
#### <code>$ sudo apt install opencompute-ready</code><br />
<br />
At this point, you should now have a working USB thumb drive with opencompute-ready installed. To test this, do the following:<br />
# Click the Gear icon in the upper right corner.<br />
# Click '''Shut Down...'''<br />
# Shut the system down.<br />
# After it is powered off fully, power it back on.<br />
## Again, you may need to enter the Boot Menu and force the system to boot from the USB stick if that is not in your default boot order.<br />
# Select '''Try Ubuntu without installing''' from the boot menu, as before.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=1078C&I Wiki Portal/OCP Ready2015-04-08T18:38:34Z<p>Jlane: /* Build your own */</p>
<hr />
<div>== OCP Ready Test Suite Packages ==<br />
<br />
OCP Ready Test Suite is a different, more generalized test to ensure your hardware works and could be ready for full OCP Certification. <br />
<br />
=== Installing ===<br />
<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
<br />
The OCP Ready packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-ready</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-ready<br />
<br />
and follow the prompts.<br />
<br />
=== Obtaining the Source Code ===<br />
The tools are fully open source under a GPL v3 license and freely available for download here:<br />
<br />
http://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
OR via bazaar:<br />
<br />
bzr branch lp:opencompute/opencompute-testing<br />
<br />
<br />
<br />
== OCP Ready Thumbdrive ==<br />
<br />
'''NOTE: The Thumbdrive is being deprecated and likely will not be maintained. Eventually, this will be updated with instructions for creating your own'''<br />
<br />
The files for the thumbdrive will be available here:<br />
http://people.canonical.com/~jlane/ocp-thumb-drive<br />
<br />
Note that this is a rather large download (> 1GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.<br />
<br />
Additionally, the OCP Ready Thumb Drive is based on old code and is not really considered reliable. It's a nice way to demo and explore, but not really something you should be using to pre-test your hardware.<br />
<br />
=== Build your own ===<br />
Building your own is not terribly difficult. The following will allow you to build your own USB Thumbdrive and assumes you are using Ubuntu to do the building. ''Note: This is not the same process we used to create the original USB images, that process is far more involved, but this will suffice for getting you a bootable device.''<br />
<br />
You will need the following:<br />
* A USB thumb drive that is at least 2GB in size<br />
* A Ubuntu Desktop or Server 14.04 LTS ISO image. (Later versions will also work)<br />
** We use the Desktop image to provide a more useful system. Please note that the actual test tools are console based so using Desktop or Server here is completely at your discretion.<br />
* A computer running Ubuntu (Desktop here, we need the GTK Tools for the following, there is likely a way to do this purely from console, this is just easier and faster to describe).<br />
<br />
# Insert the USB thumb drive into your computer running Ubuntu Desktop.<br />
# Open the Dash (Hit the Windows key or click on the top most icon on the Unity Panel).<br />
# Type USB and when you see '''Startup Disk Creator''' click on that to launch the tool.<br />
# Set the following items in USB Creator GTK (Also known as Startup Disk Creator).<br />
## Under '''Source disc image''', click '''Other''' and locate your Ubuntu ISO image and click '''Open'''. It should now appear in the upper display.<br />
## In '''Disk to use''', select the USB Thumb Drive you wish to build.<br />
## Click '''Erase Disk''' to clear the disk of all data. You will need to provide your password to do this.<br />
## Select '''Stored in reserved extra space''' and move the slider to at least 1GB, preferably more if your Thumb Drive has the space.<br />
# Click '''Make startup disk'''.<br />
## This will take a few moments. You will see a password prompt that you must use before the bootloader can be written to the thumb drive.<br />
# When complete, exit the Startup Disk Creator.<br />
<br />
Once the above steps are complete, you should now have a bootable Ubuntu USB Thumb drive with persistent storage of between 1 - 4 GB. Now you need to boot something from that and complete the next steps to have a bootable USB drive with OCP Ready on it.<br />
# Insert the USB thumb drive into a machine that has internet access.<br />
# Boot the machine and select the USB thumb drive from the boot menu if that machine doesn't automatically boot from USB.<br />
## In the boot menu, you may see two entries, one for EFI and one for Legacy BIOS. Use the Legacy BIOS entry to load the bootloader.<br />
# At the boot menu, select '''Try Ubuntu without installing'''. This is the default selection.<br />
# After Ubuntu has booted, open a console by opening the Dash and typing "Terminal" into the search box and clicking on the Terminal icon.<br />
# In the Terminal, perform the following steps:<br />
## <code>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa</code><br />
### Hit Enter to add the PPA when prompted.<br />
## <code>$ sudo add-apt-repository ppa:hardware-certification/public</code><br />
### Again, hit Enter to add the PPA when prompted.<br />
## <code>$ sudo apt update</code><br />
## <code>$ sudo apt upgrade>/code><br />
## <code>$ sudo apt install opencompute-ready</code><br />
### Note this may fail because the Live USB environment lacks the Universe repository by default. If this does fail for you, the following will resolve dependency problems:<br />
#### <code>$ sudo add-apt-repository universe</code><br />
#### <code>$ sudo apt update</code><br />
#### <code>$ sudo apt install opencompute-ready</code><br />
<br />
At this point, you should now have a working USB thumb drive with opencompute-ready installed. To test this, do the following:<br />
# Click the Gear icon in the upper right corner.<br />
# Click '''Shut Down...'''<br />
# Shut the system down.<br />
# After it is powered off fully, power it back on.<br />
## Again, you may need to enter the Boot Menu and force the system to boot from the USB stick if that is not in your default boot order.<br />
# Select '''Try Ubuntu without installing''' from the boot menu, as before.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=1077C&I Wiki Portal/OCP Ready2015-04-08T18:37:37Z<p>Jlane: </p>
<hr />
<div>== OCP Ready Test Suite Packages ==<br />
<br />
OCP Ready Test Suite is a different, more generalized test to ensure your hardware works and could be ready for full OCP Certification. <br />
<br />
=== Installing ===<br />
<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
<br />
The OCP Ready packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-ready</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-ready<br />
<br />
and follow the prompts.<br />
<br />
=== Obtaining the Source Code ===<br />
The tools are fully open source under a GPL v3 license and freely available for download here:<br />
<br />
http://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
OR via bazaar:<br />
<br />
bzr branch lp:opencompute/opencompute-testing<br />
<br />
<br />
<br />
== OCP Ready Thumbdrive ==<br />
<br />
'''NOTE: The Thumbdrive is being deprecated and likely will not be maintained. Eventually, this will be updated with instructions for creating your own'''<br />
<br />
The files for the thumbdrive will be available here:<br />
http://people.canonical.com/~jlane/ocp-thumb-drive<br />
<br />
Note that this is a rather large download (> 1GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.<br />
<br />
Additionally, the OCP Ready Thumb Drive is based on old code and is not really considered reliable. It's a nice way to demo and explore, but not really something you should be using to pre-test your hardware.<br />
<br />
=== Build your own ===<br />
Building your own is not terribly difficult. The following will allow you to build your own USB Thumbdrive and assumes you are using Ubuntu to do the building. ''''Note: This is not the same process we used to create the original USB images, that process is far more involved, but this will suffice for getting you a bootable device.''''<br />
<br />
You will need the following:<br />
* A USB thumb drive that is at least 2GB in size<br />
* A Ubuntu Desktop or Server 14.04 LTS ISO image. (Later versions will also work)<br />
** We use the Desktop image to provide a more useful system. Please note that the actual test tools are console based so using Desktop or Server here is completely at your discretion.<br />
* A computer running Ubuntu (Desktop here, we need the GTK Tools for the following, there is likely a way to do this purely from console, this is just easier and faster to describe).<br />
<br />
# Insert the USB thumb drive into your computer running Ubuntu Desktop.<br />
# Open the Dash (Hit the Windows key or click on the top most icon on the Unity Panel).<br />
# Type USB and when you see '''Startup Disk Creator''' click on that to launch the tool.<br />
# Set the following items in USB Creator GTK (Also known as Startup Disk Creator).<br />
## Under '''Source disc image''', click '''Other''' and locate your Ubuntu ISO image and click '''Open'''. It should now appear in the upper display.<br />
## In '''Disk to use''', select the USB Thumb Drive you wish to build.<br />
## Click '''Erase Disk''' to clear the disk of all data. You will need to provide your password to do this.<br />
## Select '''Stored in reserved extra space''' and move the slider to at least 1GB, preferably more if your Thumb Drive has the space.<br />
# Click '''Make startup disk'''.<br />
## This will take a few moments. You will see a password prompt that you must use before the bootloader can be written to the thumb drive.<br />
# When complete, exit the Startup Disk Creator.<br />
<br />
Once the above steps are complete, you should now have a bootable Ubuntu USB Thumb drive with persistent storage of between 1 - 4 GB. Now you need to boot something from that and complete the next steps to have a bootable USB drive with OCP Ready on it.<br />
# Insert the USB thumb drive into a machine that has internet access.<br />
# Boot the machine and select the USB thumb drive from the boot menu if that machine doesn't automatically boot from USB.<br />
## In the boot menu, you may see two entries, one for EFI and one for Legacy BIOS. Use the Legacy BIOS entry to load the bootloader.<br />
# At the boot menu, select '''Try Ubuntu without installing'''. This is the default selection.<br />
# After Ubuntu has booted, open a console by opening the Dash and typing "Terminal" into the search box and clicking on the Terminal icon.<br />
# In the Terminal, perform the following steps:<br />
## <code>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa</code><br />
### Hit Enter to add the PPA when prompted.<br />
## <code>$ sudo add-apt-repository ppa:hardware-certification/public</code><br />
### Again, hit Enter to add the PPA when prompted.<br />
## <code>$ sudo apt update</code><br />
## <code>$ sudo apt upgrade>/code><br />
## <code>$ sudo apt install opencompute-ready</code><br />
### Note this may fail because the Live USB environment lacks the Universe repository by default. If this does fail for you, the following will resolve dependency problems:<br />
#### <code>$ sudo add-apt-repository universe</code><br />
#### <code>$ sudo apt update</code><br />
#### <code>$ sudo apt install opencompute-ready</code><br />
<br />
At this point, you should now have a working USB thumb drive with opencompute-ready installed. To test this, do the following:<br />
# Click the Gear icon in the upper right corner.<br />
# Click '''Shut Down...'''<br />
# Shut the system down.<br />
# After it is powered off fully, power it back on.<br />
## Again, you may need to enter the Boot Menu and force the system to boot from the USB stick if that is not in your default boot order.<br />
# Select '''Try Ubuntu without installing''' from the boot menu, as before.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=1071C&I Wiki Portal/OCP Ready2015-04-08T16:06:09Z<p>Jlane: </p>
<hr />
<div>== OCP Ready Test Suite Packages ==<br />
<br />
OCP Ready Test Suite is a different, more generalized test to ensure your hardware works and could be ready for full OCP Certification. <br />
<br />
=== Installing ===<br />
<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
<br />
The OCP Ready packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-ready</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-ready<br />
<br />
and follow the prompts.<br />
<br />
=== Obtaining the Source Code ===<br />
The tools are fully open source under a GPL v3 license and freely available for download here:<br />
<br />
http://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
OR via bazaar:<br />
<br />
bzr branch lp:opencompute/opencompute-testing<br />
<br />
<br />
<br />
== OCP Ready Thumbdrive ==<br />
<br />
'''NOTE: The Thumbdrive is being deprecated and likely will not be maintained. Eventually, this will be updated with instructions for creating your own'''<br />
<br />
The files for the thumbdrive will be available here:<br />
http://people.canonical.com/~jlane/ocp-thumb-drive<br />
<br />
Note that this is a rather large download (> 1GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.<br />
<br />
Additionally, the OCP Ready Thumb Drive is based on old code and is not really considered reliable. It's a nice way to demo and explore, but not really something you should be using to pre-test your hardware.<br />
<br />
=== Build your own ===<br />
Building your own is not terribly difficult. The following will allow you to build your own USB Thumbdrive and assumes you are using Ubuntu to do the building. ''''Note: This is not the same process we used to create the original USB images, that process is far more involved, but this will suffice for getting you a bootable device.''''<br />
<br />
You will need the following:<br />
* A USB thumb drive that is at least 2GB in size<br />
* A Ubuntu Desktop or Server 14.04 LTS ISO image. (Later versions will also work)<br />
** We use the Desktop image to provide a more useful system. Please note that the actual test tools are console based so using Desktop or Server here is completely at your discretion.<br />
* A computer running Ubuntu (Desktop here, we need the GTK Tools for the following, there is likely a way to do this purely from console, this is just easier and faster to describe).<br />
<br />
# Insert the USB thumb drive into your computer running Ubuntu Desktop.<br />
# Open the Dash (Hit the Windows key or click on the top most icon on the Unity Panel).<br />
# Type USB and when you see '''Startup Disk Creator''' click on that to launch the tool.<br />
# Set the following items in USB Creator GTK (Also known as Startup Disk Creator).<br />
## Under '''Source disc image''', click '''Other''' and locate your Ubuntu ISO image and click '''Open'''. It should now appear in the upper display.<br />
## In '''Disk to use''', select the USB Thumb Drive you wish to build.<br />
## Click '''Erase Disk''' to clear the disk of all data. You will need to provide your password to do this.<br />
## Select '''Stored in reserved extra space''' and move the slider to at least 1GB, preferably more if your Thumb Drive has the space.<br />
# Click '''Make startup disk'''.<br />
## This will take a few moments. You will see a password prompt that you must use before the bootloader can be written to the thumb drive.<br />
# When complete, exit the Startup Disk Creator.<br />
<br />
Once the above steps are complete, you should now have a bootable Ubuntu USB Thumb drive with persistent storage of between 1 - 4 GB. Now you need to boot something from that and complete the next steps to have a bootable USB drive with OCP Ready on it.<br />
# Insert the USB thumb drive into a machine that has internet access.<br />
# Boot the machine and select the USB thumb drive from the boot menu if that machine doesn't automatically boot from USB.<br />
# At the boot menu, select '''Try Ubuntu without installing'''. This is the default selection.<br />
# After Ubuntu has booted, open a console by opening the Dash and typing "Terminal" into the search box and clicking on the Terminal icon.<br />
# In the Terminal, perform the following steps:<br />
## <code>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa</code><br />
### Hit Enter to add the PPA when prompted.<br />
## <code>$ sudo add-apt-repository ppa:hardware-certification/public</code><br />
### Again, hit Enter to add the PPA when prompted.<br />
## <code>$ sudo apt update</code><br />
## <code>$ sudo apt upgrade>/code><br />
## <code>$ sudo apt install opencompute-ready</code><br />
### Note this may fail because the Live USB environment lacks the Universe repository by default. If this does fail for you, the following will resolve dependency problems:<br />
#### <code>$ sudo add-apt-repository universe</code><br />
#### <code>$ sudo apt update</code><br />
#### <code>$ sudo apt install opencompute-ready</code><br />
<br />
At this point, you should now have a working USB thumb drive with opencompute-ready installed. To test this, do the following:<br />
# Click the Gear icon in the upper right corner.<br />
# Click '''Shut Down...'''<br />
# Shut the system down.<br />
# After it is powered off fully, power it back on.<br />
## Again, you may need to enter the Boot Menu and force the system to boot from the USB stick if that is not in your default boot order.<br />
# Select '''Try Ubuntu without installing''' from the boot menu, as before.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=1070C&I Wiki Portal/OCP Ready2015-04-08T14:49:13Z<p>Jlane: </p>
<hr />
<div>== OCP Ready Test Suite Packages ==<br />
<br />
OCP Ready Test Suite is a different, more generalized test to ensure your hardware works and could be ready for full OCP Certification. <br />
<br />
=== Installing ===<br />
<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
<br />
The OCP Ready packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-ready</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-ready<br />
<br />
and follow the prompts.<br />
<br />
=== Obtaining the Source Code ===<br />
The tools are fully open source under a GPL v3 license and freely available for download here:<br />
<br />
http://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
OR via bazaar:<br />
<br />
bzr branch lp:opencompute/opencompute-testing<br />
<br />
<br />
<br />
== OCP Ready Thumbdrive ==<br />
<br />
'''NOTE: The Thumbdrive is being deprecated and likely will not be maintained. Eventually, this will be updated with instructions for creating your own'''<br />
<br />
The files for the thumbdrive will be available here:<br />
http://people.canonical.com/~jlane/ocp-thumb-drive<br />
<br />
Note that this is a rather large download (> 1GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.<br />
<br />
Additionally, the OCP Ready Thumb Drive is based on old code and is not really considered reliable. It's a nice way to demo and explore, but not really something you should be using to pre-test your hardware.<br />
<br />
=== Build your own ===<br />
Building your own is not terribly difficult. The following will allow you to build your own USB Thumbdrive and assumes you are using Ubuntu to do the building. ''''Note: This is not the same process we used to create the original USB images, that process is far more involved, but this will suffice for getting you a bootable device.''''<br />
<br />
You will need the following:<br />
* A USB thumb drive that is at least 2GB in size<br />
* A Ubuntu Desktop or Server 14.04 LTS ISO image. (Later versions will also work)<br />
** We use the Desktop image to provide a more useful system. Please note that the actual test tools are console based so using Desktop or Server here is completely at your discretion.<br />
* A computer running Ubuntu (Desktop here, we need the GTK Tools for the following, there is likely a way to do this purely from console, this is just easier and faster to describe).<br />
<br />
# Insert the USB thumb drive into your computer running Ubuntu Desktop.<br />
# Open the Dash (Hit the Windows key or click on the top most icon on the Unity Panel).<br />
# Type USB and when you see '''Startup Disk Creator''' click on that to launch the tool.<br />
# Set the following items in USB Creator GTK (Also known as Startup Disk Creator).<br />
## Under '''Source disc image''', click '''Other''' and locate your Ubuntu ISO image and click '''Open'''. It should now appear in the upper display.<br />
## In '''Disk to use''', select the USB Thumb Drive you wish to build.<br />
## Click '''Erase Disk''' to clear the disk of all data. You will need to provide your password to do this.<br />
## Select '''Stored in reserved extra space''' and move the slider to at least 1GB, preferably more if your Thumb Drive has the space.<br />
# Click '''Make startup disk'''.<br />
## This will take a few moments. You will see a password prompt that you must use before the bootloader can be written to the thumb drive.<br />
# When complete, exit the Startup Disk Creator.<br />
<br />
Once the above steps are complete, you should now have a bootable Ubuntu USB Thumb drive with persistent storage of between 1 - 4 GB. Now you need to boot something from that and complete the next steps to have a bootable USB drive with OCP Ready on it.<br />
# Insert the USB thumb drive into a machine that has internet access.<br />
# Boot the machine and select the USB thumb drive from the boot menu if that machine doesn't automatically boot from USB.<br />
# At the boot menu, select '''Try Ubuntu without installing'''. This is the default selection.<br />
# After Ubuntu has booted, open a console by opening the Dash and typing "Terminal" into the search box and clicking on the Terminal icon.<br />
# In the Terminal, perform the following steps:<br />
## <code>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa</code><br />
### Hit Enter to add the PPA when prompted.<br />
## <code>$ sudo add-apt-repository ppa:hardware-certification/public</code><br />
### Again, hit Enter to add the PPA when prompted.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal&diff=1064C&I Wiki Portal2015-04-02T20:35:40Z<p>Jlane: /* Tools */</p>
<hr />
<div>==WELCOME==<br />
<br />
:Welcome to the OCP '''Compliance & Interoperability''' Project.<br />
<br />
:This Project is open to the public and we welcome all those who would like to be involved.<br />
<br />
:[[C&I FAQ]]: Am I in the right place and other questions.<br />
<br />
==Get Involved==<br />
<br />
===Events=== <br />
<br />
:The Open Compute Project meets annually at the OCP Summit events and periodically at regional OCP events. More information on these events and other OCP activities can be found here: http://www.opencompute.org/community/events/<br />
<br />
===Tools===<br />
<br />
:[[C&I_Wiki_Portal/OCP_Ready|OCP Ready]]<br />
:[[C&I_WIki_Portal/OCP_Checkbox|OCP Checkbox]]<br />
<br />
:[[C%26I_Wiki_Portal/ToolsDevelopmentGuideline|Upstreaming and Coding]]<br />
<br />
== Monthly Project Meetings (online)==<br />
<br />
The first (1st) and third (3rd) Wednesday 7am Taiwan.<br />
<br />
To find out the time of this meeting in your timezone go to: <br />
http://www.timeanddate.com/worldclock/converter.html<br />
<br />
This call is open to the public. <br />
<br />
===How to join the call===<br />
* Meeting URL http://fuze.me/22568267<br />
<br />
* Toll / Intl #: +1 (201) 479-4595<br />
* Toll free #: +1 (855) 346-3893<br />
* Meeting number: 23228335<br />
<br />
'''Please Note:''' Just a reminder that this call will be recorded and shared in the meeting notes.<br />
--------------<br />
<br />
===Past Meetings===<br />
<br />
:* List of meeting notes/minutes can be found here:[[C%26I_Meeting_Notes|Meeting Notes]]<br />
<br />
===Communication===<br />
<br />
:* Mailing List - opencompute-ci@lists.opencompute.org <br />
:* Subscribe to mailing list - http://lists.opencompute.org/mailman/listinfo/opencompute-ci<br />
:* IRC Channel - #ocp-ci on irc.freenode.ne<br />
:* Website - http://www.opencompute.org/projects/compliance-and-interoperability/<br />
<br />
<br />
===Participating===<br />
:A list of active projects at C&I. We welcome you to join us and contribute. [[C&I Project Tracks & Contact List]]<br />
<br />
==Contacts==<br />
<br />
===Chair===<br />
:[mailto:yf.juan@itri.org.tw YF Juan], ITRI<br />
:: Voice Mail: +1 650.257.0904<br />
<br />
===OCP Foundation Liaison===<br />
:* [mailto:thaoannguyen@fb.com Thao Nguyen], Director of Certification Labs<br />
<br />
===OCP Labs===<br />
:*[[File:ITRI_logo2.png|50px]] ITRI Labs, Hsinchu, Taiwan. [mailto:chilung@itri.org.tw Chilung Wang], Director<br />
:*[[File:TsinghuaUniversitylogo1.png|25px]] Tsinghua University, Beijing, China. [mailto:wei.xu.0@gmail.com Wei Xu], Director<br />
:*[[File:UTSA logo1.jpg|25px]] University of Texas San Antonio, San Antonio, United States. [mailto:Peyman.Najafirad@utsa.edu Paul Rad], Director<br />
----<br />
<br />
===C&I Project Chairs Emeritus===<br />
:* Matthew Liste, Goldman Sachs<br />
:* Eric Wells, Fidelity<br />
:* DaWane Wanek, Avnet</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/ToolsDevelopmentGuideline&diff=1063C&I Wiki Portal/ToolsDevelopmentGuideline2015-04-02T20:33:16Z<p>Jlane: /* Questions? */</p>
<hr />
<div>== Overview ==<br />
The official repository for the source code to OCP Certification and OCP Ready is [http://launchpad.net/opencompute Launchpad]. There are a number of projects hosted there but the most important one is [http://code.launchpad.net/opencompute-testing opencompute-testing].<br />
<br />
The '''opencompute-testing''' project contains all the code for generating the OCP Certification and OCP Ready testing tools. It is the code that creates the Plainbox Provider and Launcher Packages, hosting the job descriptions that are OCP Specific, scripts and whitelists. Package builds occur automatically once per day if the code base has been updated.<br />
<br />
=== Contacts ===<br />
If you have ANY questions at all about the code base, contributing, or anything else related, contact [mailto:jeff@canonical.com Jeff Lane]<br />
For anything related to Open Compute Certification (including code questions):<br />
* Mailing List: http://lists.opencompute.org/mailman/listinfo/opencompute-ci<br />
* IRC: [http://freenode.net Freenode IRC Servers] #ocp-ci<br />
<br />
== Basics ==<br />
This section will cover some of the basics for contributing to the code for these tools. This is a general overview, more specific information can be found on line at the links below.<br />
<br />
=== Bazaar ===<br />
==== About Bazaar ====<br />
Bazaar is a version control system that helps you track project history over time and to collaborate easily with others. Whether you're a single developer, a co-located team or a community of developers scattered across the world, Bazaar scales and adapts to meet your needs. Part of the GNU Project, Bazaar is free software sponsored by Canonical<br />
<br />
You can learn more at http://bazaar.canonical.com/en/<br />
<br />
==== Obtaining Bazaar ====<br />
To install Bazaar on a debian or ubuntu system:<br />
<code>sudo apt-get install bazaar</code><br />
<br />
For other Operating Systems, including Windows, Red Hat, SuSE, OSX and more, please see http://wiki.bazaar.canonical.com/Download<br />
<br />
==== Learning Bazaar ====<br />
Bazaar is easy to learn. There is an excellent [http://doc.bazaar.canonical.com/latest/en/mini-tutorial/ 5 Minute Tutorial] that should get you started. Beyond that, bzr has a fairly well developed man page and in-program help system. <br />
<br />
Please read and do the 5 Minute Tutorial now, which will teach you the basics like creating branches, logging into LP, pushing and merging.<br />
<br />
===== Trees, Branches, Trunks? =====<br />
The basic top level of a codebase is called the Trunk. The Trunk is the mainline code that all branches are a part of. Branches are localized copies of Trunk. Branches are where you make changes for submission and merge consideration. Once accepted, Branches are merged into Trunk and become part of the mainline codebase. <br />
<br />
===== Basic Commands =====<br />
* The most basic command is: <code>bzr</code><br />
* To get a local copy of a project: <code>bzr branch lp:opencompute/opencompute-testing</code><br />
* To review a change summary: <code>bzr status</code><br />
* To add new files that are not yet managed: <code>bzr add path/to/filename</code><br />
** Note that /path/to/filename needs to be a path INSIDE your branch. When adding files not in the branch, you need to copy those files into the branch before adding them.<br />
* To see a diff of changes: <code>bzr diff path/to/filename</code><br />
* To commit your changes to a new revision <code>bzr commit -m "Some descriptive summary of your changes"</code><br />
* To push your changes to launchpad in preparation for a Merge Request: <code>bzr push lp:~<Your Launchpad Username>/opencompute/<Your Branch Name></code><br />
** for example: <code>bzr push lp:~bladernr/opencompute/ocp-testing-job-updates</code><br />
<br />
There are many MANY more commands but those are the ones you'll likely use most frequently.<br />
<br />
===== Learning More =====<br />
* http://doc.bazaar.canonical.com/en/<br />
* http://wiki.bazaar.canonical.com/Tutorials<br />
* https://yade-dem.org/wiki/Quick_Bazaar_tutorial<br />
* http://wiki.bazaar.canonical.com/Tutorials<br />
<br />
=== Practices ===<br />
==== Languages to use ====<br />
===== Main Tooling =====<br />
Plainbox, Checkbox, the providers and such are all written in Python3.<br />
<br />
===== Job Files and Whitelists =====<br />
Job Files are the same as they were in Checkbox, that is RFC822 compliant text files. See any file in the <code>jobs/</code> directory for examples.<br />
Whitelists are also simply text files, HOWEVER, you can also use regular expressions in job naming to pattern match jobs.<br />
<br />
===== Test Scripts =====<br />
These are scripts that perform actual testing. For example, a storage IO test, or a network performance test. Scripts can be written in pretty much anything that is compiled or has a valid interpreter installed on the System Under Test. The following are just a few examples of the languages you can write test scripts in:<br />
* Python3<br />
* Bash<br />
* Ruby<br />
* Perl<br />
<br />
And some more complex ones<br />
* C/C++<br />
* Go<br />
'Please note that compiled languages will need special consideration and handling in order to get the packages to build properly. Packages are built for AMD64 and i386 for now. We have the capability to also compile them for OpenPower, ARM and ARM64, but that will require some porting work.<br />
<br />
In general, we advise the following for test scripts:<br />
* Python 3 for pretty much everything<br />
* BASH for one-liners and short simple scripts. Generally, if it starts feeling complex, it's time to re-write in Python3<br />
* If you MUST use compiled languages, C but this is exceedingly rare and should only be done when something easier to maintain is not appropriate<br />
<br />
==== Translations ====<br />
We currently do not have any Translations for the tooling, nor are there plans to enable this.<br />
<br />
==== Style Guide ====<br />
The general rule is Python code should adhere to [https://www.python.org/dev/peps/pep-0008/ Python Foundation's PEP-8]. There are several tools to help you check your code for adherence such as [https://pypi.python.org/pypi/pep8/1.6.2 pep8] and [http://www.pylint.org/ Pylint] and many others.<br />
<br />
== Project Hierarchy ==<br />
=== opencompute-certification ===<br />
==== What this provides ====<br />
This is the trunk for the Open Compute Project's certification tooling. It can be found on Launchpad:<br />
https://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
This code is used to build the launcher packages:<br />
* opencompute-ready<br />
* opencompute-certification<br />
<br />
and the provider package:<br />
* plainbox-provider-opencompute-certification<br />
<br />
==== What's a Provider? ====<br />
A Provider is a package that provides the bits necessary for Plainbox/Checkbox-NG to perform testing. These files include job descriptions, whitelists, test scripts and wrappers, helpers, custom libraries and anything else that you may need.<br />
<br />
==== What's a Launcher? ====<br />
A Launcher is a package that does two things. <br />
# It provides the actual program launcher (the command you issue to start the testing) such as <code>opencompute-certification</code><br />
# It ensures the lower-level packages and dependencies are installed<br />
=== checkbox ===<br />
==== What this provides ====<br />
This is the main project for Canonical's certification tool. This provides the back-end and underlying support structure as well as some of the general test scripts and jobs. The following packages make up our use of Checkbox, in order from top to bottom.<br />
<br />
===== plainbox-provider-checkbox =====<br />
This package provides the actual checkbox test tools used by Canonical Certification. Many tests in OCP Certification and OCP Ready are based on the jobs, scripts and data in this provider.<br />
<br />
===== plainbox-provider-resource-generic =====<br />
This package provides system resource jobs and libraries. These are used at run-time to gather system information and provide that data to CheckboxNG for job generation, to the results files for system hardware inventory, and to provide the validation that allows certain jobs to run.<br />
<br />
To explain that last statement, one job of this is to determine if certain hardware exists, and if that hardware exists, then the jobs that test that hardware are configured and executed. For example, there is a USB resource that determines if a given motherboard has a USB controller. If the USB Controller is detected by the USB Resource, then the USB test jobs will be run. If the USB Resource does NOT detect a USB controller, then the USB jobs will be skipped.<br />
<br />
===== python3-checkbox-support =====<br />
This package is an infrastructure package that provides several python3 support libraries that Checkbox needs to run.<br />
<br />
===== checkbox-ng =====<br />
This package is the back-end. It provides Checkbox-NG, the Plainbox based test harness that replaces the old Checkbox tool. It provides the internals that organize and run all the various test cases, performs all the parsing and heavy work of Certification Testing.<br />
<br />
==== Documentation? ====<br />
For the most part, Documentation for the OCP Tools can be found here on the OCP Certification Wiki. For Checkbox, documentation can be found:<br />
* http://checkbox.readthedocs.org/en/latest/<br />
* http://plainbox.readthedocs.org/en/latest/<br />
<br />
As CheckboxNG and Plainbox are under active development, this documentation is constantly evolving and improving.<br />
<br />
==== How can I contribute to this? ====<br />
There are several ways to contribute. For the most part, you'll likely want to contribute to test scripts, jobs and whitelist changes, so you will mostly be interested in the '''opencompute-testing''' project.<br />
<br />
To get started, follow the bits above to branch the project (lp:opencompute/opencompute-testing) and start hacking!<br />
<br />
If you'd like to contribute to the Checkbox project, please see the [[#Questions?|Questions?]] section below.<br />
<br />
=== Putting it all together ===<br />
So now that you know about the various packages that make up the OCP Certification and OCP Ready test tools, let's install them so you can try them out to see what they do.<br />
==== Installation of Packages ====<br />
First, we need to install the PPAs. Because this is based on Canonical's actual certification tooling instead of a fork, we need to add two PPAs:<br />
<br />
<pre><br />
sudo apt-add-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
sudo apt-add-repository ppa:hardware-certification/public<br />
</pre><br />
<br />
Next, we need to update the apt cache:<br />
<br />
<pre>sudo apt update</pre><br />
<br />
And finally, install the packages:<br />
<br />
<pre>sudo apt-get install opencompute-certification</pre><br />
<br />
==== Interaction ====<br />
After you have done the above, you will see a long list of packages that need to be installed. These all provide testing tools, libraries and other things necessary for Certification testing. The number of packages will vary depending on how you installed the OS onto the SUT, what version is being used, what tool is used (Ready vs Certification) and so forth. <br />
<br />
Hit Y and the installation will begin.<br />
<br />
Once installation is complete, to check it out qickly, simply run the command <code>opencompute-certification</code> or <code>opencompute-ready</code><br />
<br />
To do proper testing, there is some configuration and other things that need to be performed first, but this will get you an idea of how the tool works and what it looks like.<br />
<br />
== Obtaining the Code ==<br />
=== Locations ===<br />
For now, there are multiple locations to find testing tools. In the near future, these will be merged into the Launchpad branches so that all code is easily found and maintained.<br />
==== Launchpad ====<br />
The main project code is housed in the following projects on Launchpad:<br />
* [https://code.launchpad.net/opencompute/opencompute-testing lp:opencompute/opencompute-testing]<br />
* [https://code.launchpad.net/checkbox lp:checkbox]<br />
==== GitHub ====<br />
ITRI has placed the majority of the Certification test scripts, job files and such on GitHub:<br />
* https://github.com/opencomputeproject/ocp_ci_server_certification_test_cases<br />
=== Pulling the code ===<br />
See above about using '''bzr''' to obtain local branches of the Checkbox or OpenCompute Testing trunks.<br />
<br />
For the ITRI github location, you'll need to use git.<br />
<br />
== Adding new code ==<br />
The following is meant to help you understand where to add things in the code.<br />
=== opencompute-certification ===<br />
==== Trunk Contents ====<br />
The trunk contains the following directories:<br />
;bin<br />
:This directory contains all the test scripts that are not part of the lp:checkbox branch. These are the OCP specific test scripts.<br />
;COPYING<br />
:This is a copy of the license. All code is Open Source under the GPLv3. Proprietary code is not allowed in the certification suite.<br />
;data<br />
:This directory is used in case sample test data needs to be installed along with the test suite.<br />
;debian<br />
:This directory contains all the necessary Debian packaging information. It also contains the Changelog. Likely, you will only ever need to edit the changelog in this directory.<br />
;examples<br />
:This directory contains sample config files that are installed when a given package is installed.<br />
;jobs<br />
:This directory contains the job description files. Jobs are the units of Test that are performed.<br />
;launcher<br />
:This directory contains the launcher file that creates the command used to execute the test tool on the SUT<br />
;manage.py<br />
:This is a helper program for testing and building packages.<br />
;po<br />
:UNUSED CURRENTLY, but this would house translation data if implemented.<br />
;README.development<br />
:A README file that gives you basic development information. Please read this file.<br />
;README.rst <br />
:A description of the trunk contents (the same thing you're reading right now)<br />
;requirements <br />
:Package building requirements.<br />
;whitelists<br />
:The whitelists that define each test suite.<br />
<br />
==== Job Files ====<br />
As discussed before, job files contain the descriptions that define each test case or unit of test. <br />
===== Basic Job Description =====<br />
An example job description looks like this:<br />
<br />
<pre><br />
plugin: shell<br />
name: memory/ocp/stress_30min<br />
user: root<br />
command: stressapptest -s 1800<br />
_description:<br />
Test and exercise memory.<br />
</pre><br />
<br />
This is a very basic example that contains most of the common KEY:VALUE pairs that are used to describe a test.<br />
;plugin<br />
:Plugins are the type of job to perform. <br />
:'''shell''' is a fully automated job that is executed by the test harness automatically with no user intervention. <br />
:'''manual''' is a test/job that requires the user to perform some task and verify the results.<br />
:'''user-interact''' which also requires the user to initiate the test with the PASS or FAIL results being automatically determined by Checkbox<br />
:'''user-interact-verify''' requires the user to initiate the test AND validate the results, choosing PASS or FAIL afterwards.<br />
;name<br />
:This is a descriptive name of the test. In this case, the hierarchy means that it's a Memory test that is part of the OCP Specific tests and the test case is 'stress_30min'<br />
;user <br />
:Some jobs can be run as an alternate user (in fact, some require this). So specifying '''root''' here will tell Checkbox to run the '''command''' as the root user (essentially using sudo to elevate privileges for this one test only).<br />
;command<br />
:This is the command to perform. This item can be in-line code (like a bash oneliner or test) or the name of a command that is in the bin/ directory.<br />
;_description<br />
:This is a text description of the job. For a Manual job, '''description''' would list the steps necessary to perform the test. For an automated test, this is just a summary of what the test does.<br />
<br />
===== Meta-Jobs =====<br />
There are also things we refer to as "Metajobs". These jobs are actually run early on and are simply used to generate jobs on the fly. In general, you would use a metajob to create separate jobs for multiple pieces of the same hardware. In the following example, we have a metajob that creates a separate storage test for any disk device (NON-USB) discovered.<br />
<br />
<pre><br />
plugin: local<br />
name: disk/ocp/io_stress<br />
requires:<br />
device.category == 'DISK'<br />
package.name == 'stressapptest'<br />
_description: Verify that storage devices, such as Fibre Channel and RAID can be detected and perform under stress.<br />
command:<br />
cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"'<br />
plugin: shell<br />
name: disk/ocp/io_stress_`ls /sys$path/block`<br />
user: root<br />
requires:<br />
device.path == "$path"<br />
block_device.`ls /sys$path/block`_state != 'removable'<br />
description: Disk I/O stress test for $product<br />
command: disk_stress `ls /sys$path/block | sed 's|!|/|'`<br />
EOF<br />
</pre><br />
<br />
Note the differences between a Metajob and a normal Job.<br />
;Plugin<br />
:'''local''' is a special plugin that tells Checkbox to run this job as soon as the tool starts. This means this job is executed well before the user ever sees a Suite or Test Selection screen.<br />
;requires<br />
:This job also introduces the '''requires''' directive. This means that the following conditions must be met before the job will be run. <br />
:Also note that you can specify multiple requirements by placing each one on a new line with a single-space indention.<br />
;Command<br />
:In the basic example, I mentioned that a '''Command''' can be embedded shell code. This is an example of that embedded shell code. This runs the shell code that follows. In this example, it cats all the text between the 'cat' line and the 'EOF' marker (which looks exactly like the Basic Job Description) and pipes all that through the 'run-templates' helper tool which creates on-the-fly jobs. In this case, 'run-templates' also executes the udev-resource resource gathering script and ensures that child jobs are ONLY created for hardware devices that fit into the DISK category (so we don't create these Disk jobs for network devices or CPUs).<br />
<br />
==== Scripts ====<br />
Scritps are found in the bin/ directory. These can be anything from bash to python to ruby to anything you feel like coding in. Scripts can be actual tests themselves, or wrappers that call other tools or test suites. In the past, we have had wrapper scripts that ran things like Phoronix Test Suite and Piglit (for desktop testing).<br />
<br />
==== Whitelists ====<br />
Whitelists are text lists that define the test suite. This is a list of tests to run. Below is a sample whitelist:<br />
<pre><br />
# Resource Jobs<br />
miscellanea/submission-resources<br />
#Info attachment jobs<br />
__info__<br />
cpuinfo_attachment<br />
dmesg_attachment<br />
dmi_attachment<br />
dmidecode_attachment<br />
efi_attachment<br />
lspci_attachment<br />
lshw_attachment<br />
mcelog_attachment<br />
meminfo_attachment<br />
modprobe_attachment<br />
modules_attachment<br />
sysctl_attachment<br />
sysfs_attachment<br />
udev_attachment<br />
lsmod_attachment<br />
acpi_sleep_attachment<br />
info/hdparm<br />
info/hdparm_.*.txt<br />
installer_debug.gz<br />
info/disk_partitions<br />
# Actual test cases<br />
__TC-003__<br />
TC-003-0003-001/Network_performance<br />
TC-003-0003-001/Network_performance_.*<br />
</pre><br />
<br />
The first job, '''miscellanea/submission-resources''' is always required in a whitelist. It is what triggers all the various resource jobs to run as well as jobs necessary to create a full submission file. After that, add whatever jobs you need. The first group in the example above are jobs that attach files containing the output of various commands. <br />
<br />
In the next section is an example of two things. The job '''TC-003-0003-001/Network_performance''' is a "metajob" that generates other jobs. IN this case, it's a job that creates a separate network test for each NIC discovered on the SUT. The next job, '''TC-003-0003-001/Network_performance_.*''' is an example of using a regular expression. In this case, this line item tells Checkbox to run any job that starts with the text 'TC-003-0003-001/Network_performance_' (which when run would be jobs like 'TC-003-0003-001/Network_performance_eth0' and 'TC-003-0003-001/Network_performance_eth1'.<br />
<br />
== Submitting Code to the Project ==<br />
The following will cover how to submit code once you've made changes to your branch. The examples here assume you have already performed the following steps:<br />
#Branched lp:opencompute/opencompute-testing.<br />
#Made changes or additions<br />
#Tested your changes or additions<br />
#Committed your changes or additions<br />
<br />
=== Launchpad ===<br />
As noted early on, you will need to be logged in via bzr to submit code. This is accomplished using the '''bzr launchpad-login''' command:<br />
<pre><br />
bzr launchpad-login <YOUR_LAUNCHPAD_USERNAME><br />
</pre><br />
<br />
You will likely also need to have created and submitted an SSH key to Launchpad to use for submissions. This is explained here:<br />
https://help.launchpad.net/YourAccount/CreatingAnSSHKeyPair<br />
<br />
=== Pushing Changes ===<br />
Once you have committed your changes, you need to push them to YOUR OWN branch on Launchpad. To do so, cd into your branch and do the following:<br />
<pre><br />
bzr push lp:~<YOUR LP USERNAME>/opencompute/<YOUR BRANCH NAME><br />
</pre><br />
<br />
Example:<br />
<pre><br />
bzr push lp:~bladernr/opencompute/ocp-cert-cleanup<br />
</pre><br />
<br />
=== Merge Requests ===<br />
Once you have pushed your branch to Launchpad, you need to make a merge request. To do so you need to do the following things:<br />
# In a web browser, go to http://code.launchpad.net/~YOUR_LAUNCHPAD_USERNAME<br />
# Click on the name of the branch you just pushed<br />
# Under '''Branch Merges''' click on '''Propose for merging'''<br />
# Under '''Target Branch''' select '''Other''' and enter '~opencompute-developers/opencompute/opencompute-testing'<br />
# In the Description text box, enter a summary of the changes you made<br />
# Click the '''Propose Merge''' button at the bottom of that page<br />
<br />
This is submit your merge request. At this point, it will be reviewed, commented on, approved or sent back for fixing.<br />
<br />
== Questions? ==<br />
=== OCP Ready or OCP Certification===<br />
* Certification Project Team Mailing List: http://lists.opencompute.org/mailman/listinfo/opencompute-ci<br />
* [http://freenode.net Freenode IRC Servers] #ocp-ci<br />
<br />
=== CheckboxNG or Plainbox ===<br />
* [http://freenode.net Freenode IRC Servers] #checkbox<br />
* http://launchpad.net/checkbox<br />
<br />
=== Further Reading, Resources, Etc ===<br />
* [https://help.launchpad.net/ Launchpad Help]<br />
* [http://wiki.bazaar.canonical.com/BzrSupport Bazaar (bzr) Help]<br />
* [http://doc.bazaar.canonical.com/en/ Bazaar (bzr) Documentation]</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/ToolsDevelopmentGuideline&diff=1062C&I Wiki Portal/ToolsDevelopmentGuideline2015-04-02T20:25:45Z<p>Jlane: /* Installation of Packages */</p>
<hr />
<div>== Overview ==<br />
The official repository for the source code to OCP Certification and OCP Ready is [http://launchpad.net/opencompute Launchpad]. There are a number of projects hosted there but the most important one is [http://code.launchpad.net/opencompute-testing opencompute-testing].<br />
<br />
The '''opencompute-testing''' project contains all the code for generating the OCP Certification and OCP Ready testing tools. It is the code that creates the Plainbox Provider and Launcher Packages, hosting the job descriptions that are OCP Specific, scripts and whitelists. Package builds occur automatically once per day if the code base has been updated.<br />
<br />
=== Contacts ===<br />
If you have ANY questions at all about the code base, contributing, or anything else related, contact [mailto:jeff@canonical.com Jeff Lane]<br />
For anything related to Open Compute Certification (including code questions):<br />
* Mailing List: http://lists.opencompute.org/mailman/listinfo/opencompute-ci<br />
* IRC: [http://freenode.net Freenode IRC Servers] #ocp-ci<br />
<br />
== Basics ==<br />
This section will cover some of the basics for contributing to the code for these tools. This is a general overview, more specific information can be found on line at the links below.<br />
<br />
=== Bazaar ===<br />
==== About Bazaar ====<br />
Bazaar is a version control system that helps you track project history over time and to collaborate easily with others. Whether you're a single developer, a co-located team or a community of developers scattered across the world, Bazaar scales and adapts to meet your needs. Part of the GNU Project, Bazaar is free software sponsored by Canonical<br />
<br />
You can learn more at http://bazaar.canonical.com/en/<br />
<br />
==== Obtaining Bazaar ====<br />
To install Bazaar on a debian or ubuntu system:<br />
<code>sudo apt-get install bazaar</code><br />
<br />
For other Operating Systems, including Windows, Red Hat, SuSE, OSX and more, please see http://wiki.bazaar.canonical.com/Download<br />
<br />
==== Learning Bazaar ====<br />
Bazaar is easy to learn. There is an excellent [http://doc.bazaar.canonical.com/latest/en/mini-tutorial/ 5 Minute Tutorial] that should get you started. Beyond that, bzr has a fairly well developed man page and in-program help system. <br />
<br />
Please read and do the 5 Minute Tutorial now, which will teach you the basics like creating branches, logging into LP, pushing and merging.<br />
<br />
===== Trees, Branches, Trunks? =====<br />
The basic top level of a codebase is called the Trunk. The Trunk is the mainline code that all branches are a part of. Branches are localized copies of Trunk. Branches are where you make changes for submission and merge consideration. Once accepted, Branches are merged into Trunk and become part of the mainline codebase. <br />
<br />
===== Basic Commands =====<br />
* The most basic command is: <code>bzr</code><br />
* To get a local copy of a project: <code>bzr branch lp:opencompute/opencompute-testing</code><br />
* To review a change summary: <code>bzr status</code><br />
* To add new files that are not yet managed: <code>bzr add path/to/filename</code><br />
** Note that /path/to/filename needs to be a path INSIDE your branch. When adding files not in the branch, you need to copy those files into the branch before adding them.<br />
* To see a diff of changes: <code>bzr diff path/to/filename</code><br />
* To commit your changes to a new revision <code>bzr commit -m "Some descriptive summary of your changes"</code><br />
* To push your changes to launchpad in preparation for a Merge Request: <code>bzr push lp:~<Your Launchpad Username>/opencompute/<Your Branch Name></code><br />
** for example: <code>bzr push lp:~bladernr/opencompute/ocp-testing-job-updates</code><br />
<br />
There are many MANY more commands but those are the ones you'll likely use most frequently.<br />
<br />
===== Learning More =====<br />
* http://doc.bazaar.canonical.com/en/<br />
* http://wiki.bazaar.canonical.com/Tutorials<br />
* https://yade-dem.org/wiki/Quick_Bazaar_tutorial<br />
* http://wiki.bazaar.canonical.com/Tutorials<br />
<br />
=== Practices ===<br />
==== Languages to use ====<br />
===== Main Tooling =====<br />
Plainbox, Checkbox, the providers and such are all written in Python3.<br />
<br />
===== Job Files and Whitelists =====<br />
Job Files are the same as they were in Checkbox, that is RFC822 compliant text files. See any file in the <code>jobs/</code> directory for examples.<br />
Whitelists are also simply text files, HOWEVER, you can also use regular expressions in job naming to pattern match jobs.<br />
<br />
===== Test Scripts =====<br />
These are scripts that perform actual testing. For example, a storage IO test, or a network performance test. Scripts can be written in pretty much anything that is compiled or has a valid interpreter installed on the System Under Test. The following are just a few examples of the languages you can write test scripts in:<br />
* Python3<br />
* Bash<br />
* Ruby<br />
* Perl<br />
<br />
And some more complex ones<br />
* C/C++<br />
* Go<br />
'Please note that compiled languages will need special consideration and handling in order to get the packages to build properly. Packages are built for AMD64 and i386 for now. We have the capability to also compile them for OpenPower, ARM and ARM64, but that will require some porting work.<br />
<br />
In general, we advise the following for test scripts:<br />
* Python 3 for pretty much everything<br />
* BASH for one-liners and short simple scripts. Generally, if it starts feeling complex, it's time to re-write in Python3<br />
* If you MUST use compiled languages, C but this is exceedingly rare and should only be done when something easier to maintain is not appropriate<br />
<br />
==== Translations ====<br />
We currently do not have any Translations for the tooling, nor are there plans to enable this.<br />
<br />
==== Style Guide ====<br />
The general rule is Python code should adhere to [https://www.python.org/dev/peps/pep-0008/ Python Foundation's PEP-8]. There are several tools to help you check your code for adherence such as [https://pypi.python.org/pypi/pep8/1.6.2 pep8] and [http://www.pylint.org/ Pylint] and many others.<br />
<br />
== Project Hierarchy ==<br />
=== opencompute-certification ===<br />
==== What this provides ====<br />
This is the trunk for the Open Compute Project's certification tooling. It can be found on Launchpad:<br />
https://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
This code is used to build the launcher packages:<br />
* opencompute-ready<br />
* opencompute-certification<br />
<br />
and the provider package:<br />
* plainbox-provider-opencompute-certification<br />
<br />
==== What's a Provider? ====<br />
A Provider is a package that provides the bits necessary for Plainbox/Checkbox-NG to perform testing. These files include job descriptions, whitelists, test scripts and wrappers, helpers, custom libraries and anything else that you may need.<br />
<br />
==== What's a Launcher? ====<br />
A Launcher is a package that does two things. <br />
# It provides the actual program launcher (the command you issue to start the testing) such as <code>opencompute-certification</code><br />
# It ensures the lower-level packages and dependencies are installed<br />
=== checkbox ===<br />
==== What this provides ====<br />
This is the main project for Canonical's certification tool. This provides the back-end and underlying support structure as well as some of the general test scripts and jobs. The following packages make up our use of Checkbox, in order from top to bottom.<br />
<br />
===== plainbox-provider-checkbox =====<br />
This package provides the actual checkbox test tools used by Canonical Certification. Many tests in OCP Certification and OCP Ready are based on the jobs, scripts and data in this provider.<br />
<br />
===== plainbox-provider-resource-generic =====<br />
This package provides system resource jobs and libraries. These are used at run-time to gather system information and provide that data to CheckboxNG for job generation, to the results files for system hardware inventory, and to provide the validation that allows certain jobs to run.<br />
<br />
To explain that last statement, one job of this is to determine if certain hardware exists, and if that hardware exists, then the jobs that test that hardware are configured and executed. For example, there is a USB resource that determines if a given motherboard has a USB controller. If the USB Controller is detected by the USB Resource, then the USB test jobs will be run. If the USB Resource does NOT detect a USB controller, then the USB jobs will be skipped.<br />
<br />
===== python3-checkbox-support =====<br />
This package is an infrastructure package that provides several python3 support libraries that Checkbox needs to run.<br />
<br />
===== checkbox-ng =====<br />
This package is the back-end. It provides Checkbox-NG, the Plainbox based test harness that replaces the old Checkbox tool. It provides the internals that organize and run all the various test cases, performs all the parsing and heavy work of Certification Testing.<br />
<br />
==== Documentation? ====<br />
For the most part, Documentation for the OCP Tools can be found here on the OCP Certification Wiki. For Checkbox, documentation can be found:<br />
* http://checkbox.readthedocs.org/en/latest/<br />
* http://plainbox.readthedocs.org/en/latest/<br />
<br />
As CheckboxNG and Plainbox are under active development, this documentation is constantly evolving and improving.<br />
<br />
==== How can I contribute to this? ====<br />
There are several ways to contribute. For the most part, you'll likely want to contribute to test scripts, jobs and whitelist changes, so you will mostly be interested in the '''opencompute-testing''' project.<br />
<br />
To get started, follow the bits above to branch the project (lp:opencompute/opencompute-testing) and start hacking!<br />
<br />
If you'd like to contribute to the Checkbox project, please see the [[#Questions?|Questions?]] section below.<br />
<br />
=== Putting it all together ===<br />
So now that you know about the various packages that make up the OCP Certification and OCP Ready test tools, let's install them so you can try them out to see what they do.<br />
==== Installation of Packages ====<br />
First, we need to install the PPAs. Because this is based on Canonical's actual certification tooling instead of a fork, we need to add two PPAs:<br />
<br />
<pre><br />
sudo apt-add-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
sudo apt-add-repository ppa:hardware-certification/public<br />
</pre><br />
<br />
Next, we need to update the apt cache:<br />
<br />
<pre>sudo apt update</pre><br />
<br />
And finally, install the packages:<br />
<br />
<pre>sudo apt-get install opencompute-certification</pre><br />
<br />
==== Interaction ====<br />
After you have done the above, you will see a long list of packages that need to be installed. These all provide testing tools, libraries and other things necessary for Certification testing. The number of packages will vary depending on how you installed the OS onto the SUT, what version is being used, what tool is used (Ready vs Certification) and so forth. <br />
<br />
Hit Y and the installation will begin.<br />
<br />
Once installation is complete, to check it out qickly, simply run the command <code>opencompute-certification</code> or <code>opencompute-ready</code><br />
<br />
To do proper testing, there is some configuration and other things that need to be performed first, but this will get you an idea of how the tool works and what it looks like.<br />
<br />
== Obtaining the Code ==<br />
=== Locations ===<br />
For now, there are multiple locations to find testing tools. In the near future, these will be merged into the Launchpad branches so that all code is easily found and maintained.<br />
==== Launchpad ====<br />
The main project code is housed in the following projects on Launchpad:<br />
* [https://code.launchpad.net/opencompute/opencompute-testing lp:opencompute/opencompute-testing]<br />
* [https://code.launchpad.net/checkbox lp:checkbox]<br />
==== GitHub ====<br />
ITRI has placed the majority of the Certification test scripts, job files and such on GitHub:<br />
* https://github.com/opencomputeproject/ocp_ci_server_certification_test_cases<br />
=== Pulling the code ===<br />
See above about using '''bzr''' to obtain local branches of the Checkbox or OpenCompute Testing trunks.<br />
<br />
For the ITRI github location, you'll need to use git.<br />
<br />
== Adding new code ==<br />
The following is meant to help you understand where to add things in the code.<br />
=== opencompute-certification ===<br />
==== Trunk Contents ====<br />
The trunk contains the following directories:<br />
;bin<br />
:This directory contains all the test scripts that are not part of the lp:checkbox branch. These are the OCP specific test scripts.<br />
;COPYING<br />
:This is a copy of the license. All code is Open Source under the GPLv3. Proprietary code is not allowed in the certification suite.<br />
;data<br />
:This directory is used in case sample test data needs to be installed along with the test suite.<br />
;debian<br />
:This directory contains all the necessary Debian packaging information. It also contains the Changelog. Likely, you will only ever need to edit the changelog in this directory.<br />
;examples<br />
:This directory contains sample config files that are installed when a given package is installed.<br />
;jobs<br />
:This directory contains the job description files. Jobs are the units of Test that are performed.<br />
;launcher<br />
:This directory contains the launcher file that creates the command used to execute the test tool on the SUT<br />
;manage.py<br />
:This is a helper program for testing and building packages.<br />
;po<br />
:UNUSED CURRENTLY, but this would house translation data if implemented.<br />
;README.development<br />
:A README file that gives you basic development information. Please read this file.<br />
;README.rst <br />
:A description of the trunk contents (the same thing you're reading right now)<br />
;requirements <br />
:Package building requirements.<br />
;whitelists<br />
:The whitelists that define each test suite.<br />
<br />
==== Job Files ====<br />
As discussed before, job files contain the descriptions that define each test case or unit of test. <br />
===== Basic Job Description =====<br />
An example job description looks like this:<br />
<br />
<pre><br />
plugin: shell<br />
name: memory/ocp/stress_30min<br />
user: root<br />
command: stressapptest -s 1800<br />
_description:<br />
Test and exercise memory.<br />
</pre><br />
<br />
This is a very basic example that contains most of the common KEY:VALUE pairs that are used to describe a test.<br />
;plugin<br />
:Plugins are the type of job to perform. <br />
:'''shell''' is a fully automated job that is executed by the test harness automatically with no user intervention. <br />
:'''manual''' is a test/job that requires the user to perform some task and verify the results.<br />
:'''user-interact''' which also requires the user to initiate the test with the PASS or FAIL results being automatically determined by Checkbox<br />
:'''user-interact-verify''' requires the user to initiate the test AND validate the results, choosing PASS or FAIL afterwards.<br />
;name<br />
:This is a descriptive name of the test. In this case, the hierarchy means that it's a Memory test that is part of the OCP Specific tests and the test case is 'stress_30min'<br />
;user <br />
:Some jobs can be run as an alternate user (in fact, some require this). So specifying '''root''' here will tell Checkbox to run the '''command''' as the root user (essentially using sudo to elevate privileges for this one test only).<br />
;command<br />
:This is the command to perform. This item can be in-line code (like a bash oneliner or test) or the name of a command that is in the bin/ directory.<br />
;_description<br />
:This is a text description of the job. For a Manual job, '''description''' would list the steps necessary to perform the test. For an automated test, this is just a summary of what the test does.<br />
<br />
===== Meta-Jobs =====<br />
There are also things we refer to as "Metajobs". These jobs are actually run early on and are simply used to generate jobs on the fly. In general, you would use a metajob to create separate jobs for multiple pieces of the same hardware. In the following example, we have a metajob that creates a separate storage test for any disk device (NON-USB) discovered.<br />
<br />
<pre><br />
plugin: local<br />
name: disk/ocp/io_stress<br />
requires:<br />
device.category == 'DISK'<br />
package.name == 'stressapptest'<br />
_description: Verify that storage devices, such as Fibre Channel and RAID can be detected and perform under stress.<br />
command:<br />
cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"'<br />
plugin: shell<br />
name: disk/ocp/io_stress_`ls /sys$path/block`<br />
user: root<br />
requires:<br />
device.path == "$path"<br />
block_device.`ls /sys$path/block`_state != 'removable'<br />
description: Disk I/O stress test for $product<br />
command: disk_stress `ls /sys$path/block | sed 's|!|/|'`<br />
EOF<br />
</pre><br />
<br />
Note the differences between a Metajob and a normal Job.<br />
;Plugin<br />
:'''local''' is a special plugin that tells Checkbox to run this job as soon as the tool starts. This means this job is executed well before the user ever sees a Suite or Test Selection screen.<br />
;requires<br />
:This job also introduces the '''requires''' directive. This means that the following conditions must be met before the job will be run. <br />
:Also note that you can specify multiple requirements by placing each one on a new line with a single-space indention.<br />
;Command<br />
:In the basic example, I mentioned that a '''Command''' can be embedded shell code. This is an example of that embedded shell code. This runs the shell code that follows. In this example, it cats all the text between the 'cat' line and the 'EOF' marker (which looks exactly like the Basic Job Description) and pipes all that through the 'run-templates' helper tool which creates on-the-fly jobs. In this case, 'run-templates' also executes the udev-resource resource gathering script and ensures that child jobs are ONLY created for hardware devices that fit into the DISK category (so we don't create these Disk jobs for network devices or CPUs).<br />
<br />
==== Scripts ====<br />
Scritps are found in the bin/ directory. These can be anything from bash to python to ruby to anything you feel like coding in. Scripts can be actual tests themselves, or wrappers that call other tools or test suites. In the past, we have had wrapper scripts that ran things like Phoronix Test Suite and Piglit (for desktop testing).<br />
<br />
==== Whitelists ====<br />
Whitelists are text lists that define the test suite. This is a list of tests to run. Below is a sample whitelist:<br />
<pre><br />
# Resource Jobs<br />
miscellanea/submission-resources<br />
#Info attachment jobs<br />
__info__<br />
cpuinfo_attachment<br />
dmesg_attachment<br />
dmi_attachment<br />
dmidecode_attachment<br />
efi_attachment<br />
lspci_attachment<br />
lshw_attachment<br />
mcelog_attachment<br />
meminfo_attachment<br />
modprobe_attachment<br />
modules_attachment<br />
sysctl_attachment<br />
sysfs_attachment<br />
udev_attachment<br />
lsmod_attachment<br />
acpi_sleep_attachment<br />
info/hdparm<br />
info/hdparm_.*.txt<br />
installer_debug.gz<br />
info/disk_partitions<br />
# Actual test cases<br />
__TC-003__<br />
TC-003-0003-001/Network_performance<br />
TC-003-0003-001/Network_performance_.*<br />
</pre><br />
<br />
The first job, '''miscellanea/submission-resources''' is always required in a whitelist. It is what triggers all the various resource jobs to run as well as jobs necessary to create a full submission file. After that, add whatever jobs you need. The first group in the example above are jobs that attach files containing the output of various commands. <br />
<br />
In the next section is an example of two things. The job '''TC-003-0003-001/Network_performance''' is a "metajob" that generates other jobs. IN this case, it's a job that creates a separate network test for each NIC discovered on the SUT. The next job, '''TC-003-0003-001/Network_performance_.*''' is an example of using a regular expression. In this case, this line item tells Checkbox to run any job that starts with the text 'TC-003-0003-001/Network_performance_' (which when run would be jobs like 'TC-003-0003-001/Network_performance_eth0' and 'TC-003-0003-001/Network_performance_eth1'.<br />
<br />
== Submitting Code to the Project ==<br />
The following will cover how to submit code once you've made changes to your branch. The examples here assume you have already performed the following steps:<br />
#Branched lp:opencompute/opencompute-testing.<br />
#Made changes or additions<br />
#Tested your changes or additions<br />
#Committed your changes or additions<br />
<br />
=== Launchpad ===<br />
As noted early on, you will need to be logged in via bzr to submit code. This is accomplished using the '''bzr launchpad-login''' command:<br />
<pre><br />
bzr launchpad-login <YOUR_LAUNCHPAD_USERNAME><br />
</pre><br />
<br />
You will likely also need to have created and submitted an SSH key to Launchpad to use for submissions. This is explained here:<br />
https://help.launchpad.net/YourAccount/CreatingAnSSHKeyPair<br />
<br />
=== Pushing Changes ===<br />
Once you have committed your changes, you need to push them to YOUR OWN branch on Launchpad. To do so, cd into your branch and do the following:<br />
<pre><br />
bzr push lp:~<YOUR LP USERNAME>/opencompute/<YOUR BRANCH NAME><br />
</pre><br />
<br />
Example:<br />
<pre><br />
bzr push lp:~bladernr/opencompute/ocp-cert-cleanup<br />
</pre><br />
<br />
=== Merge Requests ===<br />
Once you have pushed your branch to Launchpad, you need to make a merge request. To do so you need to do the following things:<br />
# In a web browser, go to http://code.launchpad.net/~YOUR_LAUNCHPAD_USERNAME<br />
# Click on the name of the branch you just pushed<br />
# Under '''Branch Merges''' click on '''Propose for merging'''<br />
# Under '''Target Branch''' select '''Other''' and enter '~opencompute-developers/opencompute/opencompute-testing'<br />
# In the Description text box, enter a summary of the changes you made<br />
# Click the '''Propose Merge''' button at the bottom of that page<br />
<br />
This is submit your merge request. At this point, it will be reviewed, commented on, approved or sent back for fixing.<br />
<br />
== Questions? ==<br />
=== OCP Ready or OCP Certification===<br />
=== CheckboxNG or Plainbox ===</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/ToolsDevelopmentGuideline&diff=1061C&I Wiki Portal/ToolsDevelopmentGuideline2015-04-02T20:25:02Z<p>Jlane: /* Submitting Code to the Project */</p>
<hr />
<div>== Overview ==<br />
The official repository for the source code to OCP Certification and OCP Ready is [http://launchpad.net/opencompute Launchpad]. There are a number of projects hosted there but the most important one is [http://code.launchpad.net/opencompute-testing opencompute-testing].<br />
<br />
The '''opencompute-testing''' project contains all the code for generating the OCP Certification and OCP Ready testing tools. It is the code that creates the Plainbox Provider and Launcher Packages, hosting the job descriptions that are OCP Specific, scripts and whitelists. Package builds occur automatically once per day if the code base has been updated.<br />
<br />
=== Contacts ===<br />
If you have ANY questions at all about the code base, contributing, or anything else related, contact [mailto:jeff@canonical.com Jeff Lane]<br />
For anything related to Open Compute Certification (including code questions):<br />
* Mailing List: http://lists.opencompute.org/mailman/listinfo/opencompute-ci<br />
* IRC: [http://freenode.net Freenode IRC Servers] #ocp-ci<br />
<br />
== Basics ==<br />
This section will cover some of the basics for contributing to the code for these tools. This is a general overview, more specific information can be found on line at the links below.<br />
<br />
=== Bazaar ===<br />
==== About Bazaar ====<br />
Bazaar is a version control system that helps you track project history over time and to collaborate easily with others. Whether you're a single developer, a co-located team or a community of developers scattered across the world, Bazaar scales and adapts to meet your needs. Part of the GNU Project, Bazaar is free software sponsored by Canonical<br />
<br />
You can learn more at http://bazaar.canonical.com/en/<br />
<br />
==== Obtaining Bazaar ====<br />
To install Bazaar on a debian or ubuntu system:<br />
<code>sudo apt-get install bazaar</code><br />
<br />
For other Operating Systems, including Windows, Red Hat, SuSE, OSX and more, please see http://wiki.bazaar.canonical.com/Download<br />
<br />
==== Learning Bazaar ====<br />
Bazaar is easy to learn. There is an excellent [http://doc.bazaar.canonical.com/latest/en/mini-tutorial/ 5 Minute Tutorial] that should get you started. Beyond that, bzr has a fairly well developed man page and in-program help system. <br />
<br />
Please read and do the 5 Minute Tutorial now, which will teach you the basics like creating branches, logging into LP, pushing and merging.<br />
<br />
===== Trees, Branches, Trunks? =====<br />
The basic top level of a codebase is called the Trunk. The Trunk is the mainline code that all branches are a part of. Branches are localized copies of Trunk. Branches are where you make changes for submission and merge consideration. Once accepted, Branches are merged into Trunk and become part of the mainline codebase. <br />
<br />
===== Basic Commands =====<br />
* The most basic command is: <code>bzr</code><br />
* To get a local copy of a project: <code>bzr branch lp:opencompute/opencompute-testing</code><br />
* To review a change summary: <code>bzr status</code><br />
* To add new files that are not yet managed: <code>bzr add path/to/filename</code><br />
** Note that /path/to/filename needs to be a path INSIDE your branch. When adding files not in the branch, you need to copy those files into the branch before adding them.<br />
* To see a diff of changes: <code>bzr diff path/to/filename</code><br />
* To commit your changes to a new revision <code>bzr commit -m "Some descriptive summary of your changes"</code><br />
* To push your changes to launchpad in preparation for a Merge Request: <code>bzr push lp:~<Your Launchpad Username>/opencompute/<Your Branch Name></code><br />
** for example: <code>bzr push lp:~bladernr/opencompute/ocp-testing-job-updates</code><br />
<br />
There are many MANY more commands but those are the ones you'll likely use most frequently.<br />
<br />
===== Learning More =====<br />
* http://doc.bazaar.canonical.com/en/<br />
* http://wiki.bazaar.canonical.com/Tutorials<br />
* https://yade-dem.org/wiki/Quick_Bazaar_tutorial<br />
* http://wiki.bazaar.canonical.com/Tutorials<br />
<br />
=== Practices ===<br />
==== Languages to use ====<br />
===== Main Tooling =====<br />
Plainbox, Checkbox, the providers and such are all written in Python3.<br />
<br />
===== Job Files and Whitelists =====<br />
Job Files are the same as they were in Checkbox, that is RFC822 compliant text files. See any file in the <code>jobs/</code> directory for examples.<br />
Whitelists are also simply text files, HOWEVER, you can also use regular expressions in job naming to pattern match jobs.<br />
<br />
===== Test Scripts =====<br />
These are scripts that perform actual testing. For example, a storage IO test, or a network performance test. Scripts can be written in pretty much anything that is compiled or has a valid interpreter installed on the System Under Test. The following are just a few examples of the languages you can write test scripts in:<br />
* Python3<br />
* Bash<br />
* Ruby<br />
* Perl<br />
<br />
And some more complex ones<br />
* C/C++<br />
* Go<br />
'Please note that compiled languages will need special consideration and handling in order to get the packages to build properly. Packages are built for AMD64 and i386 for now. We have the capability to also compile them for OpenPower, ARM and ARM64, but that will require some porting work.<br />
<br />
In general, we advise the following for test scripts:<br />
* Python 3 for pretty much everything<br />
* BASH for one-liners and short simple scripts. Generally, if it starts feeling complex, it's time to re-write in Python3<br />
* If you MUST use compiled languages, C but this is exceedingly rare and should only be done when something easier to maintain is not appropriate<br />
<br />
==== Translations ====<br />
We currently do not have any Translations for the tooling, nor are there plans to enable this.<br />
<br />
==== Style Guide ====<br />
The general rule is Python code should adhere to [https://www.python.org/dev/peps/pep-0008/ Python Foundation's PEP-8]. There are several tools to help you check your code for adherence such as [https://pypi.python.org/pypi/pep8/1.6.2 pep8] and [http://www.pylint.org/ Pylint] and many others.<br />
<br />
== Project Hierarchy ==<br />
=== opencompute-certification ===<br />
==== What this provides ====<br />
This is the trunk for the Open Compute Project's certification tooling. It can be found on Launchpad:<br />
https://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
This code is used to build the launcher packages:<br />
* opencompute-ready<br />
* opencompute-certification<br />
<br />
and the provider package:<br />
* plainbox-provider-opencompute-certification<br />
<br />
==== What's a Provider? ====<br />
A Provider is a package that provides the bits necessary for Plainbox/Checkbox-NG to perform testing. These files include job descriptions, whitelists, test scripts and wrappers, helpers, custom libraries and anything else that you may need.<br />
<br />
==== What's a Launcher? ====<br />
A Launcher is a package that does two things. <br />
# It provides the actual program launcher (the command you issue to start the testing) such as <code>opencompute-certification</code><br />
# It ensures the lower-level packages and dependencies are installed<br />
=== checkbox ===<br />
==== What this provides ====<br />
This is the main project for Canonical's certification tool. This provides the back-end and underlying support structure as well as some of the general test scripts and jobs. The following packages make up our use of Checkbox, in order from top to bottom.<br />
<br />
===== plainbox-provider-checkbox =====<br />
This package provides the actual checkbox test tools used by Canonical Certification. Many tests in OCP Certification and OCP Ready are based on the jobs, scripts and data in this provider.<br />
<br />
===== plainbox-provider-resource-generic =====<br />
This package provides system resource jobs and libraries. These are used at run-time to gather system information and provide that data to CheckboxNG for job generation, to the results files for system hardware inventory, and to provide the validation that allows certain jobs to run.<br />
<br />
To explain that last statement, one job of this is to determine if certain hardware exists, and if that hardware exists, then the jobs that test that hardware are configured and executed. For example, there is a USB resource that determines if a given motherboard has a USB controller. If the USB Controller is detected by the USB Resource, then the USB test jobs will be run. If the USB Resource does NOT detect a USB controller, then the USB jobs will be skipped.<br />
<br />
===== python3-checkbox-support =====<br />
This package is an infrastructure package that provides several python3 support libraries that Checkbox needs to run.<br />
<br />
===== checkbox-ng =====<br />
This package is the back-end. It provides Checkbox-NG, the Plainbox based test harness that replaces the old Checkbox tool. It provides the internals that organize and run all the various test cases, performs all the parsing and heavy work of Certification Testing.<br />
<br />
==== Documentation? ====<br />
For the most part, Documentation for the OCP Tools can be found here on the OCP Certification Wiki. For Checkbox, documentation can be found:<br />
* http://checkbox.readthedocs.org/en/latest/<br />
* http://plainbox.readthedocs.org/en/latest/<br />
<br />
As CheckboxNG and Plainbox are under active development, this documentation is constantly evolving and improving.<br />
<br />
==== How can I contribute to this? ====<br />
There are several ways to contribute. For the most part, you'll likely want to contribute to test scripts, jobs and whitelist changes, so you will mostly be interested in the '''opencompute-testing''' project.<br />
<br />
To get started, follow the bits above to branch the project (lp:opencompute/opencompute-testing) and start hacking!<br />
<br />
If you'd like to contribute to the Checkbox project, please see the [[#Questions?|Questions?]] section below.<br />
<br />
=== Putting it all together ===<br />
So now that you know about the various packages that make up the OCP Certification and OCP Ready test tools, let's install them so you can try them out to see what they do.<br />
==== Installation of Packages ====<br />
First, we need to install the PPAs. Because this is based on Canonical's actual certification tooling instead of a fork, we need to add two PPAs:<br />
<br />
<code><br />
sudo apt-add-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
<br />
sudo apt-add-repository ppa:hardware-certification/public<br />
</code><br />
<br />
Next, we need to update the apt cache:<br />
<br />
<code>sudo apt update</code><br />
<br />
And finally, install the packages:<br />
<br />
<code>sudo apt-get install opencompute-certification</code><br />
<br />
==== Interaction ====<br />
After you have done the above, you will see a long list of packages that need to be installed. These all provide testing tools, libraries and other things necessary for Certification testing. The number of packages will vary depending on how you installed the OS onto the SUT, what version is being used, what tool is used (Ready vs Certification) and so forth. <br />
<br />
Hit Y and the installation will begin.<br />
<br />
Once installation is complete, to check it out qickly, simply run the command <code>opencompute-certification</code> or <code>opencompute-ready</code><br />
<br />
To do proper testing, there is some configuration and other things that need to be performed first, but this will get you an idea of how the tool works and what it looks like.<br />
<br />
== Obtaining the Code ==<br />
=== Locations ===<br />
For now, there are multiple locations to find testing tools. In the near future, these will be merged into the Launchpad branches so that all code is easily found and maintained.<br />
==== Launchpad ====<br />
The main project code is housed in the following projects on Launchpad:<br />
* [https://code.launchpad.net/opencompute/opencompute-testing lp:opencompute/opencompute-testing]<br />
* [https://code.launchpad.net/checkbox lp:checkbox]<br />
==== GitHub ====<br />
ITRI has placed the majority of the Certification test scripts, job files and such on GitHub:<br />
* https://github.com/opencomputeproject/ocp_ci_server_certification_test_cases<br />
=== Pulling the code ===<br />
See above about using '''bzr''' to obtain local branches of the Checkbox or OpenCompute Testing trunks.<br />
<br />
For the ITRI github location, you'll need to use git.<br />
<br />
== Adding new code ==<br />
The following is meant to help you understand where to add things in the code.<br />
=== opencompute-certification ===<br />
==== Trunk Contents ====<br />
The trunk contains the following directories:<br />
;bin<br />
:This directory contains all the test scripts that are not part of the lp:checkbox branch. These are the OCP specific test scripts.<br />
;COPYING<br />
:This is a copy of the license. All code is Open Source under the GPLv3. Proprietary code is not allowed in the certification suite.<br />
;data<br />
:This directory is used in case sample test data needs to be installed along with the test suite.<br />
;debian<br />
:This directory contains all the necessary Debian packaging information. It also contains the Changelog. Likely, you will only ever need to edit the changelog in this directory.<br />
;examples<br />
:This directory contains sample config files that are installed when a given package is installed.<br />
;jobs<br />
:This directory contains the job description files. Jobs are the units of Test that are performed.<br />
;launcher<br />
:This directory contains the launcher file that creates the command used to execute the test tool on the SUT<br />
;manage.py<br />
:This is a helper program for testing and building packages.<br />
;po<br />
:UNUSED CURRENTLY, but this would house translation data if implemented.<br />
;README.development<br />
:A README file that gives you basic development information. Please read this file.<br />
;README.rst <br />
:A description of the trunk contents (the same thing you're reading right now)<br />
;requirements <br />
:Package building requirements.<br />
;whitelists<br />
:The whitelists that define each test suite.<br />
<br />
==== Job Files ====<br />
As discussed before, job files contain the descriptions that define each test case or unit of test. <br />
===== Basic Job Description =====<br />
An example job description looks like this:<br />
<br />
<pre><br />
plugin: shell<br />
name: memory/ocp/stress_30min<br />
user: root<br />
command: stressapptest -s 1800<br />
_description:<br />
Test and exercise memory.<br />
</pre><br />
<br />
This is a very basic example that contains most of the common KEY:VALUE pairs that are used to describe a test.<br />
;plugin<br />
:Plugins are the type of job to perform. <br />
:'''shell''' is a fully automated job that is executed by the test harness automatically with no user intervention. <br />
:'''manual''' is a test/job that requires the user to perform some task and verify the results.<br />
:'''user-interact''' which also requires the user to initiate the test with the PASS or FAIL results being automatically determined by Checkbox<br />
:'''user-interact-verify''' requires the user to initiate the test AND validate the results, choosing PASS or FAIL afterwards.<br />
;name<br />
:This is a descriptive name of the test. In this case, the hierarchy means that it's a Memory test that is part of the OCP Specific tests and the test case is 'stress_30min'<br />
;user <br />
:Some jobs can be run as an alternate user (in fact, some require this). So specifying '''root''' here will tell Checkbox to run the '''command''' as the root user (essentially using sudo to elevate privileges for this one test only).<br />
;command<br />
:This is the command to perform. This item can be in-line code (like a bash oneliner or test) or the name of a command that is in the bin/ directory.<br />
;_description<br />
:This is a text description of the job. For a Manual job, '''description''' would list the steps necessary to perform the test. For an automated test, this is just a summary of what the test does.<br />
<br />
===== Meta-Jobs =====<br />
There are also things we refer to as "Metajobs". These jobs are actually run early on and are simply used to generate jobs on the fly. In general, you would use a metajob to create separate jobs for multiple pieces of the same hardware. In the following example, we have a metajob that creates a separate storage test for any disk device (NON-USB) discovered.<br />
<br />
<pre><br />
plugin: local<br />
name: disk/ocp/io_stress<br />
requires:<br />
device.category == 'DISK'<br />
package.name == 'stressapptest'<br />
_description: Verify that storage devices, such as Fibre Channel and RAID can be detected and perform under stress.<br />
command:<br />
cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"'<br />
plugin: shell<br />
name: disk/ocp/io_stress_`ls /sys$path/block`<br />
user: root<br />
requires:<br />
device.path == "$path"<br />
block_device.`ls /sys$path/block`_state != 'removable'<br />
description: Disk I/O stress test for $product<br />
command: disk_stress `ls /sys$path/block | sed 's|!|/|'`<br />
EOF<br />
</pre><br />
<br />
Note the differences between a Metajob and a normal Job.<br />
;Plugin<br />
:'''local''' is a special plugin that tells Checkbox to run this job as soon as the tool starts. This means this job is executed well before the user ever sees a Suite or Test Selection screen.<br />
;requires<br />
:This job also introduces the '''requires''' directive. This means that the following conditions must be met before the job will be run. <br />
:Also note that you can specify multiple requirements by placing each one on a new line with a single-space indention.<br />
;Command<br />
:In the basic example, I mentioned that a '''Command''' can be embedded shell code. This is an example of that embedded shell code. This runs the shell code that follows. In this example, it cats all the text between the 'cat' line and the 'EOF' marker (which looks exactly like the Basic Job Description) and pipes all that through the 'run-templates' helper tool which creates on-the-fly jobs. In this case, 'run-templates' also executes the udev-resource resource gathering script and ensures that child jobs are ONLY created for hardware devices that fit into the DISK category (so we don't create these Disk jobs for network devices or CPUs).<br />
<br />
==== Scripts ====<br />
Scritps are found in the bin/ directory. These can be anything from bash to python to ruby to anything you feel like coding in. Scripts can be actual tests themselves, or wrappers that call other tools or test suites. In the past, we have had wrapper scripts that ran things like Phoronix Test Suite and Piglit (for desktop testing).<br />
<br />
==== Whitelists ====<br />
Whitelists are text lists that define the test suite. This is a list of tests to run. Below is a sample whitelist:<br />
<pre><br />
# Resource Jobs<br />
miscellanea/submission-resources<br />
#Info attachment jobs<br />
__info__<br />
cpuinfo_attachment<br />
dmesg_attachment<br />
dmi_attachment<br />
dmidecode_attachment<br />
efi_attachment<br />
lspci_attachment<br />
lshw_attachment<br />
mcelog_attachment<br />
meminfo_attachment<br />
modprobe_attachment<br />
modules_attachment<br />
sysctl_attachment<br />
sysfs_attachment<br />
udev_attachment<br />
lsmod_attachment<br />
acpi_sleep_attachment<br />
info/hdparm<br />
info/hdparm_.*.txt<br />
installer_debug.gz<br />
info/disk_partitions<br />
# Actual test cases<br />
__TC-003__<br />
TC-003-0003-001/Network_performance<br />
TC-003-0003-001/Network_performance_.*<br />
</pre><br />
<br />
The first job, '''miscellanea/submission-resources''' is always required in a whitelist. It is what triggers all the various resource jobs to run as well as jobs necessary to create a full submission file. After that, add whatever jobs you need. The first group in the example above are jobs that attach files containing the output of various commands. <br />
<br />
In the next section is an example of two things. The job '''TC-003-0003-001/Network_performance''' is a "metajob" that generates other jobs. IN this case, it's a job that creates a separate network test for each NIC discovered on the SUT. The next job, '''TC-003-0003-001/Network_performance_.*''' is an example of using a regular expression. In this case, this line item tells Checkbox to run any job that starts with the text 'TC-003-0003-001/Network_performance_' (which when run would be jobs like 'TC-003-0003-001/Network_performance_eth0' and 'TC-003-0003-001/Network_performance_eth1'.<br />
<br />
== Submitting Code to the Project ==<br />
The following will cover how to submit code once you've made changes to your branch. The examples here assume you have already performed the following steps:<br />
#Branched lp:opencompute/opencompute-testing.<br />
#Made changes or additions<br />
#Tested your changes or additions<br />
#Committed your changes or additions<br />
<br />
=== Launchpad ===<br />
As noted early on, you will need to be logged in via bzr to submit code. This is accomplished using the '''bzr launchpad-login''' command:<br />
<pre><br />
bzr launchpad-login <YOUR_LAUNCHPAD_USERNAME><br />
</pre><br />
<br />
You will likely also need to have created and submitted an SSH key to Launchpad to use for submissions. This is explained here:<br />
https://help.launchpad.net/YourAccount/CreatingAnSSHKeyPair<br />
<br />
=== Pushing Changes ===<br />
Once you have committed your changes, you need to push them to YOUR OWN branch on Launchpad. To do so, cd into your branch and do the following:<br />
<pre><br />
bzr push lp:~<YOUR LP USERNAME>/opencompute/<YOUR BRANCH NAME><br />
</pre><br />
<br />
Example:<br />
<pre><br />
bzr push lp:~bladernr/opencompute/ocp-cert-cleanup<br />
</pre><br />
<br />
=== Merge Requests ===<br />
Once you have pushed your branch to Launchpad, you need to make a merge request. To do so you need to do the following things:<br />
# In a web browser, go to http://code.launchpad.net/~YOUR_LAUNCHPAD_USERNAME<br />
# Click on the name of the branch you just pushed<br />
# Under '''Branch Merges''' click on '''Propose for merging'''<br />
# Under '''Target Branch''' select '''Other''' and enter '~opencompute-developers/opencompute/opencompute-testing'<br />
# In the Description text box, enter a summary of the changes you made<br />
# Click the '''Propose Merge''' button at the bottom of that page<br />
<br />
This is submit your merge request. At this point, it will be reviewed, commented on, approved or sent back for fixing.<br />
<br />
== Questions? ==<br />
=== OCP Ready or OCP Certification===<br />
=== CheckboxNG or Plainbox ===</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/ToolsDevelopmentGuideline&diff=1060C&I Wiki Portal/ToolsDevelopmentGuideline2015-04-02T20:10:31Z<p>Jlane: Created page with "== Overview == The official repository for the source code to OCP Certification and OCP Ready is [http://launchpad.net/opencompute Launchpad]. There are a number of projects ..."</p>
<hr />
<div>== Overview ==<br />
The official repository for the source code to OCP Certification and OCP Ready is [http://launchpad.net/opencompute Launchpad]. There are a number of projects hosted there but the most important one is [http://code.launchpad.net/opencompute-testing opencompute-testing].<br />
<br />
The '''opencompute-testing''' project contains all the code for generating the OCP Certification and OCP Ready testing tools. It is the code that creates the Plainbox Provider and Launcher Packages, hosting the job descriptions that are OCP Specific, scripts and whitelists. Package builds occur automatically once per day if the code base has been updated.<br />
<br />
=== Contacts ===<br />
If you have ANY questions at all about the code base, contributing, or anything else related, contact [mailto:jeff@canonical.com Jeff Lane]<br />
For anything related to Open Compute Certification (including code questions):<br />
* Mailing List: http://lists.opencompute.org/mailman/listinfo/opencompute-ci<br />
* IRC: [http://freenode.net Freenode IRC Servers] #ocp-ci<br />
<br />
== Basics ==<br />
This section will cover some of the basics for contributing to the code for these tools. This is a general overview, more specific information can be found on line at the links below.<br />
<br />
=== Bazaar ===<br />
==== About Bazaar ====<br />
Bazaar is a version control system that helps you track project history over time and to collaborate easily with others. Whether you're a single developer, a co-located team or a community of developers scattered across the world, Bazaar scales and adapts to meet your needs. Part of the GNU Project, Bazaar is free software sponsored by Canonical<br />
<br />
You can learn more at http://bazaar.canonical.com/en/<br />
<br />
==== Obtaining Bazaar ====<br />
To install Bazaar on a debian or ubuntu system:<br />
<code>sudo apt-get install bazaar</code><br />
<br />
For other Operating Systems, including Windows, Red Hat, SuSE, OSX and more, please see http://wiki.bazaar.canonical.com/Download<br />
<br />
==== Learning Bazaar ====<br />
Bazaar is easy to learn. There is an excellent [http://doc.bazaar.canonical.com/latest/en/mini-tutorial/ 5 Minute Tutorial] that should get you started. Beyond that, bzr has a fairly well developed man page and in-program help system. <br />
<br />
Please read and do the 5 Minute Tutorial now, which will teach you the basics like creating branches, logging into LP, pushing and merging.<br />
<br />
===== Trees, Branches, Trunks? =====<br />
The basic top level of a codebase is called the Trunk. The Trunk is the mainline code that all branches are a part of. Branches are localized copies of Trunk. Branches are where you make changes for submission and merge consideration. Once accepted, Branches are merged into Trunk and become part of the mainline codebase. <br />
<br />
===== Basic Commands =====<br />
* The most basic command is: <code>bzr</code><br />
* To get a local copy of a project: <code>bzr branch lp:opencompute/opencompute-testing</code><br />
* To review a change summary: <code>bzr status</code><br />
* To add new files that are not yet managed: <code>bzr add path/to/filename</code><br />
** Note that /path/to/filename needs to be a path INSIDE your branch. When adding files not in the branch, you need to copy those files into the branch before adding them.<br />
* To see a diff of changes: <code>bzr diff path/to/filename</code><br />
* To commit your changes to a new revision <code>bzr commit -m "Some descriptive summary of your changes"</code><br />
* To push your changes to launchpad in preparation for a Merge Request: <code>bzr push lp:~<Your Launchpad Username>/opencompute/<Your Branch Name></code><br />
** for example: <code>bzr push lp:~bladernr/opencompute/ocp-testing-job-updates</code><br />
<br />
There are many MANY more commands but those are the ones you'll likely use most frequently.<br />
<br />
===== Learning More =====<br />
* http://doc.bazaar.canonical.com/en/<br />
* http://wiki.bazaar.canonical.com/Tutorials<br />
* https://yade-dem.org/wiki/Quick_Bazaar_tutorial<br />
* http://wiki.bazaar.canonical.com/Tutorials<br />
<br />
=== Practices ===<br />
==== Languages to use ====<br />
===== Main Tooling =====<br />
Plainbox, Checkbox, the providers and such are all written in Python3.<br />
<br />
===== Job Files and Whitelists =====<br />
Job Files are the same as they were in Checkbox, that is RFC822 compliant text files. See any file in the <code>jobs/</code> directory for examples.<br />
Whitelists are also simply text files, HOWEVER, you can also use regular expressions in job naming to pattern match jobs.<br />
<br />
===== Test Scripts =====<br />
These are scripts that perform actual testing. For example, a storage IO test, or a network performance test. Scripts can be written in pretty much anything that is compiled or has a valid interpreter installed on the System Under Test. The following are just a few examples of the languages you can write test scripts in:<br />
* Python3<br />
* Bash<br />
* Ruby<br />
* Perl<br />
<br />
And some more complex ones<br />
* C/C++<br />
* Go<br />
'Please note that compiled languages will need special consideration and handling in order to get the packages to build properly. Packages are built for AMD64 and i386 for now. We have the capability to also compile them for OpenPower, ARM and ARM64, but that will require some porting work.<br />
<br />
In general, we advise the following for test scripts:<br />
* Python 3 for pretty much everything<br />
* BASH for one-liners and short simple scripts. Generally, if it starts feeling complex, it's time to re-write in Python3<br />
* If you MUST use compiled languages, C but this is exceedingly rare and should only be done when something easier to maintain is not appropriate<br />
<br />
==== Translations ====<br />
We currently do not have any Translations for the tooling, nor are there plans to enable this.<br />
<br />
==== Style Guide ====<br />
The general rule is Python code should adhere to [https://www.python.org/dev/peps/pep-0008/ Python Foundation's PEP-8]. There are several tools to help you check your code for adherence such as [https://pypi.python.org/pypi/pep8/1.6.2 pep8] and [http://www.pylint.org/ Pylint] and many others.<br />
<br />
== Project Hierarchy ==<br />
=== opencompute-certification ===<br />
==== What this provides ====<br />
This is the trunk for the Open Compute Project's certification tooling. It can be found on Launchpad:<br />
https://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
This code is used to build the launcher packages:<br />
* opencompute-ready<br />
* opencompute-certification<br />
<br />
and the provider package:<br />
* plainbox-provider-opencompute-certification<br />
<br />
==== What's a Provider? ====<br />
A Provider is a package that provides the bits necessary for Plainbox/Checkbox-NG to perform testing. These files include job descriptions, whitelists, test scripts and wrappers, helpers, custom libraries and anything else that you may need.<br />
<br />
==== What's a Launcher? ====<br />
A Launcher is a package that does two things. <br />
# It provides the actual program launcher (the command you issue to start the testing) such as <code>opencompute-certification</code><br />
# It ensures the lower-level packages and dependencies are installed<br />
=== checkbox ===<br />
==== What this provides ====<br />
This is the main project for Canonical's certification tool. This provides the back-end and underlying support structure as well as some of the general test scripts and jobs. The following packages make up our use of Checkbox, in order from top to bottom.<br />
<br />
===== plainbox-provider-checkbox =====<br />
This package provides the actual checkbox test tools used by Canonical Certification. Many tests in OCP Certification and OCP Ready are based on the jobs, scripts and data in this provider.<br />
<br />
===== plainbox-provider-resource-generic =====<br />
This package provides system resource jobs and libraries. These are used at run-time to gather system information and provide that data to CheckboxNG for job generation, to the results files for system hardware inventory, and to provide the validation that allows certain jobs to run.<br />
<br />
To explain that last statement, one job of this is to determine if certain hardware exists, and if that hardware exists, then the jobs that test that hardware are configured and executed. For example, there is a USB resource that determines if a given motherboard has a USB controller. If the USB Controller is detected by the USB Resource, then the USB test jobs will be run. If the USB Resource does NOT detect a USB controller, then the USB jobs will be skipped.<br />
<br />
===== python3-checkbox-support =====<br />
This package is an infrastructure package that provides several python3 support libraries that Checkbox needs to run.<br />
<br />
===== checkbox-ng =====<br />
This package is the back-end. It provides Checkbox-NG, the Plainbox based test harness that replaces the old Checkbox tool. It provides the internals that organize and run all the various test cases, performs all the parsing and heavy work of Certification Testing.<br />
<br />
==== Documentation? ====<br />
For the most part, Documentation for the OCP Tools can be found here on the OCP Certification Wiki. For Checkbox, documentation can be found:<br />
* http://checkbox.readthedocs.org/en/latest/<br />
* http://plainbox.readthedocs.org/en/latest/<br />
<br />
As CheckboxNG and Plainbox are under active development, this documentation is constantly evolving and improving.<br />
<br />
==== How can I contribute to this? ====<br />
There are several ways to contribute. For the most part, you'll likely want to contribute to test scripts, jobs and whitelist changes, so you will mostly be interested in the '''opencompute-testing''' project.<br />
<br />
To get started, follow the bits above to branch the project (lp:opencompute/opencompute-testing) and start hacking!<br />
<br />
If you'd like to contribute to the Checkbox project, please see the [[#Questions?|Questions?]] section below.<br />
<br />
=== Putting it all together ===<br />
So now that you know about the various packages that make up the OCP Certification and OCP Ready test tools, let's install them so you can try them out to see what they do.<br />
==== Installation of Packages ====<br />
First, we need to install the PPAs. Because this is based on Canonical's actual certification tooling instead of a fork, we need to add two PPAs:<br />
<br />
<code><br />
sudo apt-add-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
<br />
sudo apt-add-repository ppa:hardware-certification/public<br />
</code><br />
<br />
Next, we need to update the apt cache:<br />
<br />
<code>sudo apt update</code><br />
<br />
And finally, install the packages:<br />
<br />
<code>sudo apt-get install opencompute-certification</code><br />
<br />
==== Interaction ====<br />
After you have done the above, you will see a long list of packages that need to be installed. These all provide testing tools, libraries and other things necessary for Certification testing. The number of packages will vary depending on how you installed the OS onto the SUT, what version is being used, what tool is used (Ready vs Certification) and so forth. <br />
<br />
Hit Y and the installation will begin.<br />
<br />
Once installation is complete, to check it out qickly, simply run the command <code>opencompute-certification</code> or <code>opencompute-ready</code><br />
<br />
To do proper testing, there is some configuration and other things that need to be performed first, but this will get you an idea of how the tool works and what it looks like.<br />
<br />
== Obtaining the Code ==<br />
=== Locations ===<br />
For now, there are multiple locations to find testing tools. In the near future, these will be merged into the Launchpad branches so that all code is easily found and maintained.<br />
==== Launchpad ====<br />
The main project code is housed in the following projects on Launchpad:<br />
* [https://code.launchpad.net/opencompute/opencompute-testing lp:opencompute/opencompute-testing]<br />
* [https://code.launchpad.net/checkbox lp:checkbox]<br />
==== GitHub ====<br />
ITRI has placed the majority of the Certification test scripts, job files and such on GitHub:<br />
* https://github.com/opencomputeproject/ocp_ci_server_certification_test_cases<br />
=== Pulling the code ===<br />
See above about using '''bzr''' to obtain local branches of the Checkbox or OpenCompute Testing trunks.<br />
<br />
For the ITRI github location, you'll need to use git.<br />
<br />
== Adding new code ==<br />
The following is meant to help you understand where to add things in the code.<br />
=== opencompute-certification ===<br />
==== Trunk Contents ====<br />
The trunk contains the following directories:<br />
;bin<br />
:This directory contains all the test scripts that are not part of the lp:checkbox branch. These are the OCP specific test scripts.<br />
;COPYING<br />
:This is a copy of the license. All code is Open Source under the GPLv3. Proprietary code is not allowed in the certification suite.<br />
;data<br />
:This directory is used in case sample test data needs to be installed along with the test suite.<br />
;debian<br />
:This directory contains all the necessary Debian packaging information. It also contains the Changelog. Likely, you will only ever need to edit the changelog in this directory.<br />
;examples<br />
:This directory contains sample config files that are installed when a given package is installed.<br />
;jobs<br />
:This directory contains the job description files. Jobs are the units of Test that are performed.<br />
;launcher<br />
:This directory contains the launcher file that creates the command used to execute the test tool on the SUT<br />
;manage.py<br />
:This is a helper program for testing and building packages.<br />
;po<br />
:UNUSED CURRENTLY, but this would house translation data if implemented.<br />
;README.development<br />
:A README file that gives you basic development information. Please read this file.<br />
;README.rst <br />
:A description of the trunk contents (the same thing you're reading right now)<br />
;requirements <br />
:Package building requirements.<br />
;whitelists<br />
:The whitelists that define each test suite.<br />
<br />
==== Job Files ====<br />
As discussed before, job files contain the descriptions that define each test case or unit of test. <br />
===== Basic Job Description =====<br />
An example job description looks like this:<br />
<br />
<pre><br />
plugin: shell<br />
name: memory/ocp/stress_30min<br />
user: root<br />
command: stressapptest -s 1800<br />
_description:<br />
Test and exercise memory.<br />
</pre><br />
<br />
This is a very basic example that contains most of the common KEY:VALUE pairs that are used to describe a test.<br />
;plugin<br />
:Plugins are the type of job to perform. <br />
:'''shell''' is a fully automated job that is executed by the test harness automatically with no user intervention. <br />
:'''manual''' is a test/job that requires the user to perform some task and verify the results.<br />
:'''user-interact''' which also requires the user to initiate the test with the PASS or FAIL results being automatically determined by Checkbox<br />
:'''user-interact-verify''' requires the user to initiate the test AND validate the results, choosing PASS or FAIL afterwards.<br />
;name<br />
:This is a descriptive name of the test. In this case, the hierarchy means that it's a Memory test that is part of the OCP Specific tests and the test case is 'stress_30min'<br />
;user <br />
:Some jobs can be run as an alternate user (in fact, some require this). So specifying '''root''' here will tell Checkbox to run the '''command''' as the root user (essentially using sudo to elevate privileges for this one test only).<br />
;command<br />
:This is the command to perform. This item can be in-line code (like a bash oneliner or test) or the name of a command that is in the bin/ directory.<br />
;_description<br />
:This is a text description of the job. For a Manual job, '''description''' would list the steps necessary to perform the test. For an automated test, this is just a summary of what the test does.<br />
<br />
===== Meta-Jobs =====<br />
There are also things we refer to as "Metajobs". These jobs are actually run early on and are simply used to generate jobs on the fly. In general, you would use a metajob to create separate jobs for multiple pieces of the same hardware. In the following example, we have a metajob that creates a separate storage test for any disk device (NON-USB) discovered.<br />
<br />
<pre><br />
plugin: local<br />
name: disk/ocp/io_stress<br />
requires:<br />
device.category == 'DISK'<br />
package.name == 'stressapptest'<br />
_description: Verify that storage devices, such as Fibre Channel and RAID can be detected and perform under stress.<br />
command:<br />
cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"'<br />
plugin: shell<br />
name: disk/ocp/io_stress_`ls /sys$path/block`<br />
user: root<br />
requires:<br />
device.path == "$path"<br />
block_device.`ls /sys$path/block`_state != 'removable'<br />
description: Disk I/O stress test for $product<br />
command: disk_stress `ls /sys$path/block | sed 's|!|/|'`<br />
EOF<br />
</pre><br />
<br />
Note the differences between a Metajob and a normal Job.<br />
;Plugin<br />
:'''local''' is a special plugin that tells Checkbox to run this job as soon as the tool starts. This means this job is executed well before the user ever sees a Suite or Test Selection screen.<br />
;requires<br />
:This job also introduces the '''requires''' directive. This means that the following conditions must be met before the job will be run. <br />
:Also note that you can specify multiple requirements by placing each one on a new line with a single-space indention.<br />
;Command<br />
:In the basic example, I mentioned that a '''Command''' can be embedded shell code. This is an example of that embedded shell code. This runs the shell code that follows. In this example, it cats all the text between the 'cat' line and the 'EOF' marker (which looks exactly like the Basic Job Description) and pipes all that through the 'run-templates' helper tool which creates on-the-fly jobs. In this case, 'run-templates' also executes the udev-resource resource gathering script and ensures that child jobs are ONLY created for hardware devices that fit into the DISK category (so we don't create these Disk jobs for network devices or CPUs).<br />
<br />
==== Scripts ====<br />
Scritps are found in the bin/ directory. These can be anything from bash to python to ruby to anything you feel like coding in. Scripts can be actual tests themselves, or wrappers that call other tools or test suites. In the past, we have had wrapper scripts that ran things like Phoronix Test Suite and Piglit (for desktop testing).<br />
<br />
==== Whitelists ====<br />
Whitelists are text lists that define the test suite. This is a list of tests to run. Below is a sample whitelist:<br />
<pre><br />
# Resource Jobs<br />
miscellanea/submission-resources<br />
#Info attachment jobs<br />
__info__<br />
cpuinfo_attachment<br />
dmesg_attachment<br />
dmi_attachment<br />
dmidecode_attachment<br />
efi_attachment<br />
lspci_attachment<br />
lshw_attachment<br />
mcelog_attachment<br />
meminfo_attachment<br />
modprobe_attachment<br />
modules_attachment<br />
sysctl_attachment<br />
sysfs_attachment<br />
udev_attachment<br />
lsmod_attachment<br />
acpi_sleep_attachment<br />
info/hdparm<br />
info/hdparm_.*.txt<br />
installer_debug.gz<br />
info/disk_partitions<br />
# Actual test cases<br />
__TC-003__<br />
TC-003-0003-001/Network_performance<br />
TC-003-0003-001/Network_performance_.*<br />
</pre><br />
<br />
The first job, '''miscellanea/submission-resources''' is always required in a whitelist. It is what triggers all the various resource jobs to run as well as jobs necessary to create a full submission file. After that, add whatever jobs you need. The first group in the example above are jobs that attach files containing the output of various commands. <br />
<br />
In the next section is an example of two things. The job '''TC-003-0003-001/Network_performance''' is a "metajob" that generates other jobs. IN this case, it's a job that creates a separate network test for each NIC discovered on the SUT. The next job, '''TC-003-0003-001/Network_performance_.*''' is an example of using a regular expression. In this case, this line item tells Checkbox to run any job that starts with the text 'TC-003-0003-001/Network_performance_' (which when run would be jobs like 'TC-003-0003-001/Network_performance_eth0' and 'TC-003-0003-001/Network_performance_eth1'.<br />
<br />
== Submitting Code to the Project ==<br />
=== Launchpad ===<br />
=== Pushing Changes ===<br />
=== Merge Requests ===<br />
<br />
== Questions? ==<br />
=== OCP Ready or OCP Certification===<br />
=== CheckboxNG or Plainbox ===</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=1039C&I WIki Portal/OCP Checkbox2015-03-25T22:12:51Z<p>Jlane: </p>
<hr />
<div>==IMPORTANT ANNOUNCEMENT==<br />
'''ANNOUNCEMENT''' There are now two versions of this tool available. The old version, known as OCP Checkbox will sunset at some point in the future. The new version, based on a new harness called Plainbox will take it's place. In the interim, both versions will be available to allow for continued Certification Testing activities while the new version is beta tested, tweaked and made ready for General availability. The following is an estimated outline of the transition process. There are no dates attached to these as each stage could take a variable amount of time.<br />
<br />
# opencompute-certification available in the ppa '''[DONE]'''<br />
# opencompute-certification tested '''[IN PROGRESS]'''<br />
# Option to use either OCP Checkbox or opencompute-certification for official testing '''[TO DO]'''<br />
# Require opencompute-certification for official testing (OCP Checkbox still available) '''[TO DO]'''<br />
# OCP Checkbox Sunests, opencpmpute-certification the only available toolset '''[TO DO]'''<br />
<br />
==OCP Checkbox==<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc). As of OCP Summit 2015 in San Jose, OCP Checkbox has reached End of Life. It is currently in essentially non-maintained mode though bug fixes will be applied in emergencies. It is being replaced by a new tool based on Plainbox, which will be described below. At some point in the future, OCP Checkbox will be removed entirely from the Open Compute Tools PPA.<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time. <br />
<br />
Checkbox (and it's next generation) is an extensible test harness that is capable of running tests that are in pretty much any language, interpreted or compiled, and even running external test suites like Phoronix, NGINX, etc.<br />
<br />
opencompute-certification is a new package that is responsible for providing the test launcher and whitelists and certification specific job files and scripts that are not already a part of the plainbox toolset from Canonical. This next generation test harness is a complete, ground-up replacement for the old Checkbox, designed from the ground up to be modular, extensible, useful, and adaptable. It is under very active development and because of the new process for maintaining packaging for OCP Certification, testers should always have the latest bug fixes made available.<br />
<br />
==OCP Certification==<br />
opencompute-certification is the package name of the next generation Open Compute Certification Test tool. It is based on a tool called Plainbox, developed by the Hardware Certification team at Canonical. Essentially, it is a drop-in replacement for OCP Checkbox and the end user should not notice much change. <br />
<br />
===End User Differences===<br />
Mostly, the end user will notice that the menuing system is better, the options for picking tests and suites are improved, as is job handling, runtime output, logging and results output. <br />
<br />
===Developer Differences===<br />
The Test Developer will see some more difference, but again, the difference should not be overwhelming.<br />
* Plainbox honors the same job definition format that Checkbox does. It adds several new definition items that are entirely optional<br />
* Plainbox honors the same whitelist format. It also includes better parsing and handling of regexs in whitelists. Additionally, the long list of "resource" jobs has been replaced with a single job, making whitelist development a bit easier.<br />
* Plainbox still runs anything you ask it too, be it Python, C, C++, Bash, Perl, Go, etc). Anything that needs to be compiled is compiled per architecture during packaging so the end user will never see a difference in that respect. <br />
* Plainbox provides better output. XML, HTML and XLSX is available<br />
* Plainbox adds better built in unit testing ability (This is not currently active in the codebase, but can be added in)<br />
* The package hierarchy is different and will be explained below in more detail.<br />
<br />
===About the Tools===<br />
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.<br />
<br />
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:<br />
;opencompute-certification : Provides the launcher and ensures the necessary package dependencies are installed<br />
;plainbox-provider-opencompute-testing : Provides OCP Specific whitelists and test scripts, job definitions and other data for testing<br />
;plainbox-provider-checkbox : Provides the generic test scripts and job descriptions. This is the exact same package used by Canonical Hardware Certification, and thus is considered an upstream package for opencompute-certification.<br />
;plainbox-provider-resource-generic : Provides resource jobs and scripts and libraries necessary to ascertain system hardware inventory and system specific data. This too is the exact same package used by Canonical Hardware Certification and thus is considered an upstream package.<br />
;checkbox-ng : The underlying test harness based on Plainbox. This is also the exact package used by Canonical Certification.<br />
<br />
The design relies heavily on the existing, current Canonical packages because it is far easier to maintain just the provider and launcher packages necessary for OCP while reusing existing, well maintained tools underneath. It avoids having to maintain an active fork and further avoids issues with forking the fork causing an irreconcilable delta when updates and bug fixes are necessary.<br />
<br />
For rare occasions where a fork IS necessary, it can be more gracefully handled.<br />
<br />
Additionally, the design allows the easy ability to spin off new providers or launchers as necessary to create custom tailored test packages for each OCP Project or Platform. (Imagine opencompute-certification-winterfell and opencompute-certification-leopard and opencompute-storage-certification-knox)<br />
<br />
This will change slightly as development progresses.<br />
<br />
Developer level documentation can be found here:<br />
;[http://checkbox.readthedocs.org CheckBox-NG]<br />
;[http://plainbox.readthedocs.org PlainBox]<br />
<br />
Keep in mind that the documentation is developer focused, not user focused, and is also actively developed and thus could change in the future.<br />
<br />
==Testers==<br />
===Installation===<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
* The SUT will need a method to access the internet for package installation<br />
<br />
The OCP Certification packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-certification</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-certification<br />
<br />
and follow the prompts.<br />
<br />
===Suite Selection===<br />
After the welcome screen, you will see the Suite Selection screen. This will list all available whitelists for testing. You must select at least one (The "local" whitelist is selected by default) but you may also select more than one. <br />
<br />
Selecting multiple whitelists will cause opencompute-certification to create a superset of the tests in all selected whitelists, properly ordered according to dependency, job type and whitelist order.<br />
<br />
===Test Selection===<br />
Once you have selected the suite or suites, you will see a test selection screen where you can review or edit the tests to be run. For Certification, you should not de-select any tests.<br />
<br />
===After Testing===<br />
Once testing is complete, you will see a test summary on the console as well as indications on where to find the result output files. You will be provided output files in HTML, XML and XLSX.<br />
<br />
==Developers==<br />
Developers are welcome to participate. The test tools, both the OCP Specific packages and the Canonical Certification packages are completely Open Source software under the GPL v3.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
We hang out on Freenode IRC in the #opencompute-ci channel<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The OCP Certification source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/opencompute-testing<br />
:The Canonical Certification source code here: https://code.launchpad.net/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
Coding can be anything from fixing bugs (if any) to assisting with the plainbox port to writing new test scripts and jobs. The latter is most important as we want to ensure that the OCP Ready and OCP Certified specs are thoroughly tested and test cases are represented in the test tools. <br />
<br />
Specifically, help is appreciated in upstreaming individual jobs and whitelists and other files stored in ITRI's github location. The goal is to move all that code upstream into the OCP Certification code base and maintain it there. This will provide a single location for ALL OCP Certification testing tools, and eliminate some of the current complexity of installing tools from one place then overlaying additional tools from other places.<br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
Currently, there is limited documentation available, and this is more akin to generic documentation and development documentation. We could really use user guides from an OCP point of view that explains how to perform testing, how to install the tools, how to USE the tools as well as documentation on how to write test cases and mapping existing test cases to OCP Ready and OCP Certified test cases in the specs.<br />
<br />
Additionally, documentation on building alternate packaging (RPM) and installing/running the tools on OSs other than Ubuntu Server would be useful.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases. It is packaged in .deb only so any additional formats will need to be created (.rpm, for instance).<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.<br />
<br />
Feel free to ping Jeff Lane (bladernr, bladernr_ on freenode) in #ocp-ci or via [mailto:jeff@ubuntu.com E-Mail]</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=1038C&I WIki Portal/OCP Checkbox2015-03-25T22:11:07Z<p>Jlane: </p>
<hr />
<div>==IMPORTANT ANNOUNCEMENT==<br />
'''ANNOUNCEMENT''' There are now two versions of this tool available. The old version, known as OCP Checkbox will sunset at some point in the future. The new version, based on a new harness called Plainbox will take it's place. In the interim, both versions will be available to allow for continued Certification Testing activities while the new version is beta tested, tweaked and made ready for General availability. The following is an estimated outline of the transition process. There are no dates attached to these as each stage could take a variable amount of time.<br />
<br />
# opencompute-certification available in the ppa '''[DONE]'''<br />
# opencompute-certification tested '''[IN PROGRESS]'''<br />
# Option to use either OCP Checkbox or opencompute-certification for official testing '''[TO DO]'''<br />
# Require opencompute-certification for official testing (OCP Checkbox still available) '''[TO DO]'''<br />
# OCP Checkbox Sunests, opencpmpute-certification the only available toolset '''[TO DO]'''<br />
<br />
==OCP Checkbox==<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc). As of OCP Summit 2015 in San Jose, OCP Checkbox has reached End of Life. It is currently in essentially non-maintained mode though bug fixes will be applied in emergencies. It is being replaced by a new tool based on Plainbox, which will be described below. At some point in the future, OCP Checkbox will be removed entirely from the Open Compute Tools PPA.<br />
<br />
==OCP Certification==<br />
opencompute-certification is the package name of the next generation Open Compute Certification Test tool. It is based on a tool called Plainbox, developed by the Hardware Certification team at Canonical. Essentially, it is a drop-in replacement for OCP Checkbox and the end user should not notice much change. <br />
<br />
===End User Differences===<br />
Mostly, the end user will notice that the menuing system is better, the options for picking tests and suites are improved, as is job handling, runtime output, logging and results output. <br />
<br />
===Developer Differences===<br />
The Test Developer will see some more difference, but again, the difference should not be overwhelming.<br />
* Plainbox honors the same job definition format that Checkbox does. It adds several new definition items that are entirely optional<br />
* Plainbox honors the same whitelist format. It also includes better parsing and handling of regexs in whitelists. Additionally, the long list of "resource" jobs has been replaced with a single job, making whitelist development a bit easier.<br />
* Plainbox still runs anything you ask it too, be it Python, C, C++, Bash, Perl, Go, etc). Anything that needs to be compiled is compiled per architecture during packaging so the end user will never see a difference in that respect. <br />
* Plainbox provides better output. XML, HTML and XLSX is available<br />
* Plainbox adds better built in unit testing ability (This is not currently active in the codebase, but can be added in)<br />
* The package hierarchy is different and will be explained below in more detail.<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time. <br />
<br />
Checkbox (and it's next generation) is an extensible test harness that is capable of running tests that are in pretty much any language, interpreted or compiled, and even running external test suites like Phoronix, NGINX, etc.<br />
<br />
opencompute-certification is a new package that is responsible for providing the test launcher and whitelists and certification specific job files and scripts that are not already a part of the plainbox toolset from Canonical. This next generation test harness is a complete, ground-up replacement for the old Checkbox, designed from the ground up to be modular, extensible, useful, and adaptable. It is under very active development and because of the new process for maintaining packaging for OCP Certification, testers should always have the latest bug fixes made available.<br />
<br />
===About the Tools===<br />
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.<br />
<br />
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:<br />
;opencompute-certification : Provides the launcher and ensures the necessary package dependencies are installed<br />
;plainbox-provider-opencompute-testing : Provides OCP Specific whitelists and test scripts, job definitions and other data for testing<br />
;plainbox-provider-checkbox : Provides the generic test scripts and job descriptions. This is the exact same package used by Canonical Hardware Certification, and thus is considered an upstream package for opencompute-certification.<br />
;plainbox-provider-resource-generic : Provides resource jobs and scripts and libraries necessary to ascertain system hardware inventory and system specific data. This too is the exact same package used by Canonical Hardware Certification and thus is considered an upstream package.<br />
;checkbox-ng : The underlying test harness based on Plainbox. This is also the exact package used by Canonical Certification.<br />
<br />
The design relies heavily on the existing, current Canonical packages because it is far easier to maintain just the provider and launcher packages necessary for OCP while reusing existing, well maintained tools underneath. It avoids having to maintain an active fork and further avoids issues with forking the fork causing an irreconcilable delta when updates and bug fixes are necessary.<br />
<br />
For rare occasions where a fork IS necessary, it can be more gracefully handled.<br />
<br />
Additionally, the design allows the easy ability to spin off new providers or launchers as necessary to create custom tailored test packages for each OCP Project or Platform. (Imagine opencompute-certification-winterfell and opencompute-certification-leopard and opencompute-storage-certification-knox)<br />
<br />
This will change slightly as development progresses.<br />
<br />
Developer level documentation can be found here:<br />
;[http://checkbox.readthedocs.org CheckBox-NG]<br />
;[http://plainbox.readthedocs.org PlainBox]<br />
<br />
Keep in mind that the documentation is developer focused, not user focused, and is also actively developed and thus could change in the future.<br />
<br />
==Testers==<br />
===Installation===<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
* The SUT will need a method to access the internet for package installation<br />
<br />
The OCP Certification packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-certification</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-certification<br />
<br />
and follow the prompts.<br />
<br />
===Suite Selection===<br />
After the welcome screen, you will see the Suite Selection screen. This will list all available whitelists for testing. You must select at least one (The "local" whitelist is selected by default) but you may also select more than one. <br />
<br />
Selecting multiple whitelists will cause opencompute-certification to create a superset of the tests in all selected whitelists, properly ordered according to dependency, job type and whitelist order.<br />
<br />
===Test Selection===<br />
Once you have selected the suite or suites, you will see a test selection screen where you can review or edit the tests to be run. For Certification, you should not de-select any tests.<br />
<br />
===After Testing===<br />
Once testing is complete, you will see a test summary on the console as well as indications on where to find the result output files. You will be provided output files in HTML, XML and XLSX.<br />
<br />
==Developers==<br />
Developers are welcome to participate. The test tools, both the OCP Specific packages and the Canonical Certification packages are completely Open Source software under the GPL v3.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
We hang out on Freenode IRC in the #opencompute-ci channel<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The OCP Certification source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/opencompute-testing<br />
:The Canonical Certification source code here: https://code.launchpad.net/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
Coding can be anything from fixing bugs (if any) to assisting with the plainbox port to writing new test scripts and jobs. The latter is most important as we want to ensure that the OCP Ready and OCP Certified specs are thoroughly tested and test cases are represented in the test tools. <br />
<br />
Specifically, help is appreciated in upstreaming individual jobs and whitelists and other files stored in ITRI's github location. The goal is to move all that code upstream into the OCP Certification code base and maintain it there. This will provide a single location for ALL OCP Certification testing tools, and eliminate some of the current complexity of installing tools from one place then overlaying additional tools from other places.<br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
Currently, there is limited documentation available, and this is more akin to generic documentation and development documentation. We could really use user guides from an OCP point of view that explains how to perform testing, how to install the tools, how to USE the tools as well as documentation on how to write test cases and mapping existing test cases to OCP Ready and OCP Certified test cases in the specs.<br />
<br />
Additionally, documentation on building alternate packaging (RPM) and installing/running the tools on OSs other than Ubuntu Server would be useful.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases. It is packaged in .deb only so any additional formats will need to be created (.rpm, for instance).<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.<br />
<br />
Feel free to ping Jeff Lane (bladernr, bladernr_ on freenode) in #ocp-ci or via [mailto:jeff@ubuntu.com E-Mail]</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=1037C&I WIki Portal/OCP Checkbox2015-03-25T22:10:20Z<p>Jlane: /* OCP Checkbox */</p>
<hr />
<div>==IMPORTANT ANNOUNCEMENT==<br />
'''ANNOUNCEMENT''' There are now two versions of this tool available. The old version, known as OCP Checkbox will sunset at some point in the future. The new version, based on a new harness called Plainbox will take it's place. In the interim, both versions will be available to allow for continued Certification Testing activities while the new version is beta tested, tweaked and made ready for General availability. The following is an estimated outline of the transition process. There are no dates attached to these as each stage could take a variable amount of time.<br />
<br />
# opencompute-certification available in the ppa '''[DONE]'''<br />
# opencompute-certification tested '''[IN PROGRESS]'''<br />
# Option to use either OCP Checkbox or opencompute-certification for official testing '''[TO DO]'''<br />
# Require opencompute-certification for official testing (OCP Checkbox still available) '''[TO DO]'''<br />
# OCP Checkbox Sunests, opencpmpute-certification the only available toolset '''[TO DO]'''<br />
<br />
==OCP Checkbox==<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc). As of OCP Summit 2015 in San Jose, OCP Checkbox has reached End of Life. It is currently in essentially non-maintained mode though bug fixes will be applied in emergencies. It is being replaced by a new tool based on Plainbox, which will be described below. At some point in the future, OCP Checkbox will be removed entirely from the Open Compute Tools PPA.<br />
<br />
==opencompute-certification==<br />
opencompute-certification is the package name of the next generation Open Compute Certification Test tool. It is based on a tool called Plainbox, developed by the Hardware Certification team at Canonical. Essentially, it is a drop-in replacement for OCP Checkbox and the end user should not notice much change. <br />
<br />
===End User Differences===<br />
Mostly, the end user will notice that the menuing system is better, the options for picking tests and suites are improved, as is job handling, runtime output, logging and results output. <br />
<br />
===Developer Differences===<br />
The Test Developer will see some more difference, but again, the difference should not be overwhelming.<br />
* Plainbox honors the same job definition format that Checkbox does. It adds several new definition items that are entirely optional<br />
* Plainbox honors the same whitelist format. It also includes better parsing and handling of regexs in whitelists. Additionally, the long list of "resource" jobs has been replaced with a single job, making whitelist development a bit easier.<br />
* Plainbox still runs anything you ask it too, be it Python, C, C++, Bash, Perl, Go, etc). Anything that needs to be compiled is compiled per architecture during packaging so the end user will never see a difference in that respect. <br />
* Plainbox provides better output. XML, HTML and XLSX is available<br />
* Plainbox adds better built in unit testing ability (This is not currently active in the codebase, but can be added in)<br />
* The package hierarchy is different and will be explained below in more detail.<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time. <br />
<br />
Checkbox (and it's next generation) is an extensible test harness that is capable of running tests that are in pretty much any language, interpreted or compiled, and even running external test suites like Phoronix, NGINX, etc.<br />
<br />
opencompute-certification is a new package that is responsible for providing the test launcher and whitelists and certification specific job files and scripts that are not already a part of the plainbox toolset from Canonical. This next generation test harness is a complete, ground-up replacement for the old Checkbox, designed from the ground up to be modular, extensible, useful, and adaptable. It is under very active development and because of the new process for maintaining packaging for OCP Certification, testers should always have the latest bug fixes made available.<br />
<br />
===About the Tools===<br />
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.<br />
<br />
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:<br />
;opencompute-certification : Provides the launcher and ensures the necessary package dependencies are installed<br />
;plainbox-provider-opencompute-testing : Provides OCP Specific whitelists and test scripts, job definitions and other data for testing<br />
;plainbox-provider-checkbox : Provides the generic test scripts and job descriptions. This is the exact same package used by Canonical Hardware Certification, and thus is considered an upstream package for opencompute-certification.<br />
;plainbox-provider-resource-generic : Provides resource jobs and scripts and libraries necessary to ascertain system hardware inventory and system specific data. This too is the exact same package used by Canonical Hardware Certification and thus is considered an upstream package.<br />
;checkbox-ng : The underlying test harness based on Plainbox. This is also the exact package used by Canonical Certification.<br />
<br />
The design relies heavily on the existing, current Canonical packages because it is far easier to maintain just the provider and launcher packages necessary for OCP while reusing existing, well maintained tools underneath. It avoids having to maintain an active fork and further avoids issues with forking the fork causing an irreconcilable delta when updates and bug fixes are necessary.<br />
<br />
For rare occasions where a fork IS necessary, it can be more gracefully handled.<br />
<br />
Additionally, the design allows the easy ability to spin off new providers or launchers as necessary to create custom tailored test packages for each OCP Project or Platform. (Imagine opencompute-certification-winterfell and opencompute-certification-leopard and opencompute-storage-certification-knox)<br />
<br />
This will change slightly as development progresses.<br />
<br />
Developer level documentation can be found here:<br />
;[http://checkbox.readthedocs.org CheckBox-NG]<br />
;[http://plainbox.readthedocs.org PlainBox]<br />
<br />
Keep in mind that the documentation is developer focused, not user focused, and is also actively developed and thus could change in the future.<br />
<br />
==Testers==<br />
===Installation===<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
* The SUT will need a method to access the internet for package installation<br />
<br />
The OCP Certification packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-certification</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-certification<br />
<br />
and follow the prompts.<br />
<br />
===Suite Selection===<br />
After the welcome screen, you will see the Suite Selection screen. This will list all available whitelists for testing. You must select at least one (The "local" whitelist is selected by default) but you may also select more than one. <br />
<br />
Selecting multiple whitelists will cause opencompute-certification to create a superset of the tests in all selected whitelists, properly ordered according to dependency, job type and whitelist order.<br />
<br />
===Test Selection===<br />
Once you have selected the suite or suites, you will see a test selection screen where you can review or edit the tests to be run. For Certification, you should not de-select any tests.<br />
<br />
===After Testing===<br />
Once testing is complete, you will see a test summary on the console as well as indications on where to find the result output files. You will be provided output files in HTML, XML and XLSX.<br />
<br />
==Developers==<br />
Developers are welcome to participate. The test tools, both the OCP Specific packages and the Canonical Certification packages are completely Open Source software under the GPL v3.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
We hang out on Freenode IRC in the #opencompute-ci channel<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The OCP Certification source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/opencompute-testing<br />
:The Canonical Certification source code here: https://code.launchpad.net/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
Coding can be anything from fixing bugs (if any) to assisting with the plainbox port to writing new test scripts and jobs. The latter is most important as we want to ensure that the OCP Ready and OCP Certified specs are thoroughly tested and test cases are represented in the test tools. <br />
<br />
Specifically, help is appreciated in upstreaming individual jobs and whitelists and other files stored in ITRI's github location. The goal is to move all that code upstream into the OCP Certification code base and maintain it there. This will provide a single location for ALL OCP Certification testing tools, and eliminate some of the current complexity of installing tools from one place then overlaying additional tools from other places.<br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
Currently, there is limited documentation available, and this is more akin to generic documentation and development documentation. We could really use user guides from an OCP point of view that explains how to perform testing, how to install the tools, how to USE the tools as well as documentation on how to write test cases and mapping existing test cases to OCP Ready and OCP Certified test cases in the specs.<br />
<br />
Additionally, documentation on building alternate packaging (RPM) and installing/running the tools on OSs other than Ubuntu Server would be useful.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases. It is packaged in .deb only so any additional formats will need to be created (.rpm, for instance).<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.<br />
<br />
Feel free to ping Jeff Lane (bladernr, bladernr_ on freenode) in #ocp-ci or via [mailto:jeff@ubuntu.com E-Mail]</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=1025C&I Wiki Portal/OCP Ready2015-03-19T15:39:28Z<p>Jlane: </p>
<hr />
<div>== OCP Ready Test Suite Packages ==<br />
<br />
OCP Ready Test Suite is a different, more generalized test to ensure your hardware works and could be ready for full OCP Certification. <br />
<br />
=== Installing ===<br />
<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
<br />
The OCP Ready packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki>$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-ready</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-ready<br />
<br />
and follow the prompts.<br />
<br />
=== Obtaining the Source Code ===<br />
The tools are fully open source under a GPL v3 license and freely available for download here:<br />
<br />
http://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
OR via bazaar:<br />
<br />
bzr branch lp:opencompute/opencompute-testing<br />
<br />
<br />
<br />
== OCP Ready Thumbdrive ==<br />
<br />
'''NOTE: The Thumbdrive is being deprecated and likely will not be maintained. Eventually, this will be updated with instructions for creating your own'''<br />
<br />
The files for the thumbdrive will be available here:<br />
http://people.canonical.com/~jlane/ocp-thumb-drive<br />
<br />
Note that this is a rather large download (> 1GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.<br />
<br />
Additionally, the OCP Ready Thumb Drive is based on old code and is not really considered reliable. It's a nice way to demo and explore, but not really something you should be using to pre-test your hardware.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=1024C&I Wiki Portal/OCP Ready2015-03-19T15:37:58Z<p>Jlane: </p>
<hr />
<div>== OCP Ready Test Suite Packages ==<br />
<br />
OCP Ready Test Suite is a different, more generalized test to ensure your hardware works and could be ready for full OCP Certification. <br />
<br />
=== Installing ===<br />
<br />
Installing is pretty easy. You will need a couple things to get started:<br />
<br />
* A SUT (Server Under Test). This is the system you are going to be testing.<br />
* A second machine on the same LAN segment running iperf in server mode<br />
<br />
The OCP Ready packages are built for Ubuntu 14.04, 14.10 and 15.04. They are based on the new OCP Certification Suite tool which is in turn based on the new Checkbox-NG tool from Canonical.<br />
<br />
To install:<br />
<br />
* Install Ubuntu 14.04, 14.10 or 15.04 to the SUT<br />
* Once Installation is complete and the SUT is rebooted, log in as the non-privileged user.<br />
* Once logged in:<br><br />
<nowiki><br />
$ sudo add-apt-repository ppa:opencompute-developers/ocp-certification-tools-ppa<br />
$ sudo add-apt-repository ppa:hardware-certification/public<br />
$ sudo apt-get update<br />
$ sudo apt-get install opencompute-ready<br />
</nowiki><br />
And that will install all the necessary packages.<br />
<br />
=== Configuring and running the Tool ===<br />
<br />
Edit the file /etc/xdg/opencompute-testing.conf<br />
<br />
Uncomment the following four lines:<br />
<br />
#TEST_TARGET_FTP = your-ftp-server.example.com<br />
#TEST_USER = anonymous<br />
#TEST_PASS = password<br />
#TEST_TARGET_IPERF = your-iperf-server.example.com<br />
<br />
<br />
Edit the TEST_TARGET_IPERF line replacing '''your-iperf-server.example.com''' with the IP address (Or FQDN if applicable) of your iperf server<br />
<br />
Now simply issue the command:<br />
<br />
$ opencompute-ready<br />
<br />
and follow the prompts.<br />
<br />
=== Obtaining the Source Code ===<br />
The tools are fully open source under a GPL v3 license and freely available for download here:<br />
<br />
http://code.launchpad.net/opencompute/opencompute-testing<br />
<br />
OR via bazaar:<br />
<br />
bzr branch lp:opencompute/opencompute-testing<br />
<br />
<br />
<br />
== OCP Ready Thumbdrive ==<br />
<br />
The files for the thumbdrive will be available here:<br />
http://people.canonical.com/~jlane/ocp-thumb-drive<br />
<br />
Note that this is a rather large download (> 1GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.<br />
<br />
Additionally, the OCP Ready Thumb Drive is based on old code and is not really considered reliable. It's a nice way to demo and explore, but not really something you should be using to pre-test your hardware.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=695C&I WIki Portal/OCP Checkbox2014-09-02T15:29:59Z<p>Jlane: /* Questions? */</p>
<hr />
<div>==OCP Checkbox==<br />
<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc).<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time. <br />
<br />
Checkbox (and it's next generation) is an extensible test harness that is capable of running tests that are in pretty much any language, interpreted or compiled, and even running external test suites like Phoronix, NGINX, etc.<br />
<br />
===About the Tools===<br />
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.<br />
<br />
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:<br />
;plainbox-provider-opencompute-certification : Provides the whitelists and test scripts, job definitions and other data for testing<br />
;checkbox-ng : The underlying test harness based on Plainbox<br />
<br />
This will change slightly as development progresses.<br />
<br />
Developer level documentation can be found here:<br />
;[http://checkbox.readthedocs.org CheckBox-NG]<br />
;[http://plainbox.readthedocs.org PlainBox]<br />
<br />
Keep in mind that the documentation is developer focused, not user focused, and is also actively developed and thus could change in the future.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
We hang out on Freenode IRC in the #opencompute-ci channel<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
Coding can be anything from fixing bugs (if any) to assisting with the plainbox port to writing new test scripts and jobs. The latter is most important as we want to ensure that the OCP Ready and OCP Certified specs are thoroughly tested and test cases are represented in the test tools. <br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
Currently, there is limited documentation available, and this is more akin to generic documentation and development documentation. We could really use user guides from an OCP point of view that explains how to perform testing, how to install the tools, how to USE the tools as well as documentation on how to write test cases and mapping existing test cases to OCP Ready and OCP Certified test cases in the specs.<br />
<br />
Additionally, documentation on building alternate packaging (RPM) and installing/running the tools on OSs other than Ubuntu Server would be useful.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases. It is packaged in .deb only so any additional formats will need to be created (.rpm, for instance).<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.<br />
<br />
Feel free to ping Jeff Lane (bladernr, bladernr_ on freenode) in #ocp-ci or via [mailto:jeff@ubuntu.com E-Mail]</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=694C&I WIki Portal/OCP Checkbox2014-09-02T15:28:18Z<p>Jlane: /* How to Participate */</p>
<hr />
<div>==OCP Checkbox==<br />
<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc).<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time. <br />
<br />
Checkbox (and it's next generation) is an extensible test harness that is capable of running tests that are in pretty much any language, interpreted or compiled, and even running external test suites like Phoronix, NGINX, etc.<br />
<br />
===About the Tools===<br />
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.<br />
<br />
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:<br />
;plainbox-provider-opencompute-certification : Provides the whitelists and test scripts, job definitions and other data for testing<br />
;checkbox-ng : The underlying test harness based on Plainbox<br />
<br />
This will change slightly as development progresses.<br />
<br />
Developer level documentation can be found here:<br />
;[http://checkbox.readthedocs.org CheckBox-NG]<br />
;[http://plainbox.readthedocs.org PlainBox]<br />
<br />
Keep in mind that the documentation is developer focused, not user focused, and is also actively developed and thus could change in the future.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
We hang out on Freenode IRC in the #opencompute-ci channel<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
Coding can be anything from fixing bugs (if any) to assisting with the plainbox port to writing new test scripts and jobs. The latter is most important as we want to ensure that the OCP Ready and OCP Certified specs are thoroughly tested and test cases are represented in the test tools. <br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
Currently, there is limited documentation available, and this is more akin to generic documentation and development documentation. We could really use user guides from an OCP point of view that explains how to perform testing, how to install the tools, how to USE the tools as well as documentation on how to write test cases and mapping existing test cases to OCP Ready and OCP Certified test cases in the specs.<br />
<br />
Additionally, documentation on building alternate packaging (RPM) and installing/running the tools on OSs other than Ubuntu Server would be useful.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases. It is packaged in .deb only so any additional formats will need to be created (.rpm, for instance).<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=693C&I WIki Portal/OCP Checkbox2014-09-02T15:15:15Z<p>Jlane: /* About the Tools */</p>
<hr />
<div>==OCP Checkbox==<br />
<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc).<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time. <br />
<br />
Checkbox (and it's next generation) is an extensible test harness that is capable of running tests that are in pretty much any language, interpreted or compiled, and even running external test suites like Phoronix, NGINX, etc.<br />
<br />
===About the Tools===<br />
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.<br />
<br />
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:<br />
;plainbox-provider-opencompute-certification : Provides the whitelists and test scripts, job definitions and other data for testing<br />
;checkbox-ng : The underlying test harness based on Plainbox<br />
<br />
This will change slightly as development progresses.<br />
<br />
Developer level documentation can be found here:<br />
;[http://checkbox.readthedocs.org CheckBox-NG]<br />
;[http://plainbox.readthedocs.org PlainBox]<br />
<br />
Keep in mind that the documentation is developer focused, not user focused, and is also actively developed and thus could change in the future.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases.<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=692C&I WIki Portal/OCP Checkbox2014-09-02T15:10:37Z<p>Jlane: /* Origins */</p>
<hr />
<div>==OCP Checkbox==<br />
<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc).<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time. <br />
<br />
Checkbox (and it's next generation) is an extensible test harness that is capable of running tests that are in pretty much any language, interpreted or compiled, and even running external test suites like Phoronix, NGINX, etc.<br />
<br />
===About the Tools===<br />
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.<br />
<br />
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:<br />
;plainbox-provider-opencompute-certification : Provides the whitelists and test scripts, job definitions and other data for testing<br />
;checkbox-ng : The underlying test harness based on Plainbox<br />
<br />
This will change slightly as development progresses.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases.<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=691C&I WIki Portal/OCP Checkbox2014-09-02T15:08:47Z<p>Jlane: </p>
<hr />
<div>==OCP Checkbox==<br />
<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc).<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time. <br />
<br />
At this time, the upstream checkbox code is in maintenance only mode and no new enhancements or features will be added to the code base. Upstream, the background code for the test tools is being replaced with a new tool referred to as Plainbox. Plainbox provides a cleaner interface and several improvements over the older Checkbox code. Work is actively under way to port OCP test tools to the new Plainbox back end.<br />
<br />
===About the Tools===<br />
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.<br />
<br />
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:<br />
;plainbox-provider-opencompute-certification : Provides the whitelists and test scripts, job definitions and other data for testing<br />
;checkbox-ng : The underlying test harness based on Plainbox<br />
<br />
This will change slightly as development progresses.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases.<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=600C&I WIki Portal/OCP Checkbox2014-07-09T21:24:20Z<p>Jlane: </p>
<hr />
<div>==OCP Checkbox==<br />
<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc).<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time.<br />
<br />
===About the Tools===<br />
As of Ubuntu 14.04 LTS, Checkbox has been deprecated in favor of a new generation test harness called Plainbox. Because of this, the last version of OCP Checkbox is 1.17.4. Going forward, this code is in Maintenance/Bug Fix only mode and will not receive any major updates. OCP Checkbox is still perfectly valid for testing use but will be replaced in the future by a plainbox based test tool. The difference to the tester will be minor, though for the developer, the difference will be great.<br />
<br />
The development focus going forward is on the new Plainbox based test tool also stored on Launchpad. There are currently two packages necessary:<br />
;plainbox-provider-opencompute-certification : Provides the whitelists and test scripts, job definitions and other data for testing<br />
;checkbox-ng : The underlying test harness based on Plainbox<br />
<br />
This will change slightly as development progresses.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases.<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_WIki_Portal/OCP_Checkbox&diff=599C&I WIki Portal/OCP Checkbox2014-07-09T21:16:51Z<p>Jlane: Created page with "==OCP Checkbox== Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP c..."</p>
<hr />
<div>==OCP Checkbox==<br />
<br />
Checkbox is a test harness used to execute test cases for OCP Ready and OCP Certification testing. It's a command line driven tool written in Python. OCP checkbox is capable of executing test scripts written in a variety of languages (Python, Bash, Perl, C, etc).<br />
<br />
===Origins===<br />
Checkbox began as a script called hwtest and was written by people at Canonical. As the initial script grew into a more functional test harness, it was renamed "Checkbox" and has been the testing tool used for Ubuntu certification for many years. The upstream checkbox project is fully open sourced as is the OCP branch of Checkbox. Anyone can contribute code to the tool at any time.<br />
<br />
===How to Participate===<br />
There are a number of ways you can participate in making this tool better for OCP.<br />
<br />
====Join the dev team====<br />
There is a team set up on Launchpad for people who wish to contribute:<br />
:https://launchpad.net/~opencompute-developers<br />
<br />
====Code!====<br />
You can always submit code to the project.<br />
:The source code can be found here: https://code.launchpad.net/~opencompute-developers/opencompute/checkbox<br />
:It is stored on launchpad and uses bzr as it's control system: http://bazaar.canonical.com/en/<br />
<br />
====Docs====<br />
We can always use new documentation, be it user guides, development guides, wiki pages, etc.<br />
<br />
====Testing====<br />
Feel free to install and run Checkbox. It runs well on Ubuntu and should run on Debian. It will also run on CentOS, RHEL and other Linuxes, but the test scripts themselves will need porting as they rely on Ubuntuisms in some cases.<br />
<br />
====Bug Reporting====<br />
Please feel free to file bugs on problems encountered or requests for enhancements: <br />
:https://bugs.launchpad.net/opencompute/checkbox<br />
<br />
===Questions?===<br />
Primarily, we hang out on the opencompute-ci mailing list and IRC Channel #ocp-ci on Freenode.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=598C&I Wiki Portal/OCP Ready2014-07-09T20:51:52Z<p>Jlane: /* OCP Ready Thumbdrive */</p>
<hr />
<div>== OCP Ready Thumbdrive ==<br />
<br />
The files for the thumbdrive will be available here:<br />
http://people.canonical.com/~jlane/ocp-thumb-drive<br />
<br />
Note that this is a rather large download (> 1GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=597C&I Wiki Portal/OCP Ready2014-07-09T20:36:32Z<p>Jlane: /* OCP Ready Thumbdrive */</p>
<hr />
<div>== OCP Ready Thumbdrive ==<br />
<br />
The files for the thumbdrive will be available here:<br />
http://people.canonical.com/~jlane/ocp-thumb-drive<br />
<br />
Note that this is a rather large download (1.5GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal&diff=596C&I Wiki Portal2014-07-09T20:35:21Z<p>Jlane: </p>
<hr />
<div>==WELCOME==<br />
<br />
:Welcome to the OCP '''Compliance & Interoperability''' Project.<br />
<br />
:This Project is open to the public and we welcome all those who would like to be involved.<br />
<br />
:[[C&I FAQ]]: Am I in the right place and other questions.<br />
<br />
==Get Involved==<br />
<br />
===Events=== <br />
<br />
:This project meets annually at the OCP Summit events and periodically at regional OCP events. More information on these events and other OCP activities can be found on the OCP events page at: http://www.opencompute.org/community/events/<br />
<br />
===Tools===<br />
<br />
:[[C&I_Wiki_Portal/OCP_Ready|OCP Ready]]<br />
:[[C&I_WIki_Portal/OCP_Checkbox|OCP Checkbox]]<br />
<br />
== Monthly Project Meetings (online)==<br />
<br />
The first (1st) and third (3rd) Wednesday 7am Taiwan.<br />
<br />
To find out the time of this meeting in your timezone go to: <br />
http://www.timeanddate.com/worldclock/converter.html<br />
<br />
This call is open to the public. <br />
<br />
===How to join the call===<br />
* Meeting URL http://fuze.me/22568267<br />
<br />
* Toll / Intl #: +1 (201) 479-4595<br />
* Toll free #: +1 (855) 346-3893<br />
* Meeting number: 23228335<br />
<br />
'''Please Note:''' Just a reminder that this call will be recorded and shared in the meeting notes.<br />
--------------<br />
<br />
===Past Meetings===<br />
<br />
:* List of meeting notes/minutes can be found here:[[C%26I_Meeting_Notes|Meeting Notes]]<br />
<br />
===Communication===<br />
<br />
:* Mailing List - opencompute-ci@lists.opencompute.org <br />
:* Subscribe to mailing list - http://lists.opencompute.org/mailman/listinfo/opencompute-ci<br />
:* IRC Channel - #ocp-ci on irc.freenode.ne<br />
:* Website - http://www.opencompute.org/projects/compliance-and-interoperability/<br />
<br />
<br />
===Participating===<br />
:A list of active projects at C&I. We welcome you to join us and contribute. [[C&I Project Tracks & Contact List]]<br />
<br />
==Contacts==<br />
<br />
===Chair===<br />
:[mailto:yf.juan@itri.org.tw YF Juan], ITRI<br />
:: Voice Mail: +1 650.257.0904<br />
<br />
===Vice Chairs===<br />
:* (Community) [mailto:DaWane.Wanek@AVNET.COM DaWane Wanek], Avnet<br />
:* (Certification) [mailto:itria20348@itri.org.tw Samantha Lee], ITRI<br />
<br />
===OCP Foundation Liaison===<br />
:* [mailto:hugh@opencompute.org Hugh Blemings], Director of Certification<br />
:* [mailto:thaoannguyen@fb.com Thao Nguyen], Director of Certification Labs<br />
<br />
<br />
----<br />
===C&I Project Chairs Emeritus===<br />
:* Matthew Liste, Goldman Sachs<br />
:* Eric Wells, Fidelity<br />
:* DaWane Wanek, Avnet</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=570C&I Wiki Portal/OCP Ready2014-06-27T16:42:27Z<p>Jlane: /* OCP Ready Thumbdrive */</p>
<hr />
<div>== OCP Ready Thumbdrive ==<br />
<br />
The files for the thumbdrive are available on github:<br />
git clone https://github.com/opencomputeproject/OCP-Ready.git<br />
<br />
Note that this is a rather large download (1.5GB) and will take some time to pull all the files.<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.</div>Jlanehttps://www.opencompute.org/w/index.php?title=C%26I_Wiki_Portal/OCP_Ready&diff=569C&I Wiki Portal/OCP Ready2014-06-27T16:41:23Z<p>Jlane: Created page with "== OCP Ready Thumbdrive == The files for the thumbdrive are available on github: git clone https://github.com/opencomputeproject/OCP-Ready.git The '''README.thumbdrive''' f..."</p>
<hr />
<div>== OCP Ready Thumbdrive ==<br />
<br />
The files for the thumbdrive are available on github:<br />
git clone https://github.com/opencomputeproject/OCP-Ready.git<br />
<br />
The '''README.thumbdrive''' file includes instructions on building and using the OCP Ready Thumbdrive.<br />
<br />
There is also a /OCP directory that includes some OCP Ready related documentation.</div>Jlane