The following Python modules of Submitty are unit tested:
- Migrator (under
- Autograding (under
- python_submitty_utils (under
- submitty_jobs (under
Running Python Unit Tests
To locally run the Python unit tets, you need to
cd into their directory and then use the
unittest module to
run them. To run all the tests for a particular module:
python3 -m unittest discover
To gather code coverage of the unit tests, run:
coverage -m unittest discover
For example, to test the Migrator module:
cd migrator python3 -m unittest discover # or coverage -m unittest discover
To test a portion of a module, pass in the module name (filename), class name, and then function name depending on how specific you wish to go. For example, for the migrator:
# run whole test module python3 -m unittest tests.test_cli # run one test function python3 -m unittest tests.test_cli.TestCli.test_no_args
Submitty Daemon Jobs
Make sure the python system dependencies are locally installed, this can be done by running the following command at the root of the project structure.
python3 -m pip install -r .setup/pip/system_requirements.txt
For the submitty_daemon_jobs unit tests, you will need to set the top level directory to
sbin/submitty_daemon_jobs when running them. This can be done by the following command
#in the sbin/submitty_daemon_jobs directory python3 -m unittest discover tests -t .
Or if you’re using
#in the sbin/submitty_daemon_jobs directory coverage run -m unittest discover tests -t .
Writing Python Unit Tests
The unittest documentation includes good examples and general information for writing tests.
Getting started, the structure of the unit tests is such that the python module lives under a source directory,
and the tests are under the
tests/ directory. There should be a one-to-one correspondence between test file
and source file. For example, for migrator, there’s
tests/test_cli.py. For particularly
complex modules, it may be beneficial to break up the tests for that module into separate files focusing on one
aspect. However, this is an exception to the norm, and for most things, the one-to-one design works well.
For actually writing a test, we start with a basic example:
import unittest class TestStringMethods(unittest.TestCase): def test_upper(self): self.assertEqual('foo'.upper(), 'FOO') def test_isupper(self): self.assertTrue('FOO'.isupper()) self.assertFalse('Foo'.isupper()) def test_split(self): s = 'hello world' self.assertEqual(s.split(), ['hello', 'world']) # check that s.split fails when the separator is not a string with self.assertRaises(TypeError): s.split(2)
Here, we see that to create a
unittest module, we import it, create a class that extends
unittest.TestCase and then every
method under that class is prefixed with
test_. With each function, there is a series of
assert methods that can be used to verify behavior.
Python Unit Test Fixtures
While most unit tests can be largely stateless, there may be times you wish to have common initialization for a particular
test module. This is called a “test fixture”, which involve some preparation for running a test, and then the cleanup afterwards.
An example of where this might be useful is if each test generates a file, you could have a test fixture to create a new random
directory for that file to go into, and then a test fixture to delete that folder after the test runs.
unittest uses the
tearDown to represent that:
import os import shutil import unittest class TestModule(unittest.TestCase): def setUp(self): self.directory = os.makedirs('test') def tearDown(self): shutil.rmtree('test')
For more details, see the section on Organizing Tests from