dtf is a test runner, that coordinates consistency test specification and definition, to facilitate easy, regular and automated testing of documentation and other similar resources.
dtf does not run Python unittest or coordinate with existing test frameworks, and is designed to ease the burden of manual consistency editing and checking, and help increase the quality of tests without increasing the workload for editors and documentation maintainers. See Use dtf for Documentation Consistency Testing for more information.
These options control the primary behavior and output of dtf, but do not effect how or which tests run.
Enables verbose output, causing dtf to return all output from operations including successful test validation and successful fest cases.
By default, dtf only returns output for tests that fail, and suppresses all passing output.
Turns all test and validation errors into exceptions, and dtf will stop at the first test failure. By default, dtf prints all test and validation errors and continues until it hits an invalid test definition or completes all tests.
Implementation of the testing response is up to the specific test and its implementation. Tests may not be able to automatically generate passing output, and these tests will ignore this option.
These options allow you to specify paths for case definitions and test specifications to run suites of tests. You can specify these options more than once to run multiple groups of tests in a single dtf invocation.
These are the Python modules that define the parameters and procedure that the tests enforce. Arguments should be paths relative to current working directory.
Defaults to cases/.
These are the YAML test definitions that specify a case and a condition that dtf will enforce.
Defaults to tests/.
The “single mode” allows dtf to load and run a single test at a time. It exists for troubleshooting and development purpose to isolate a single test without needing to run the full suite.
Furthermore, with single mode, you can delegate running the test suite to another build tool, depending on the requirements of your build and test infrastructure.
Enable “single” mode, to run a single test.
Defaults to false.
Specify the full path to a YAML test specification.
Specify the full path to the Python module that implements the case definition.
In most cases, running suites of tests serially will be more efficient than running tests in parallel given the overhead of the parallelism strategies and the typical workloads of these test suites. Additionally, for very large test suites you may find similar or better results and better control by splitting the test suite into parts and running each suite concurrently using an external tool.
Nevertheless, dtf provides the following approaches to parallel test operations:
Specify thread, process or event. Controls the mechanism of parallelism.
Specify the size of the thread or worker pool. Defaults to 2. Should not typically exceed the number of processors, though thread pools and especially event worker pools may be slightly larger as needed.