Global Lua Modules/Testharness — dev

For method-based Scribunto testing, see Global Lua Modules/ScribuntoUnit.

Testharness is a meta-module that enables modular, automated testing and test report generation for Scribunto. This ensures that a particular module functions properly.

Description

This module is a standalone variant of the deprecated UnitTests module. Rather than registering methods for every single package member in your module, all one needs to do is write a (mostly) static table.

Usage

Write your testdata

Create a local module in Module:YOURMOD/testcases with code similar to this:

-- Unit tests.  Click talk page to run tests.
return {
    hello = { -- member being tested
        options = { -- <= options 
            mode       = 'invocation', -- <= test mode
        },
        tests = {
        --    | Parameters    | Expected result
            { 'Hello World!', 'Hello World!' },
            {  'Hello|World',        'Hello' },
        }
    }
}

Test data format

The "testcases" module has a subtable for every member you are testing; for example, if you are testing p.lang in Module:SharedTest the key name would be lang.

There are two tables in every test configuration: options and tests.

options: Options to configure testing of that member. See the test options section.
tests: Array of test cases to compare in the test harness. Subheadings can be added in the member's test case table by placing strings in the array. These strings support wikitext, and are preprocessed before rendering with zero effect on testing.

Test case format

Each test case is written in the following format:

{ 'test case input', 'expected test output', { --[[ options --]] } }

The first two items are required, but the last table is optional.

The first string is the test case input, as described in the test mode section.
The second string is the expected output from the test harness.
The third option provides optional configuration:
- err - whether an error is expected in the test case.
- pp - whether the expected output should be preprocessed before comparison

Create your test report

Then create Module talk:YOURMOD/testcases with {{testunit}}, or the following invocation format:

{{#invoke:testharness|run_tests|modulename=<module>|testdata=<module with tests>|differs_at=<"differs at" inclusion - 1/0>}}

Note that the Module: isn't necessary. See the harness configuration table below.

Assessing and improving your results

The test harness generates an extensive report with the following information:

Overall test suite status and score
Rudimentary code coverage score
Execution time

The report table pinpoints the presence of any failures, errors and output differences. Use this information to improve the code quality in your Scribunto module.

Configuration

Test harness configuration

The following parameters are available in your #invoke:

Parameter	Default	Optional?	Example	Description
`modulename`	The base page name of your module	Yes	`\|modulename = Links`	Test module
`testdata`	The /testcases subpage of your module name	Yes	`\|testdata = Links/testcases`	Test data submodule
`differs_at`	true	Yes	`\|differs_at = 0`	Boolean for "Differs at" column

Test member options

Option	Default	Optional?	Description
`mode`	n/a	No	Mode of testing
`preprocess`	false	Yes	Preprocess expected text from test data.
`template`	false	Yes	Skip to parent frame when applying test case arguments to parent frame first.
`self`	false	Yes	Self-invoke the package function in 'method' tests.
`unpk`	false	Yes	Unpack the table argument in 'method' tests.
`deep`	false	Yes	Enable deep comparison for table cases in 'method' tests.
`nowiki`	false	Yes	Apply nowiki formatting to report field

Member test modes

The following test modes are available:

'method'

Tests your method as a standalone Lua function.

Expects arguments as a table or single variable: 'test' or {1,2,3}

'table'

Tests your method as a static Lua table.

Expects one argument:

a key to access the expected table value, OR
a table of keys leading to the expected table value

'invocation'

Tests your method as a Scribunto invocation.

Accepts parameters in '1|2|3|arg1=arg1|arg2=arg2' format.