Unit Testing/1.0

STATUS: APPROVED BY DISCUSSION PARTICIPANTS

Pardicipants

Tom Robinson, Ash Berlin, George Moschovitis, Chris Zumbrunn, Hannes Wallnöfer, Kris Kowal, Kevin Dangoor, Ates Goral, and Davey Waterson (Discussion)

Implementations

[[Implementations/Flusspferd|]], [[Implementations/Narwhal|]] (0.2), [[Implementations/SproutCore|]], [[Implementations/node.js|]], [[Implementations/TeaJS|]], [[Implementations/Wakanda|]], [[Implementations/RingoJS|]], [[Implementations/common-node|]]

Specification

1.0

Assert

1. The assert module provides functions that throw AssertionError's when particular conditions are not met. The assert module must conform to the following interface.

var assert = require("assert");

2. The AssertionError is defined in assert.

new assert.AssertionError({message: message, actual: actual, expected: expected})
assert.AssertionError instanceof Error

At present only the three keys mentioned above are used and understood by the spec. Implementations or sub modules can pass other keys to the AssertionError's constructor - they will be ignored.

3. All of the following functions must throw an AssertionError when a corresponding condition is not met, with a message that may be undefined if not provided. All assertion methods provide both the actual and expected values to the assertion error for display purposes.

4. Pure assertion tests whether a value is truthy, as determined by !!guard.

assert.ok(guard, message_opt);

This statement is equivalent to assert.equal(guard, true, message_opt);. To test strictly for the value true, use assert.strictEqual(guard, true, message_opt);.

5. The equality assertion tests shallow, coercive equality with ==.

assert.equal(actual, expected, message_opt);

6. The non-equality assertion tests for whether two objects are not equal with !=

assert.notEqual(actual, expected, message_opt);

7. The equivalence assertion tests a deep equality relation.

• 7.1. All identical values are equivalent, as determined by ===.
• 7.2. If the expected value is a Date object, the actual value is equivalent if it is also a Date object that refers to the same time.
• 7.3. Other pairs that do not both pass typeof value == "object", equivalence is determined by ==.
• 7.4. For all other Object pairs, including Array objects, equivalence is determined by having the same number of owned properties (as verified with Object.prototype.hasOwnProperty.call), the same set of keys (although not necessarily the same order), equivalent values for every corresponding key, and an identical "prototype" property. Note: this accounts for both named and indexed properties on Arrays.

assert.deepEqual(actual, expected, message_opt);

8. The non-equivalence assertion tests for any deep inequality.

assert.notDeepEqual(actual, expected, message_opt);

9. The strict equality assertion tests strict equality, as determined by ===.

assert.strictEqual(actual, expected, message_opt);

10. The strict non-equality assertion tests for strict inequality, as determined by !==.

assert.notStrictEqual(actual, expected, message_opt);

11. Expected to throw an error:

assert.throws(block, Error_opt, message_opt);

Test

12. The "test" module provides a "run" method that runs unit tests and catalogs their results. The "run" method must return the total number of failures, suitable for use as a process exit status code. The idiom for a self-running test module program would be:

if (module === require.main) 
    require("test").run(exports);

13. run must accept any Object, usually a unit test module's exports. "run" will scan the object for all functions and object properties that have names that begin with but are not equal to "test", and other properties for specific flags. Sub-objects with names that start with but are not equal to "test" will be run as sub-tests.

A future version of this specification may add a mechanism for changing the mode of test functions from fail-fast (where failed assertions throw) to fail-slow (where failed assertions just log/print) via a logger argument or similar, but in the interim, this is implementation specific.

// ./sub-test
exports.testPatience = function () {
    require('os').sleep(1000);
};

// ./test
exports.testSubTest = require("./sub-test");

14. Test names may be any String that begins with "test", not necessarily respecting a case convention.

[
    [{"a": 10}, {"a": 10}],
    [[1, 2, 3], [1, 2, 3]]
].forEach(function (pair) {
     exports['test ' + pair[0].toSource()] = function () {
         assert.equal.apply(assert, pair);
     };
});
 
exports["test behaviour X"] = function() {
    behaviour.X()
}

Notes

Custom Assert Modules

The assertions defined above will not be to everyone's taste style wise (or infact behaviour wise.) Authors are free to define their own assertion methods in new modules as desired. To make it possible to combine assertions from different modules in one test suite, all assert methods must throw a AssertionError (or something that is an instanceof AssertionError).

Below is an example of how a custom assert module could be defined:

// file: my-check.js
exports.check = function(actual) {
    return {
        is: function(expect, message) {
            if (actual != expect)
                throw new AssertionError({actual: actual, expected: expected, message: message});
        },
    }
}

// This would be used as follows:
var check = require("my-check").check;
exports.testSomething = function() {
    var sq = Math.pow(3, 2);
    check(sq).is(9, "3 squared is 9");
}

If future versions of this spec add a logger mechanism (with the intent of making it so that a failed assertion doesn't abort the above case) then the implementation above for check.is will need to be updated.

Implementations

1. Narwhal, implemented by Zachary Carter. http://github.com/280north/narwhal/blob/master/lib/assert.js and http://github.com/280north/narwhal/blob/master/lib/test.js
2. Flusspferd, based largely on the Narwhal version.
3. SproutCore CoreTest - for the browser http://github.com/sproutit/core_test
4. v8cgi (from r766)
5. Wakanda

Unit Tests

1. http://github.com/commonjs/commonjs/tree/master/tests/unit-testing/1.0/

Show of Hands

Unit_Testing/A/Voting

Relevant Discussions

• Unit Testing 1.0 Candidate
• Recount on Unit Testing API (was: Vote: use QUnit API or Unit Testing/A)
• Vote: use QUnit API or Unit Testing/A
• Revisiting "include" (was: QUnit and CommonJS)
• QUnit and CommonJS
• Unit Testing A Show of Hands III
• Unit Testing Show of Hands II
• Unit Testing, Testing for Missing "var"s.
• Unit Testing Show of Hands