Unit Testing/1.1

Unit Testing/1.1

Specification

This specification adds following on top of Testing/1.0:

• Support for running tests in "fail-slow" mode (where failed

assertions are logged) side by side to the already existing "fail-fast" mode (where failed assertions throw AssertionError's and fail entire test).

• Support for running asynchronous tests.

Fail-slow

• 'assert' module MUST exports constructor function Assert.
• Constructor Assert MUST return new instance of itself

when called with or without keyword new.

var assert = require('assert').Assert()

• Instances of Assert MUST conform to the following interface:
• Following logging methods must be defined on instance:
• Method pass that is called by all assertion methods on

instance if corresponding conditions are met.

assert.pass(message_opt)

If assertion method was called with an optional message argument it MUST be passed through to the pass method.

• Method fail that is called by all assertion methods on

instance if corresponding conditions are not met.

assert.fail(
{ message: message
, actual: actual
, expected: expected
})

At present only the three keys mentioned above are used and understood by the spec. Implementations or sub modules can pass other keys to the fail method (they will be ignored).

• Method error that is called by test runner if the

corresponding test throws an exception that is not an AssertionError.

assert.error(e)

Exception thrown must be passed as an argument to the method.

• Following assertion methods MUST be defined on instance:

ok, equal, notEqual, deepEqual, notDeepEqual, strictEqual, notStrictEqual, throws. They behave as the same named functions exported by assert module with one difference that they don't throw AssertionError's, instead they log results as was described by pass, fail, error methods.

• Test runner must pass a new instance of Assert as a first

argument, to each test it runs, where Assert is constructor that is defined as a property of an object passed to the run function. If test runner object does not defines that property Assert exported by 'assert' module should be used instead.

• Custom Assert constructor scoped to the test runner object

provides way for providing custom assertion methods.

var AssertBase = require('assert').Assert
var AssertDescriptor =
{ constructor: { value: Assert }
, inRange: { value: function (lower, inner, upper, message) {
    if (lower < inner && inner < upper) {
      this.fail(
      { actual: inner,
      , expected: lower + '> ' + ' < ' + upper
      , operator: "inRange"
      , message: message
      })
    } else {
      this.pass(message);
    }
  }, enumerable: true }
}
function Assert() {
  return Object.create
  ( AssertBase.apply(null, arguments)
  , AssertDescriptor
  )
}

Please note that specification does not defines arguments that Assert may take. It is implementation specific. For this reason assert constructors MUST pass all the arguments through all the arguments to the Assert.

Asynchronous tests

All the test functions that run tests asynchronously MUST expect at least two named arguments. Such a test functions will have special behavior:

• Asynchronous test functions MUST be called with a second

argument. (for simplicity we'll be referring to it as done).

• typeof done named argument passed to the asynchronous

test must be a "function".

• Test runner MUST complete asynchronous test only after

done callback is called.

• All the assertions performed for the test after it's completion

_(after done was called)_ MUST be logged as errors by calling assert.error for that test.

• All the calls to the done function for the test after it's

completion _(after done is called)_ MUST be logged as an errors by calling assert.error for that test.

• After all tests are completed, test runner MUST call

second callback argument that was passed to the run function exported by 'test' module.

require("test").run
(
  { Assert: Assert  // <- was defined in previos example
  , 'test fail slow async': function (assert, done) {
      var begin = +new Date()
      setTimeout(function () {
        var end = +new Date()
    assert.inRange(1000 - 100, end - begin, 1000 + 100, "setTimeout accurate");
        done()
      }, 1000)
    }
  }
, function(result) { 
    console.log('Passed: ', result.passes.length)
    console.log('Failed: ', result.fails.length)
    console.log('Errors: ', result.errors.length)
  }
)

Callback MUST be called with an argument that is an object with a properties: passes, fails, errors. Each property MUST be an array of first arguments passed to the corresponding logging method.

result.passes.length // -> 1
result.passes[0] // -> "setTimeout accurate"

Recommendation

There are cases where tests MAY want to disable logging. Good example of that are the tests for test runner itself. For that matter we recommend to disable all the logging for the tests whose container objects mute property is strictly equal to true.