Modules/Metadata

STATUS: PROPOSALS

Overview

Packages/1.0 specifies a JSON format for describing packages, this page proposes that we extend that benefit to modules, for the purposes and goals of this proposal see: Modules/Metadata/Purpose.

Format

Module metadata should be stored in JSON as package metadata is, since JSON is built right into JavaScript.

{
 
    // This manually specifies how this metadata will be handled.
    // See the "handlers section"
    "handler": "http://github.com/jhuni/Module-Metadata/blob/master/lib/META/Handlers/bundler.js",
 
    // Extends is basically a way of combining JSON files.
    "extends": "json-Modules-1.1.1-common",
 
    // This uses the format:
    // language, name, version, authority
    "id": "js-Utils.slurp-1.0-jsan+jhuni",
 
    // This makes the module return exports like in Modules/1.0.
    "returns": "exports",
 
    // This will pass the variables io and each to your module.
    "use": {
 
        "each": "js-Utils.each-1.0-jsan+jhuni",
 
        // This is for rhino users:
        "io.java": "java-java.io-1.4-sun",
 
        // This is for parrot users:
        "io.perl": "perl-File::Slurp-1.2-cpan+DROLSKY"
 
    },
 
    // This will make sure XMLHttpRequest works in IE.
    "builtins": [
        "XMLHttpRequest"
    ],
 
    // This is to aid some implementations of require()
    "requires": [
        "js-Utils.more-1.2-jsan+jhuni"
    ]
 
}

Examples

Sample Code

math.js

META({
 
    "id": "js-Utils.add-1.0-jsan+jhuni",
 
    "builtins": [
        "Array.prototype.reduce",
        "Function.prototype.call"
    ]
 
});
 
// Package
return function() {
 
    return( Array.prototype.reduce.call(arguments, function(a, b) {
        return a + b;
    }) );
 
};

increment.js

META({
 
    "id": "js-Utils.inc-1.0-jsan+jhuni",
 
    "use": {
        "add": "js-Utils.add-1.0-jsan+jhuni"
    }
 
});
 
// Package:
return function(val) {
 
    return add(val, 1);
 
};

program.js

META({ 
 
    "id": "js-Utils.program-1.0-jsan+jhuni",
 
    "use": {
        "inc": "js-Utils.inc-1.0-jsan+jhuni"
    }
 
});
 
// Package
var a = 1;
inc(a); // 2

Modules/1.1.1

META({
 
    // Specify what version of the CommonJS module system you are using
    "extends": "json-Modules-1.1.1-common",
 
    // Aid some static analysis tools by giving them shallow-deps
    "requires": [
        "inc"
    ]
 
});
 
/* Modules/1.1.1 code */

Loader

One of the advantages of this proposal is that even the loader script can have metadata, because usually loader scripts require lots of utilities, builtins, and other functionality. The yabble loader actually manually includes utilities similar to the ones below, furthermore the JSAN loader also requires lots of utilities.

This sort of practice will leave normalizing XMLHttpRequest, and other builtins to external scripts. For example there may be special normalizers for some server-side platforms which don't have XMLHttpRequest, and some old browsers such as Opera 7.0 have normalizers based upon java. [1]

loader.js

META({
 
    "id": "js-ensure-1.2-commonjs",
 
    "returns": "exports",
 
    "builtins": [
        "XMLHttpRequest", 
        "Array.prototype.indexOf",
        "Array.prototype.filter"
    ],
 
    "use": {
        "loadModuleByScript": "js-Get.script-1.0-jsan+jhuni"
    }
 
});
 
 
exports.ensure = function(modules, callback) {
    /* ... */
};

Most modules won't even need to have access to on-demand JavaScript, they just need the ability to handle dependencies, so you will rarely have to use such an ensure function.

META({
 
    "use": {
        "ensure": "js-ensure-1.2-commonjs"
    }
 
});
 
ensure(['Utils.inc', 'Utils.add'], callback);

Furthermore, when you request the above script that requires Utils.add, and Utils.inc it will just combine the ensure like any other module, this is one case where this metadata proposal will boost the performance of client-side websites.

Handlers

Overview

A handler is a function in which handles a metadata format, there may be multiple handlers which handle the same data differently, or handlers built to handle an entirely different metadata format.

Bundler

The bundler is a handler in which takes a module, removes its metadata and includes any of its dependencies to build a completely standalone file, for example the file Utils.program might look something like this if it was stand-alone:

(function() {
 
// From "use": {"add": js-Utils.add-1.0-jsan+jhuni"}
var deps1 = (function() {
 
    return function() {
 
        return( Array.prototype.reduce.call(arguments, function(a, b) {
            return a + b;
        }) );
 
    };
 
})();
 
// From "use": {"inc": "js-Utils.inc-1.0-jsan+jhuni"}
var deps2 = (function(add) {
 
    return function(val) {
        return add(val, 1);
    };
 
})(deps1);
 
// This is the package itself:
var requestedModule = (function(inc) {
 
    var a = 1;
    inc(a); // 2
 
})(deps2);
 
 
// Now lets register the requested modules
// Using some register handler:
 
if (typeof Utils === 'undefined') {
    Utils = {};    
}
 
Utils.add     = deps1;
Utils.inc     = deps2;
Utils.program = requestedModule;
 
})();

Implementation: http://github.com/jhuni/Module-Metadata/blob/master/lib/META/Handlers/bundler.js

Modules/Transport

Furthermore you can write META handlers to turn scripts into one of the Modules/Transport formats.

Transport D: http://github.com/jhuni/Module-Metadata/blob/master/lib/META/Handlers/transportD.js

Usage

Local Scripts

On the server-side where you don't suffer from serious IO costs, you can just fetch all of a modules dependencies using asynchronous slurping and do the META handling locally.

Localized metadata system: http://github.com/jhuni/Module-Metadata/blob/master/lib/META.js

External Scripts

If you are getting a script from an external location then you should be able to delegate the META handling process to the server who is delivering the script.

// Possible Example:
script.src = "http://openjsan.org/?v=Utils.program-1.0&b=Firefox-3.5.9"

Then it that case the server in which contains that script will go through the META handling process for you and it will perform combo loading and packing to optimize performance.

Package Managers

Package managers may create separate JSON files for module metadata, for example for the module /Module/Stub.js the package manager may create /data/Module/Stub.json which stores the module's metadata.