Packages/A

STATUS: DISCUSSION

This is a stub. Please fill me in.

Regarding layout, metadata, management, catalogs, sources, loader extensions.

Design goals

• The Modules/SecurableModules specification must be supported.

• The pattern of installing modules on a PATH-like structure, where there exists a knowledgeable administrator maintaining that PATH, must be supported.

• Modules may be installed by existing tools, including those provided by operating system vendors, like apt-get.

• Running code must be able to cause modules hitherto unknown to the system to be installed from the network and loaded on the fly.

• Any namespace of modules must avoid name collisions even in the cases of (a) completely independent authors who have no pre-arranged cooperation; or (b) malicious authors who wish to mount a "substitution attack" by providing faulty modules under a previously used name.

• Running code must be able to load, into the same CommonJS sandbox, more than one "version" of the "same" module, for reasonable definitions of "version" and "same". In other words, if CommonJS bakes in a module versioning scheme, it must not require that only one version be running in the same JavaScript VM, or even in the same sandbox.

• It is preferable for CommonJS to not have a baked-in versioning scheme.

• It must be possible to support CommonJS clients who have limited bandwidth or storage space.

• The common case of module inclusion must be implementable synchronously; it must not require callbacks or promises.

Definition of a package

A package is a container holding a directory structure of JavaScript modules, where each module is a contiguous piece of program text satisfying the ECMAScript Program production. In the following diagram, we show an example of a package and introduce our graphical notation.

The diagram shows one module, hashSet, require-ing another, hashMap, from the same package. To require a module from the same package, a module specifies its relative path within the package directory structure, as described under Module Identifiers in Modules/SecurableModules.

Package files

A package file is the interoperable exchange format for packages. Each package file is an archive (ZIP or TGZ) of a directory structure. For example, the following shows a typical, small package file:

Note the manifest.json file at the top (the contents of which we will discuss later), and the lib directory which contains a singly rooted hierarchy of CommonJS modules, each of which is a JavaScript file with an extension *.js.

The default package

A CommonJS installation is endowed by the system administrator or other out-of-band container with a repository of modules organized in a logical single-rooted hierarchy, similarly to a PYTHONPATH. These modules are part of the so-called default package.

The default package is a local, logical construct and is not represented via an interoperable package file.

A CommonJS installation may choose to run various tasks (e.g., command line shells) as though they were each a module in the default package.

The modules in the default package are populated by initial installation of a CommonJS system; by operating system tools (such as apt-get); and by direct administrator involvement (such as editing environment variables and adding directories in the right places). The implementation of the default package, and the ways in which it is maintained, is outside the scope of CommonJS. However, CommonJS may specify some particular modules, with standardized paths, as minimal CommonJS requirements that must exist in the default package.

In the following example, the default package is implemented as an ordered list of root directories containing JavaScript files. Imagine that /usr/local/commonjs/lib/ was installed using apt-get, and /usr/local/commonjs/site-lib/ contains local edits made by the Unix user. Note how standard PATH-like behavior, where the local strHash.js overrides the system-provided one, can be implemented easily.

Package descriptors

A package descriptor is a set of instructions to the CommonJS runtime identifying a specific package file required by a client. It answers the questions:

• Where can the CommonJS runtime find the package file; and

• How can the runtime verify that it has fetched the correct one?

The following describes a package available as a zipfile in two different locations, and asks that its checksum be verified:

 {
   locate: [
     'http://allmysoftware.com/archive/bar/bar_2.4.8.zip',
     'http://barsoft.com/distros/bar_2.4.8.zip'
   ],
   verify: [
     {
       checksum: '24f7c8cbf635f4c03bad2dde7bf07531',
       algorithm: 'md5'
     }
   ]
 }

and the following provides a public key fingerprint, thus asking the CommonJS runtime to verify that the retrieved content has been signed by the corresponding private key:

 {
   locate: [
     'http://foosoft.com/public/latest/foo.zip'
   ],
   verify: [
     {
       signature: 'c1:b1:30:29:d7:b8:de:6c:97:77:10:d7:46:41:63:87'
       algorithm: 'rsa-sha1'
     }
   ]
 }

Issuing require from the default package

In the following example, a module in the default package require.async()s a module specifying a module descriptor as the first argument. The local CommonJS runtime may or may not already have a copy of fontUtils.zip, but the application writer does not have to worry about that. The call is nonblocking, via require.async(), so the CommonJS runtime can issue a remote request for fontUtils.zip and satisfy the request whenever the data is downloaded and the signature verified. Also, the runtime may report any failures as part of the result of require.async(), and windowSystem.js can arrange to react to these failures.

In general, any requires from a module in the default package to a module in another package must specify the target package descriptor in this manner, and be asynchronous (thus using require.async).

Issuing require targeting the default package

A module in any package can require a module from the default package using the 2-argument form of require where the first argument is the string 'default'. Access to the default package may or may not be permitted by the local CommonJS implementation but, if it is, this require will succeed:

It's important to note that no "search path" exists between packages. Therefore, if the module in fontUtils.zip failed to specify the package 'default', it would get nothing:

General require between packages

In the most common case where a module to requires a module from another package:

• The client package includes, in its manifest.json, a descriptor for the target package, and assigns that descriptor to a local alias; and

• The client module mentions that alias in the 2-argument form of the require function.

In the following example, package fontUtils.zip assigns an alias 'bez' to some foreign package, and a module then uses it in require.

Note that a synchronous require is possible here, since the CommonJS runtime has a place to stand to introspect on fontUtils.zip, prefetch its dependencies, and only run the courier.js module of fontUtils.zip when it has fetched the dependency curve.js.

Additional notes

Discussion

• modules packaging