IO/B/Filesystem/Level0

This proposal defines a io/file/raw module which acts as a level 0 module for filesystem access. This module is not meant for use in normal programming but instead acts as the lowest level layer between pure-JavaScript CommonJS code and native filesystem code. It is intended only for the purpose of providing a raw access layer that higher level modules may use to create file apis with.

Some comments about this proposal are on the Talk:IO/B/Filesystem/Level0

Terminology

Node

A Node refers to an object which exists or does not exist on the filesystem, this is basically a path which irrelevant parts of (like the difference between / and \ on Windows) may be cleaned up but may not necessarily be a path string.

Opaque*

"Opaque*" is a piece of terminology (started in Filesystem/A) used to refer to a value of some sort which is implementation dependent. An Opaque value is one that only has a very small and specific set of defined behaviors, other than those there is no definition of how or what the implementation should use to create them. Something opaque can range in anything from an object literal, wrapped native data, a resource as a number, and so on... Inside a spec an Opaque value is NEVER introduced with something that requires dot notation (an Opaque is never expected to have the ability to look at a property on it), the only action used with an Opaque is to pass it as an argument to a function within the same implementation which will use it.

The module

The io/file/raw module accessed by require('io/file/raw'); exports these functions:

.pathToNode(string path) -> OpaqueNode

Returns an OpaqueNode that refers to a non-mutable path on the filesystem which may be used in methods within this module.

Hint: Even if your native layer accepts path strings, rather than just passing the path string through this is a good place to clean up that path string into a format that your implementation expects. Like how NSPR expects / even on Windows, while others expect \ on Windows. You can clean up both / and \ to the separator your underlying layer is expecting of you, then return the cleaned up string.

.nodeToPath(OpaqueNode node) -> string

Returns the string path for a OpaqueNode (this may have been cleaned up in terms of separator conversion).

.nodeToName(OpaqueNode node) -> string

Returns the file name for an OpaqueNode.

.stat(OpaqueNode node) -> OpaqueStat

Returns an OpaqueStat for the OpaqueNode specified by node.

Methods that use the OpaqueStat object are not guaranteed to not change between the time you call .stat and the time you call the method that uses the OpaqueStat.

.canonical(OpaqueNode node) -> OpaqueNode

Returns a OpaqueNode refering to the canonical

.exists(OpaqueStat stat) -> boolean

Tests for existence of the node on the filesystem.

.size(OpaqueStat stat) -> uint|false

Returns the size of the file specified by stat, or returns false.

.isFile(OpaqueStat stat) -> boolean|false

.isDirectory(OpaqueStat stat) -> boolean|false

Checks the stat to see if the node it refers to is a file or is a directory.

isFile and isDirectory may both return false if the node's path does not exist on the filesystem, or is a existing path on the filesystem that refers to something that is neither file nor directory.

.lastModified(OpaqueStat stat) -> Date|false

Returns the last modified time (mtime) of the file refered to by stat as a date or returns false.

.setLastModified(OpaqueNode node, Date time) -> boolean

Sets the last modified time (mtime) of the file refered to by node.

.isReadable(OpaqueStat stat) -> boolean

.isWriteable(OpaqueStat stat) -> boolean

Returns booleans indicating if the stat refers to a file that is readable or writable.

.createDirectory(OpaqueNode node, [boolean recursive=false]) -> boolean

Create a directory at the node specified by node. If resursive is specified this should attempt to create any parent directories instead of failing because they are missing.

Returns a boolean to indicate if it created the directory or failed.

.listPaths(OpaqueNode node) -> Array*string

.listPathsByPath(OpaqueNode node, Function pathCallback) -> Array*string

.listNodes(OpaqueNode node) -> Array*OpaqueNode

.listNodesByPath(OpaqueNode node, Function pathCallback) -> Array*OpaqueNode

.listNodesByNode(OpaqueNode node, Function nodeCallback) -> Array*OpaqueNode

.listNodesByStat(OpaqueNode node, Function statCallback) -> Array*OpaqueNode

List out the files within a directory.

• These methods will throw if used on a file or nonexistent directory.
• pathCallback(OpaqueNode dir, string name);
• nodeCallback(OpaqueNode dir, OpaqueNode node);
• statCallback(OpaqueNode dir, OpaqueStat stat);

.open(OpaqueNode node, Object flags);

Opens a file in a mode specified by flags and returns a stream in binary or text mode.

• .open will throw if used on a directory.
  • encoding: A character encoding to use when opening this in text mode. If this is falsey (undefined, false, or absent) the file shall be opened in binary mode instead of text mode
  • read: True if opening for read
  • write: True if opening for write
  • append: True if all writes should be made to the end of the file
  • truncate: True if the system should truncate the file after opening
  • create: True if the system should create the file instead of failing if it doesn't exist (only applies when opening for writing)
  • sync: True if writes should synchronously wait for the os to write the data to disk

.remove(OpaqueNode node) -> boolean

Deletes the file or empty directory specified by node. Returns true or false to indicate if deletion worked or failed.

.move(OpaqueNode from, OpaqueNode to) -> boolean

Rename/moves the node specified by from to the location specified by to.

Stream

Streams returned by .open are defined by IO/B/Stream/Level0 with the following additional requirements:

• In binary mode .tell MUST return the location in bytes, not an OpaqueCookie.
• Programs using streams opened by .open in text mode MUST expect .tell to return an OpaqueCookie rather than a position in characters.
  • Actual implementations may use whichever works best for their implementation

Rationales

• Java and perhaps some other native abstractions use instances of a class or some other resource to non-mutably refer to a node on the filesystem, while others only use paths. For this reason we use an OpaqueNode which may take on an implementation dependent form. Java implementations could return a java.io.File object directly, implementations using a native layer that uses some sort of resource identifier could encapsulate that and return that as an OpaqueNode, and implementations which use paths could simply return the path string.
• Similarly posix systems usually provide a stat object, while systems like Java have separate lastModified, and such calls. For that reason we use an OpaqueStat which could be some sort of stat object, or even just return the OpaqueNode that was passed to it.
• ctime is inconsistent. Windows records creation time, but not change time of metadata. Unix records change time of metadata, but not creation time of the file. Then there are filesystems like ZFS which have separate fields that record both. Additionally Java does not provide a means to access ctime without significant work using JNI or something like JNA to access this data at C/C++ level. The majority of applications also have little need for ctime info and at most only need mtime (lastModified). For these reasons combined any sort of ctime (lastChanged/creationTime) is not specified by this platform independent filesystem access spec. That should be specified separately in something such as a posix module meant for posix specific filesystem access.
• While low level systems provide a directory open and read, Java only provides directory list* methods with Filename or File object filters. For this reason as a compromise we specify 4 separate directory listing methods, 3 of which can filter in different ways.
• All native level systems so far do not provide a copy mechanism. For this reason copy is omitted at this level and expected to be implemented at a higher level. (If there are any systems that provide a high performance copy this could be changed).
• .open at this level only specifies the raw things needed as a simple object. Any mode string parsing or more flexible object should be done in a higher level.
• Some implementations may optimize binary->text using native layering rather than js based layering so text/binary modes are included in .open at this level and a .raw binary layer is not exposed in a text layer.

ToDo

• setCurrentWorkingDirectory. Should this use a path or a OpaqueNode?
• Permission modification
• Equivalence
• comparison?
• Hidden
• Roots and filesystem space
• isExecutable
• Temp files / delete on exit