IO/B/Stream/Level0

This proposal defines the raw io interface portion of IO/B.

Please also see Binary/C and IO/B/Stream/Level1.

Preamble

IO/B's streams are inspired by the InputStream/OutputStreams and Reader/Writers inside of Java's io system in which most classes only require you overload a method or two to provide the functionality for the rest of the methods in the class.

IO/B/Stream defines the API for generic stream classes. This spec defines a raw io JS API which may reside on Object, be it a literal or a subclass. JavaScript side is may be simpler to just use an object Literal, but there is nothing stoping JavaScript or the underlying host language (C/C++/Java depending on the interpreter) from instead defining a new class type to handle raw IO for a specific type of data source.

Every method in this api is optional, except for the contentConstructor which is required. Some operations are broken up into complementary methods. The Stream API MUST work arround missing methods by using the complementary method as a fallback. (For example, if readInto is implemented but not readChunk then to read just a chunk the Stream API must use readInto on a buffer then extract the chunk from there). If all methods from a set are missing then that signifies that the stream does not support a feature (ie: if readChunk and readInto are both missing, then read is not supported)

The stream api makes no assumptions or requirements about the underlying data source of the stream. It may or may not be read buffered, and may or may not be write buffered, but SHOULD implement .flush to flush data if it is buffered. It may or may not use resources that can be closed but should use .close to clean up after itself.

The API

s.contentConstructor -> Function;

contentConstructor MUST return either Blob or String to indicate the underlying type of the stream content.

s.read(Buffer buf, uint offset, uint max) -> uint;

Read data from the stream into a buffer.

• .read may only return 0 when EOF has been reached.
• offset denotes an offset within buf to start inserting at.
• max is measured in the same units as the type of sequence specified by .contentConstructor;
• .skip returns the number of units skipped in the skip. This number must be greater than 0.
• This SHOULD attempt to memcopy from data right into the buffer.

s.skip(uint max);

Skip over a length of data in the stream as if it were read.

• APIs using this MUST not call .skip with anything other than a max integer greater than (NOT equal to) 0.
• .skip may only return 0 when EOF has been reached.
• max is measured in the same units as the type of sequence specified by .contentConstructor;
• .skip does partial skips behaving similarly to .read;
• .skip returns the number of units skipped in the skip. This number must be greater than 0.

s.write(Sequence|Buffer|OpaqueRange data) -> uint;

Write data from a sequence or buffer into a stream. Returns the amount of data from the data that was written.

• This SHOULD attempt to memcopy from data from the data source.

s.flush();

Flushes any write buffered data in this layer or any underlying data layer forcing it to be written.

s.tell() -> OpaqueCookie|uint;

Return the current location within the stream.

• If this returns a number it MUST be an integer indicating the location within the stream measured in the same units as the type of sequence specified by .contentConstructor;
• An OpaqueCookie MUST contain enough information for .seek to later return to this absolute location.
  • This means that for a text layer backed by a binary position your OpaqueCookie MUST contain at least both the binary index AND the text index.
• An OpaqueCookie returned by this function is only valid for this stream.
• An OpaqueCookie returned by this function is invalid if a write has been made to a position in the stream before the location specified by this OpaqueCookie.

s.seek(OpaqueCookie|uint to) -> uint;

Seeks to another location within the file.

• This method MUST accept any valid number or opaque cookie outputted by .tell on the same stream.
• If an OpaqueCookie is used as input and a write was made to a location before the location specified by that OpaqueCookie in between the time .tell was called and .seek is called behaviour is not defined and will likely break.
• .seek MUST return the new location in the stream measured int the same units as the type of sequence specified by .contentConstructor;
• If .seek and .tell make use of OpaqueCookies rather than numbers .seek MUST as a special case accept the number 0 in order to rewind to the beginning of the stream.

s.close();

Closes the stream. If this stream object uses any resources .close MUST be implemented to close or destroy those resources.

Implementations SHOULD try to call .close if the raw stream class is about to be destroyed.

Rationales

• Data sources may not always be binary (such as a stream that uses a string, string buffer, or other string api) so this api is written to work abstractly over either binary or textual data.
• Data may not be stored in the same sequence type as it outputs (such as a text stream reading from a file which is binary), which do not have a 1 to 1 correspondence for this reason we use an OpaqueCookie sometimes so we can have an absolute location in both units at the same time.
  • We do NOT specify this as OpaqueCookie on text mode because it's possible to have textual backings that are fine with working with character based indexes.

ToDo

• Identifying when read/write/etc... will block.
• Do locks fit at this level?
• truncate