JSGI/ForEachStreaming

JSGI Streaming Body Using forEach

Status: Proposed

Reference Implementation: http://github.com/kriszyp/jsgi-node

Rationale

KISS - Make it as simple as possible to return a response.

Read request input and write to the response body without buffering data without requiring extra effort on the part of the programmer.

Consistency between synchronous and asynchronous streaming of the body with backwards compatibility with JSGI 0.3.

Consistency with the API on how asynchronous actions are modeled, keeping in mind that in JSGI, response status and headers are asynchronously provided using promises.

Addition JSGI 0.3

• add url member to the request object, which is the requested URL exactly as it appears on the first line of the HTTP request.
• request.body MUST be a forEach-able Stream object.
• response.body MUST be a forEach-able Stream object.
• request.jsgi.error MUST be a forEach-able Stream object.

Stream Objects

forEach-able Stream Objects are representations of a stream of data.

Methods

forEach-able Stream Objects have the following method:

• forEach(write) - When the stream can be written to, this will be called. The first parameter, write, MUST be a function. The forEach function can call the write function to write strings to the stream. The stream will be closed when forEach returns with any value that does not have a then() function, or if it returns a then() function (a promise), then() should be called with a callback that can be called when the stream is finished. Each time the write callback is called, it may optionally also return a promise with a then() function if wishes to indicate that it has not yet completed handling processing the data. Once the data consumer is ready to continue receiving data, it call the callbacks to indicate that the forEach should continue. This allows for throttling of data.

Properties

• charset - This property can be set to the desired charset to use for encoding or decoding strings.

Examples

Simple sync app:

app = function(request){
  return {
    status: 200,
    headers: {},
    body: ["hello", "world"]
  }
};

Streaming app, this could be sync or async, as along as "data" is a forEach-able source data structure:

app = function(request){
  return {
    status: 200,
    headers: {},
    body: {
      forEach: function(write){
        return data.forEach(function(item){
          return serialize(item);
        })
      }
    }
  }
};

Reading input:

app = function(request){
  var file = new File("fileToSave");
  var fileUpload = request.body.forEach(function(data){
    file.write(data);
  }).then(file.close);

  return fileUpload.then(function(){
    return {
      status: 200,
      headers: {},
      body: ["file successfully uploaded"]
    };
  };
};

Streaming app, where the source is an event emitter:

app = function(request){
  return {
    status: 200,
    headers: {},
    body: {
      forEach: function(write){
        var promise = new Promise();
        someEventSource.addListener("data", function(data){ write(data); });
        someEventSource.addListener("finished", function(data){ promise.resolve() });
        return promise;
      }
    }
  }
};

Middleware:

MiddleWare = function(nextApp){
  return function(request){
    var response = nextApp(request);
    var inBody = response.body;
    response.body = {
      forEach: function(write){
        return inBody.forEach(function(block){
          return modify(block);
        })
      }
    }
    return response;
  }
};