Async Generators are currently proposed for ES7 and are at the strawman phase. This proposal builds on the async function proposal.
JavaScript programs are single-threaded and therefore must streadfastly avoid blocking on IO operations. Today web developers must deal with a steadily increasing number of push stream APIs:
Developers should be able to easily consume these push data sources, as well as compose them together to build complex concurrent programs.
ES6 introduced generator functions for producing data via iteration, and a new for...of loop for consuming data via iteration.
// data producer{1;2;3;}// data consumer{forvar x ofconsole;}
These features are ideal for progressively consuming data stored in collections or lazily-produced by computations. However they are not well-suited to consuming asynchronous streams of information, because Iteration is synchronous.
The async generator proposal attempts to solve this problem by adding symmetrical support for Observation to ES7. It would introduce asynchronous generator functions for producing data via observation, and a new for...on loop for consuming data via observation.
// data producer{1;2;3;}// data consumer{forvar x onconsole;}
The for...on loop would allow any of the web's many push data streams to be consumed using the simple and familiar loop syntax. Here's an example that returns the a stream of stock price deltas that exceed a threshold.
{var deltaoldPriceprice;forvar price onif oldPrice == nulloldPrice = price;elsedelta = Math;oldPrice = price;if delta > thresholdprice oldPrice;}// get the first price that differs from previous spike by $5.00;
An ES6 generator function differs from a normal function, in that it returns multiple values:
{1;2;}forvar num of numsconsole;
An async function (currently proposed for ES7) differs from a normal function, in that it pushes its return value asynchronously via a Promise. A value is pushed if it is delivered in the argument position, rather than in the return position.
{return ;}try// data delivered in return positionvar price = ;console;catcheconsole;// async version of getStockPrice function{return await ;};
We can view these language features in a table like so:
Sync | Async | |
---|---|---|
function | T | Promise |
function* | Iterator |
??? |
An obvious question presents itself: "What does an async generator function return?"
{forvar price on;}// What type is prices?var prices = ;
If a generator function modifies a function and causes it to return multiple values and the async modifier causes functions to push their values, an asynchronous generator function must push multiple values. What data type fits this description?
ES6 introduces the Generator interface, which is a combination of two different interfaces:
The Iterator is a data source that can return a value, an error (via throw), or a final value (value where IterationResult::done).
interface IteratorIterationResult ;type IterationResult = done: boolean value: anyinterface IterableIterator ;
The Observer is a data sink which can be pushed a value, an error (via throw()), or a final value (return()):
interface Observervoid ;void returnreturnValue;void throwerror;
These two data types mixed together forms a Generator:
interface GeneratorIterationResult ;IterationResult returnreturnValue;IterationResult throwerror;
Iteration and Observation both enable a consumer to progressively retrieve 0...N values from a producer. The only difference between Iteration and Observation is the party in control. In iteration the consumer is in control because the consumer initiates the request for a value, and the producer must synchronously respond.
In this example a consumer requests an Iterator from an Array, and progressively requests the next value until the stream is exhausted.
{// requesting an iterator from the Array, which is an Iterablevar iterator = arr@@iteratorpair;// consumer (this function)while!pair = iteratornextdoneconsole;}
This code relies on the fact that in ES6, all collections implement the Iterable interface. ES6 also added special support for...of syntax, the program above can be rewritten like this:
{forvar value of arrconsole;}
ES6 added great support for Iteration, but currently there is no equivalent of the Iterable type for Observation. How would we design such a type? By taking the dual of the Iterable type.
interface IterableGenerator @@
The dual of a type is derived by swapping the argument and return types of its methods, and taking the dual of each term. The dual of a Generator is a Generator, because it is symmetrical. The generator can both accept and return the same three types of notifications:
Therefore all that is left to do is swap the arguments and return type of the Iterator's iterator method and then we have an Observable.
interface Observablevoid @@
This interface is too simple. If iteration and observation can be thought of as long running functions, the party that is not in control needs a way to short-circuit the operation. In the case of observation, the producer is in control. As a result the consumer needs a way of terminating observation. If we use the terminology of events, we would say the consumer needs a way to unsubscribe. To allow for this, we make the following modification to the Observable interface:
interface ObservableGenerator @@
This version of the Observable interface both accepts and returns a Generator. The consumer can short-circuit observation (unsubscribe) by invoking the return() method on the Generator object returned for the Observable @@observer method. To demonstrate how this works, let's take a look at how we can adapt a common push stream API (DOM event) to an Observable.
// The decorate method accepts a generator and dynamically inherits a new generator from it// using Object.create. The new generator wraps the next, return, and throw methods,// intercepts any terminating operations, and invokes an onDone callback.// This includes calls to return, throw, or next calls that return a pair with done === true{var decoratedGenerator = Object;decoratedGenerator {var pair = generatornextv;// if generator returns done = true, invoke onDone callbackif pair && pairdone;return pair;};"throw""return";}// Convert any DOM event into an async generatorObservable {// An Observable is created by passing the defn of its observer method to its constructorreturn {var handlerdecoratedGenerator =;{decoratedGeneratornexte;};dom;return decoratedGenerator;};};// Adapt a DOM element's mousemoves to an Observablevar mouseMoves = Observable;// subscribe to Observable stream of mouse movesvar decoratedGenerator = mouseMoves@@observer{console;};// unsubscribe 2 seconds later;
Observable is the data type that a function modified by both * and async returns, because it pushes multiple values.
Sync | Async | |
---|---|---|
function | T | Promise |
function* | Iterator |
Observable |
An Observable accepts a generator and pushes it 0...N values and optionally terminates by either pushing an error or a return value. The consumer can also short-circuit by calling return() on the Generator object returned from the Observable's @@observer method.
In ES7, any collection that is Iterable should also be Observable. Here is an implementation for Array.
It's easy to adapt the web's many push stream APIs to Observable.
Observable {// An Observable is created by passing the defn of its observer method to its constructorreturn {var handlerdecoratedGenerator =;{decoratedGeneratornexte;};dom;return decoratedGenerator;};};
Observable {// An Observable is created by passing the defn of its observer method to its constructorreturn {var handlerdecoratedGenerator =;handler = decoratedGeneratornext;;return decoratedGenerator;};};Object {return Observable;};
Observable {// An Observable is created by passing the defn of its observer method to its constructorreturn {var done = falsedecoratedGenerator =;ws {decoratedGeneratornextm;};ws {done = true;decoratedGenerator;};ws {done = true;decoratedGenerator;};return decoratedGenerator;};}
Observable {return {var handledecoratedGenerator = ;handle =;return decoratedGenerator;};};
The Observable type is composable. Once the various push stream APIs have been adapted to the Observable interface, it becomes possible to build complex asynchronous applications via composition instead of state machines. Third party libraries (a la Underscore) can easily be written which allow developers to build complex asynchronous applications using a declarative API similar to that of JavaScript's Array. Examples of such methods defined for Observable are included in this repo, but are not proposed for standardization.
Let's take the following three Array methods:
123 // [2,3,4]123 // [2,3]
Now let's also imagine that Array had the following method:
123 // [2,3,3,4,4,5]
The concatMap method is a slight variation on map. The function passed to concatMap must return an Array for each value it receives. This creates a tree. Then concatMap concatenates each inner array together left-to-right and flattens the tree by one dimension.
123 // [[2,3],[3,4],[4,5]]123 // [2,3,3,4,4,5]
Note: Some may know concatMap by the name "flatMap", but I use the name concatMap deliberately and the reasons will soon become obvious.
These three methods are surprisingly versatile. Here's an example of some code that retrieves your favorite Netflix titles.
var user =genreLists:name: "Drama"titles:id: 66 name: "House of Cards" rating: 5id: 22 name: "Orange is the New Black" rating: 5// more titles snippedname: "Comedy"titles:id: 55 name: "Arrested Development" rating: 5id: 22 name: "Orange is the New Black" rating: 5// more titles snipped// more genre lists snipped// for each genreList, the map fn returns an array of all titles with// a rating of 5.0. These title arrays are then concatenated together// to create a flat list of the user's favorite titles.{return usergenreLists;}// we consume the titles and write the to the console;
Using nearly the same code, we can build a drag and drop event. Observables are streams of values that arrive over time. They can be composed using the same Array methods we used in the example above (and a few more). In this example we compose Observables together to create a mouse drag event for a DOM element.
// for each mouse down event, the map fn returns the stream// of all the mouse move events that will occur until the// next mouse up event. This creates a stream of streams,// each of which is concatenated together to form a flat// stream of all the mouse drag events there ever will be.{var mouseDowns = Observablevar documentMouseMoves = Observablevar documentMouseUps = Observable;return mouseDowns;};var image = document;documentbody;;
The fact that the Observable and Iterable interface are not strict duals is a smell. If Observation and Iteration are truly dual, the correct definition of Iterable should be this:
interface IterableGenerator ;
In fact this definition is more useful than the current ES6 definition. In iteration, the party not in control is the producer. Using the same decorator pattern, the producer can now short-circuit the iterator without waiting for the consumer to call next. All the producer must do is invoke return() on the Generator passed to it, and the consumer will be notified. Now we have achieved duality, and given the party that is not in control the ability to short-circuit. I contend that collections should implement this new Iterable contract in ES7.
Async generators can be transpiled into Async functions. A transpiler is in the works. Here's an example of the expected output.
The following code...
{var stockPriceServiceUrl = await ;// observable.buffer() -> AsyncObservable that supports backpressure by bufferingforvar name on stockNames// accessing arguments array instead of named paramater to demonstrate necessary renamingvar price = awaittopStories = ;forvar topStory ontopStories;if topStorylength === 2000break;// grab the last value in getStories - technically it's actually the return value, not the last next() value.var firstEverStory = await* ;// grab all similar stock prices and return them in the stream immediately// short-hand for: for(var x on obs) { yield x };// This is just here to demonstrate that you shouldn't replace yields inside a function// scope that was present in the unexpanded source. Note that this is a regular// generator function, not an async one.var {somePromise;};name: name price: price topStories: topStories firstEverStory: firstEverStory ;}
...can be transpiled into the async/await feature proposed for ES7:
{var $args = Arrayprototype;return {var $done$decoratedObserver =;// inline invoke async function. This is like using a promise as a scheduler. Necessary// because ES6 doesn't expose microtask API{var stockPriceServiceUrlname$v0pricetopStoriestopStoryfirstEverStorynoop;// might've returned before microtask runs. This first check must run before any other// code. Not that the first await inline in the variable declaration has been moved down// beneath this line.if $done return;stockPriceServiceUrl = await ;if $done return;// for...on becomes forEach.// The function passed to observable.forEach becomes a next() function the observer.// If the next method returns a Promise, the Observable must wait until the Promise is// resolved before pushing more values. This is how backpressure works.// If there is any await expression (or another for on) in the body of the for on, the// function passed to forEach becomes an async function. Async functions return promises// so backpressure will be applied.await stockNames};return decoratedObserver;};}