Effective programming. Part 1: iterators and generators

Javascript is currently the most popular programming language according to versions of many sites (for example, Github). At the same time, is it the most advanced or favorite language? It lacks constructs that are integral parts for other languages: an extensive standard library, immutability, macros. But there is one detail in it that, in my opinion, does not receive enough attention - generators.



Further, the reader is offered an article, which, in the case of a positive response, can develop into a cycle. If I successfully write this cycle, and the Reader has successfully mastered it, it will be clear about the following code not only what it does, but also how it works under the hood:



while (true) {
    const data = yield getNextChunk(); //   
    const processed = processData(data);
    try {
        yield sendProcessedData(processed);
        showOkResult();
    } catch (err) {
        showError();
    }
}


This is the first, pilot part: Iterators and Generators.



Iterators



So, an iterator is an interface that provides sequential access to data.



As you can see, the definition says nothing about data or memory structures. Indeed, a sequence of undefined s can be represented as an iterator without taking up any memory space.



I suggest the reader to answer the question: is an array an iterator?



Answer
. shift pop .



Why, then, are iterators needed if an array, one of the basic structures of the language, allows you to work with data both sequentially and in arbitrary order?



Let's imagine that we need an iterator that implements a sequence of natural numbers. Or Fibonacci numbers. Or any other endless sequence. It is difficult to place an endless sequence in an array; you need a mechanism for gradually filling the array with data, as well as removing old data so as not to fill the entire process memory. This is an unnecessary complication, which carries with it additional complexity of implementation and support, despite the fact that a solution without an array can fit into several lines:



const getNaturalRow = () => {
    let current = 0;
    return () => ++current;
};


Also, an iterator can represent receiving data from an external channel, such as a websocket.



In javascript, an iterator is any object that has a next () method that returns a structure with fields value - the current value of the iterator and done - a flag indicating the end of the sequence (this convention is described in the ECMAScript language standard ). Such an object implements the Iterator interface. Let's rewrite the previous example in this format:



const getNaturalRow = () => ({
    _current: 0,
    next() { return {
        value: ++this._current,
        done: false,
    }},
});


Javascript also has an Iterable interface, which is an object that has an @@ iterator method (this constant is available as Symbol.iterator) that returns an iterator. For objects implementing such an interface, operator traversal is available for..of. Let's rewrite our example one more time, only this time as an Iterable implementation:



const naturalRowIterator = {
    [Symbol.iterator]: () => ({
        _current: 0,
        next() { return {
            value: ++this._current,
            done: this._current > 3,
       }},
   }),
}

for (num of naturalRowIterator) {
    console.log(num);
}
// : 1, 2, 3


As you can see, we had to make the done flag at some point become positive, otherwise the loop would be infinite.



Generators



Generators became the next stage in the evolution of iterators. They provide syntactic sugar to return iterator values ​​like a function value. A generator is a function (declared with an asterisk: function * ) that returns an iterator. In this case, the iterator is not returned explicitly; the functions only return the values ​​of the iterator using the yield statement . When the function finishes its execution, the iterator is considered complete (the results of subsequent calls to the next method will have the done flag equal to true)



function* naturalRowGenerator() {
    let current = 1;
    while (current <= 3) {
        yield current;
        current++;
    }
}

for (num of naturalRowGenerator()) {
    console.log(num);
}
// : 1, 2, 3


Already in this simple example, the main nuance of generators is visible to the naked eye: the code inside the generator function is not executed synchronously . The generator code is executed in stages, as a result of calls to next () on the corresponding iterator. Let's see how the generator code is executed in the previous example. We will use a special cursor to mark where the generator stopped.



When naturalRowGenerator is called, an iterator is created.



function* naturalRowGenerator() {
    β–·let current = 1;
    while (current <= 3) {
        yield current;
        current++;
    }
}


Further, when we call the next method for the first three times, or, in our case, we iterate through the loop, the cursor is positioned after the yield statement.



function* naturalRowGenerator() {
    let current = 1;
    while (current <= 3) {
        yield current; β–·
        current++;
    }
}


And for all subsequent calls to next and after exiting the loop, the generator completes its execution and, the results of calling next will be { value: undefined, done: true }



Passing parameters to an iterator



Let's imagine that we need to add the ability to reset the current counter and start counting from the beginning to our iterator of natural numbers.



naturalRowIterator.next() // 1
naturalRowIterator.next() // 2
naturalRowIterator.next(true) // 1
naturalRowIterator.next() // 2


It is clear how to handle such a parameter in a self-written iterator, but what about generators?

It turns out that generators support parameter passing!



function* naturalRowGenerator() {
    let current = 1;
    while (true) {
        const reset = yield current;
        if (reset) {
          current = 1;
        } else {
          current++;
        }
    }
}


The passed parameter is made available as a result of the yield statement. Let's try to add clarity with a cursor approach. When the iterator was created, nothing has changed. This is followed by the first call to the next () method:



function* naturalRowGenerator() {
    let current = 1;
    while (true) {
        const reset = β–·yield current;
        if (reset) {
          current = 1;
        } else {
          current++;
        }
    }
}


The cursor froze at the moment of returning from the yield statement. On the next call to next, the value passed to the function will set the value of the reset variable. Where does the value passed to the very first call to next end up, since there has not yet been a call to yield? Nowhere! It will dissolve in the vastness of the garbage collector. If you need to pass some initial value to the generator, then this can be done using the arguments of the generator itself. Example:



function* naturalRowGenerator(start = 1) {
    let current = start;
    while (true) {
        const reset = yield current;
        if (reset) {
          current = start;
        } else {
          current++;
        }
    }
}

const iterator = naturalRowGenerator(10);
iterator.next() // 10
iterator.next() // 11
iterator.next(true) // 10


Conclusion



We have discussed the concept of iterators and its implementation in the javascript language. We also studied generators - a syntactic construct for conveniently implementing iterators.



Although I've given examples with number sequences in this article, javascript iterators can do a lot more. They can represent any sequence of data and even many finite state machines. In the next article, I would like to talk about how you can use generators to build asynchronous processes (coroutines, goroutines, csp, etc.).



All Articles