Clever Geek Handbook

Iterables and Iterators: An In-Depth Guide to JavaScript



This article is an in-depth introduction to iterables and iterators in JavaScript. My main motivation for writing this was to prepare myself to learn about generators. In fact, I was planning to experiment later with combining generators and React hooks. If you are interested, then follow my Twitter or YouTube !



Actually, I planned to start with an article on generators, but it soon became obvious that they are difficult to talk about without a good understanding of iterables and iterators. We will focus on them now. I will assume that you do not know anything on this topic, but at the same time we will delve into it significantly. So if you are something know about iterables and iterators, but don't feel comfortable using them, this article will help you.



Introduction



As you noticed, we are discussing iterables and iterators. These are interrelated, but different concepts, so when reading the article, pay attention to which one is being discussed in a particular case.



Let's start with iterable objects. What it is? This is something that can be iterated over, for example: 



for (let element of iterable) {
    // do something with an element
}


Please note that here we are only looking at loops for ... of



that were introduced in ES6. And loops for ... in



are an older construct that we will not refer to at all in this article.



Now you might be thinking, "Okay, this iterable variable is just an array!" That's right, arrays are iterable. But now there are other structures in native JavaScript that you can use in a loop for ... of



. That is, besides arrays, there are other iterable objects.



For example, we can iterate Map



, introduced in ES6: 



const ourMap = new Map();

ourMap.set(1, 'a');
ourMap.set(2, 'b');
ourMap.set(3, 'c');

for (let element of ourMap) {
    console.log(element);
}


This code will display: 



[1, 'a']
[2, 'b']
[3, 'c']


That is, the variable element



at each iteration stage stores an array of two elements. The first is the key, the second is the value.



That we were able to use a loop for ... of



to iterate Map



proves that Map



's are iterable. Again for ... of



, only iterable objects can be used in loops . That is, if something works with this loop, then it is an iterable object.



It's funny that the constructor Map



optionally accepts iterables of key-value pairs. That is, this is an alternative way of constructing the same Map



: 



const ourMap = new Map([
    [1, 'a'],
    [2, 'b'],
    [3, 'c'],
]);


And since it Map



is an iterable, we can make copies of it very easily: 



const copyOfOurMap = new Map(ourMap);


We now have two different ones Map



, although they store the same keys with the same values.



So we saw two examples of iterable objects - array and ES6 Map



. But we don't yet know how they got the ability to be iterable. The answer is simple: there are iterators associated with them . Be careful: iterators are not iterable .



How is an iterator associated with an iterable object? A simply iterable object must contain a function in its property Symbol.iterator



. When called, the function must return an iterator for this object.



For example, you can retrieve an array iterator: 



const ourArray = [1, 2, 3];

const iterator = ourArray[Symbol.iterator]();

console.log(iterator);


This code outputs to the console Object [Array Iterator] {}



. Now we know that the array has an associated iterator, which is some kind of object.



What is an iterator?



It's simple. An iterator is an object containing a method next



. When this method is called, it should return:



  • next value in a sequence of values;
  • information about whether the iterator has finished generating values.


Let's test this by calling a method next



on our array iterator: 



const result = iterator.next();

console.log(result);


We will see the object in the console { value: 1, done: false }



. The first element of the array we created is 1, and here it appeared as a value. We also received information that the iterator has not finished yet, that is, we can still call the function next



and get some values. Let's try! Let's call next



it two more times: 



console.log(iterator.next());
console.log(iterator.next());


Received one by one { value: 2, done: false }



and { value: 3, done: false }



.



There are only three elements in our array. What happens if you call it again next



? 



console.log(iterator.next());


This time we'll see { value: undefined, done: true }



. This indicates that the iterator is complete. There is no point in calling again next



. If we do this, over and over again we will receive an object { value: undefined, done: true }



. done: true



means to stop iterating.



Now you can understand what it does for ... of



under the hood:



  • the first method [Symbol.iterator]()



    is called to get the iterator;
  • the method next



    is called cyclically on the iterator until we get it done: true



    ;
  • after each call next



    , the property is used in the body of the loop value



    .


Let's write all this in code: 



const iterator = ourArray[Symbol.iterator]();

let result = iterator.next();

while (!result.done) {
    const element = result.value;

    // do some something with element

    result = iterator.next();
}


This code is equivalent to this: 



for (let element of ourArray) {
    // do something with element
}


You can verify this, for example, by inserting console.log(element)



instead of a comment // do something with element



.



Create your own iterator



We now know what iterables and iterators are. The question arises: "Can I write my own instances?"



Certainly!



There is nothing mysterious about iterators. These are just objects with a method next



that behave in a special way. We've already figured out which native values ​​in JS are iterable. No objects were mentioned among them. Indeed, they are not iterated over natively. Consider an object like this: 



const ourObject = {
    1: 'a',
    2: 'b',
    3: 'c'
};


If we iterate over it with for (let element of ourObject)



, we get an error object is not iterable



.



Let's write our own iterators by making such an object iterable!



To do this, you have to patch the prototype Object



with your own method [Symbol.iterator]()



. Since patching the prototype is bad practice, let's create our own class by extending Object



: 



class IterableObject extends Object {
    constructor(object) {
        super();
        Object.assign(this, object);
    }
}


The constructor of our class takes an ordinary object and copies its properties into an iterable object (although it is not actually iterable yet!).



Let's create an iterable object: 



const iterableObject = new IterableObject({
    1: 'a',
    2: 'b',
    3: 'c'
})


To make a class IterableObject



truly iterable, we need a method [Symbol.iterator]()



. Let's add it. 



class IterableObject extends Object {
    constructor(object) {
        super();
        Object.assign(this, object);
    }

    [Symbol.iterator]() {

    }
}


Now you can write a real iterator!



We already know that it must be an object with a method next



. Let's start with this. 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        return {
            next() {}
        }
    }
}


After each call, next



you need to return a view object { value, done }



. Let's make it with fictitious values. 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        return {
            next() {
                return {
                    value: undefined,
                    done: false
                }
            }
        }
    }
}


Given an iterable object like this: 



const iterableObject = new IterableObject({
    1: 'a',
    2: 'b',
    3: 'c'
})


we will output key-value pairs, similar to how ES6 iteration does Map



: 



['1', 'a']
['2', 'b']
['3', 'c']


In our iterator, property



we will store an array in the value [key, valueForThatKey]



. Please note that this is our own solution compared to the previous steps. If we wanted to write an iterator that returns only keys or just property values, we could do it without any problems. We just decided to return key-value pairs now.



We need an array of the type [key, valueForThatKey]



. The easiest way to get it is with the method Object.entries



. We can use it right before creating the iterator object in the method [Symbol.iterator]()



: 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        // we made an addition here
        const entries = Object.entries(this);

        return {
            next() {
                return {
                    value: undefined,
                    done: false
                }
            }
        }
    }
}


The iterator returned in the method will access the variable thanks to the JavaScript closure entries



.



We also need a state variable. It will tell us which key-value pair should be returned on the next call next



. Let's add it: 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        const entries = Object.entries(this);
        // we made an addition here
        let index = 0;

        return {
            next() {
                return {
                    value: undefined,
                    done: false
                }
            }
        }
    }
}


Note that we declared the variable index



c let



because we know that we plan to update its value after each call next



.



We are now ready to return the actual value in the method next



: 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        const entries = Object.entries(this);
        let index = 0;

        return {
            next() {
                return {
                    // we made a change here
                    value: entries[index],
                    done: false
                }
            }
        }
    }
}


It was easy. We only use variables entries



and index



to access the correct key-value pair from the array entries



.



Now we need to deal with the property done



, because now it will always be false



. You can make one more variable besides entries



and index



, and update it after each call next



. But there is an even easier way. Let's check if index



the array is out of bounds entries



: 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        const entries = Object.entries(this);
        let index = 0;

        return {
            next() {
                return {
                    value: entries[index],
                    // we made a change here
                    done: index >= entries.length
                }
            }
        }
    }
}


Our iterator ends when the variable index



is equal to or greater than its length entries



. For example, if y has entries



length 3, then it contains values ​​at indices 0, 1 and 2. And when the variable index



is equal to or greater than 3, it means that there are no more values ​​left. We're done.



This code almost works. There is only one thing left to add.



The variable index



starts at 0, but ... we don't update it! It's not that simple. We need to update the variable after we have returned { value, done }



. But when we returned it, the method next



stops immediately even if there is some code after the expression return



. But we can create an object { value, done }



, store it in a variable, update it, index



and only then return the object: 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        const entries = Object.entries(this);
        let index = 0;

        return {
            next() {
                const result = {
                    value: entries[index],
                    done: index >= entries.length
                };

                index++;

                return result;
            }
        }
    }
}


After our changes, the class IterableObject



looks like this: 



class IterableObject extends Object {
    constructor(object) {
        super();
        Object.assign(this, object);
    }

    [Symbol.iterator]() {
        const entries = Object.entries(this);
        let index = 0;

        return {
            next() {
                const result = {
                    value: entries[index],
                    done: index >= entries.length
                };

                index++;

                return result;
            }
        }
    }
}


The code works great, but it got pretty confusing. This is because it shows a smarter but less obvious way to update index



after object creation result



. We can just initialize index



to -1! And although it is updated before the object returns from next



, everything will work fine, because the first update will replace -1 with 0.



So let's do it: 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        const entries = Object.entries(this);
        let index = -1;

        return {
            next() {
                index++;

                return {
                    value: entries[index],
                    done: index >= entries.length
                }
            }
        }
    }
}


As you can see, now we don't need to juggle the order of object creation result



and update index



. During the second call, it index



will be updated to 1, and we will return a different result, and so on. Everything works as we wanted, and the code looks much simpler.



But how do we check the correctness of the work? You can manually run a method [Symbol.iterator]()



to instantiate an iterator and then directly check the results of the calls next



. But you can do much easier! It was said above that any iterable object can be inserted into a loop for ... of



. Let's do just that, logging the values ​​returned by our iterable object along the way: 



const iterableObject = new IterableObject({
    1: 'a',
    2: 'b',
    3: 'c'
});

for (let element of iterableObject) {
    console.log(element);
}


Works! This is what is displayed in the console: 



[ '1', 'a' ]
[ '2', 'b' ]
[ '3', 'c' ]


Cool! We started with an object that could not be used in loops for ... of



, because they do not natively contain built-in iterators. But we created our own IterableObject



, which has an associated self-written iterator.



Hope you can now see the potential of iterables and iterators. It is a mechanism that allows you to create your own data structures to work with JS functions like loops for ... of



, and they work just like native structures! This is a very useful feature that can greatly simplify your code in certain situations, especially if you plan to iterate your data structures frequently.



In addition, we can customize what exactly these iterations should return. Our iterator now returns key-value pairs. What if we only want values? Easy, just rewrite the iterator: 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        // changed `entries` to `values`
        const values = Object.values(this);
        let index = -1;

        return {
            next() {
                index++;

                return {
                    // changed `entries` to `values`
                    value: values[index],
                    // changed `entries` to `values`
                    done: index >= values.length
                }
            }
        }
    }
}


And that's it! If we now start the loop for ... of



, we will see in the console: 



a
b
c


We returned only the values ​​of the objects. All of this proves the flexibility of self-written iterators. You can make them return whatever you want.



Iterators as ... iterable objects



It is very common for people to confuse iterators and iterables. This is a mistake and I have tried to neatly separate the two. I suspect I know the reason why people confuse them so often.



It turns out that iterators ... are sometimes iterables!



What does this mean? As you remember, an iterable is the object with which an iterator is associated. Every native JavaScript iterator has a method [Symbol.iterator]()



that returns another iterator! This makes the first iterator an iterable object.



You can check this if you take an iterator returned from an array and call on it [Symbol.iterator]()



: 



const ourArray = [1, 2, 3];

const iterator = ourArray[Symbol.iterator]();

const secondIterator = iterator[Symbol.iterator]();

console.log(secondIterator);


After running this code, you will see Object [Array Iterator] {}



. That is, an iterator not only contains another iterator associated with it, it is also an array.



If you compare both iterators with, ===,



it turns out that they are exactly the same: 



const iterator = ourArray[Symbol.iterator]();

const secondIterator = iterator[Symbol.iterator]();

// logs `true`
console.log(iterator === secondIterator);


At first, you may find it strange the behavior of an iterator that is its own iterator. But this is a very useful feature. You cannot stick a naked iterator into a loop for ... of



, it only accepts an iterable object - an object with a method [Symbol.iterator]()



.



However, the situation where an iterator is its own iterator (and therefore an iterable object) hides the problem. Since native JS iterators contain methods [Symbol.iterator]()



, you can pass them directly into loops without hesitation for ... of



.



As a result, this snippet: 



const ourArray = [1, 2, 3];

for (let element of ourArray) {
    console.log(element);
}


and this one: 



const ourArray = [1, 2, 3];
const iterator = ourArray[Symbol.iterator]();

for (let element of iterator) {
    console.log(element);
}


work seamlessly and do the same thing. But why would anyone use iterators like this directly in loops for ... of



? Sometimes it's just inevitable.



First, you may need to create an iterator without belonging to any iterable. We'll look at this example below, and it's not uncommon. Sometimes we just don't need the iterable itself.



And it would be very awkward if having a bare iterator meant that you cannot use it in for ... of



. Of course, you can do this manually using a method next



and, for example, a loop while



, but we have seen that for this you have to write a lot of code, moreover, repetitive.



The solution is simple: if you want to avoid boilerplate code and use an iterator in a loop for ... of



, you have to make the iterator an iterable object.



On the other hand, we also get iterators quite often from methods other than [Symbol.iterator]()



. For example, ES6 Map



contains methods entries



, values



and keys



. They all return iterators.



If native JS iterators weren't also iterable objects, you could not use these methods directly in loops for ... of



, like this: 



for (let element of map.entries()) {
    console.log(element);
}

for (let element of map.values()) {
    console.log(element);
}

for (let element of map.keys()) {
    console.log(element);
}


This code works because the iterators returned by the methods are also iterable objects. Otherwise, you would have to, say, wrap the call result map.entries()



in some stupid iterable object. Fortunately, we don't need to do this.



It is considered good practice to make your own iterable objects. Especially if they are returned from methods other than [Symbol.iterator]()



. Making an iterator an iterable object is very easy. Let me show with an example of an iterator IterableObject



: 



class IterableObject extends Object {
    // same as before

    [Symbol.iterator]() {
        // same as before

        return {
            next() {
                // same as before
            },

            [Symbol.iterator]() {
                return this;
            }
        }
    }
}


We have created a method [Symbol.iterator]()



below the method next



. Made this iterator its own iterator by simply returning this



, meaning it returns itself. Above we have already seen how an array iterator behaves. This is enough for our iterator to work in loops for ... of



even directly.



Iterator state



It should now be obvious that each iterator has a state associated with it. For example, in an iterator, IterableObject



we stored a state - a variable index



- as a closure. And we updated it after each iteration step.



What happens after the iteration process is complete? The iterator becomes useless and you can (should!) Delete it. You can see that this is happening even by the example of native JS objects. Let's take an array iterator and try to run it twice in a loop for ... of



. 



const ourArray = [1, 2, 3];

const iterator = ourArray[Symbol.iterator]();

for (let element of iterator) {
    console.log(element);
}

for (let element of iterator) {
    console.log(element);
}


You might expect the console to display the numbers twice 1



, 2



and 3



. But the result will be like this: 



1
2
3


Why?



Let's manually call next



after the loop ends: 



const ourArray = [1, 2, 3];

const iterator = ourArray[Symbol.iterator]();

for (let element of iterator) {
    console.log(element);
}

console.log(iterator.next());


The last log is output to the console { value: undefined, done: true }



.



That's it. After the loop ends, the iterator goes into the "done" state. Now it will always return an object { value: undefined, done: true }



.



Is there a way to "reset" the state of the iterator so that it can be used a second time in for ... of



? In some cases, it is possible, but it makes no sense. Therefore, it [Symbol.iterator]



is a method, not just a property. You can call the method again and get another iterator: 



const ourArray = [1, 2, 3];

const iterator = ourArray[Symbol.iterator]();

for (let element of iterator) {
    console.log(element);
}

const secondIterator = ourArray[Symbol.iterator]();

for (let element of secondIterator) {
    console.log(element);
}


Everything now works as expected. Let's see why multiple forward looping through the array works: 



const ourArray = [1, 2, 3];

for (let element of ourArray) {
    console.log(element);
}

for (let element of ourArray) {
    console.log(element);
}


All loops for ... of



use different iterators! Once the iterator and loop have finished, this iterator is no longer used.



Iterators and Arrays



Since we use iterators (albeit indirectly) in loops for ... of



, they can look deceivingly like arrays. But there are two important differences. Iterator and Array use the concepts of greedy and lazy values. When you create an array, at any given moment in time it has a certain length, and its values ​​are already initialized. Of course, you can create an array with no values ​​at all, but that's not the case. My point is that it is impossible to create an array that initializes its values ​​only after you access them by writing array[someIndex]



. It might be possible to get around this with a proxy or some other trick, but by default, JavaScript arrays don't behave that way.



And when they say that an array has a length, they mean that this length is finite. There are no infinite arrays in JavaScript.



These two qualities indicate the greediness of the arrays.



And iterators are lazy .



To demonstrate this, we will create two of our iterators: the first will be infinite, unlike finite arrays, and the second will initialize its values ​​only when they are requested by the iterator user.



Let's start with an infinite iterator. Sounds intimidating, but it's very simple to create: the iterator starts at 0 and returns the next number in the sequence at each step. Forever. 



const counterIterator = {
    integer: -1,

    next() {
        this.integer++;
        return { value: this.integer, done: false };
    },

    [Symbol.iterator]() {
        return this;
    }
}


And that's it! We started with a property of integer



-1. On each call, next



we increment it by 1 and return it in the object as value



. Note that we used the aforementioned trick again: we started at -1 to return 0 the first time.



Also take a look at the property done



. It will always be false. This iterator does not end!



In addition, we made the iterator an iterable by giving it a simple implementation [Symbol.iterator]()



.



One last thing: this is the case I mentioned above - we created an iterator, but it doesn't need an iterable parent to work.



Now let's try this iterator in a loop for ... of



. You just need to remember to stop the loop at some point, otherwise the code will be executed forever. 



for (let element of counterIterator) {
    if (element > 5) {
        break;
    }
    
    console.log(element);
}


After launch, we will see in the console: 



0
1
2
3
4
5


We've actually created an infinite iterator that returns as many numbers as you like. And it was very easy to make it!



Now let's write an iterator that doesn't create values ​​until they are requested.



Well ... we've already done it!



Have you noticed that it counterIterator



only stores one property number at any given time integer



? This is the last number returned on the call next



. And this is the same laziness. An iterator can potentially return any number (more precisely, a positive integer). But it creates them only when they are needed: when the method is called next



.



This can look pretty gimmicky. After all, numbers are created quickly and do not take up much memory space. But if you are working with very large objects that take up a lot of memory, then sometimes replacing arrays with iterators can be very useful, speeding up the program and saving memory.



The larger the object (or the longer it takes to create), the greater the benefit.



Other Ways to Use Iterators



So far, we have only consumed iterators in a loop for ... of



or manually using the next



. But these are not the only ways.



We have already seen that the constructor Map



takes iterables as an argument. You can also Array.from



easily convert an iterable to an array using the method . But be careful! As I said, the laziness of the iterator can sometimes be a big advantage. Converting to an array takes away laziness. All values ​​returned by the iterator are initialized immediately and then placed into an array. This means that if we try to convert infinite counterIterator



to an array, it will lead to disaster. Array.from



will execute forever without returning a result. So before converting an iterable / iterator to an array, you need to make sure the operation is safe.



Interestingly, iterables also work well with the spread operator (...



.) Remember that this works the same Array.from



way when all of the iterator values ​​are generated at once. For example, you can create your own version using the spread operator Array.from



. Just apply the operator to the iterable and then put the values ​​into an array: 



const arrayFromIterator = [...iterable];


You can also get all the values ​​from the iterable and apply them to the function: 



someFunction(...iterable);


Conclusion



I hope you now understand the title of the article Iterable Objects and Iterators. We learned what they are, how they differ, how to use them, and how to create our own instances. We are now completely ready to work with generators. If you are familiar with iterators, then moving on to the next topic shouldn't be too difficult.


More articles:


All Articles