Rerouting the wiring under the hood of generator functions and their iterators, where is the iter**able**?

user_7666635 0
Viewed 25

Digging deep under the hood of GeneratorFunction and Generators. I want to add useful properties to the IteratorResult and possibly to the IteratorResult.value itself. This is all part of a grand scheme to take over the world. But I wonder how I can get a hold of the actual Iterator which the Generator function provides?

First I notice that a generator function itself when being executed has no this value (it's the global object), regardless if it was called first or continues after next():

function* range(start = 0, end = Infinity, step = 1) {      
    console.debug("awon", this);
    for(let i = start; Math.sign(step)*(end - i) > 0; i += step) {
        console.debug("tawoo", this);
        i = (yield i)||i; // capricious trickery allowing us to go back or skip with next(arg)
        console.debug("esree", this);
    }
}

Now I make myself a useful side-entrance to do all sorts of magic to Generators (like define functions map, reduce, filter, etc.):

const Generator_prototype = Object.getPrototypeOf(range.prototype);

and with that, here I want to trap the provision of the Iterator object itself:

const originalIterate = Generator_prototype[Symbol.iterator];
function iterate() {
    console.log("iter", this);
    const iterator = originalIterate.apply(this);
    iterator.parent = this;
    return iterator;
}
Generator_prototype[Symbol.iterator] = iterate;

because, like I said, I want to massage the IterationResult coming from next():

const originalNext = Generator_prototype.next;
function next(...args) {
    console.log("next", this);
    let value = originalNext.apply(this, args);
    if(this.decorate)
        value = this.decorate(value); 
    return this.current = value; 
}
Generator_prototype.next = next;

And with this:

[...range(1,3)]

gives the output:

iter range {<suspended>}
next range {<suspended>, parent: range}
next range {<suspended>, parent: range, current: {…}}
next range {<suspended>, parent: range, current: {…}}
> (2) [1, 2]

so, I never got a hold of the Iterable, only the Iterator. The function [Symbol.iterator] is called on some object, but that object somehow has the thisArg bound to the iterator that it is supposed to return. Notably:

iterator = range(1,3)[Symbol.iterator]()
iterator.parent == iterator // > true

how can that even happen? I am replacing the @@iterator method with my own, and when it is called it already has an iterator as the thisArg, which is the exact same iterator that I would get if I am asking for the standard iterator by applying the originalIterator to the this? My head is spinning!

How can I get a hold on the iterable, not the iterator?

iterable generator iterator javascript
1 Answers

Two main things:

  1. Not all iterators have an iterable (which is, after all, just an object that implements the Symbol.iterator method, directly or indirectly).

  2. All built-in iterators use a prototype (which has no public name attached to it, sadly) that makes them iterable by implementating the Symbol.iterator method, having it just return this. This may seem odd, but it's why we can use for-of on built-in iterators (because they're also iterables). This built-in behavior is why you get iterator.parent === iterator being true in the code at the end of your question.

    Note that this only applies to built-in iterators. People can create their own iterators without inheriting from the standard prototype (which is, after all, a pain to get hold of), and sometimes do.

There's no iterable involved in your range function or the generator object it returns (other than the generator object itself, because of #2 above).

Your primary goal seems to be to augment or monkey-patch the built-in generator. You can do that via its prototype, but I wouldn't. This example doubles the values returned by your range generator:

// Get the prototype used by built-in generators and its `next` method
const genProto = Object.getPrototypeOf(Object.getPrototypeOf((function* () {})()));
const defaultGenNext = genProto.next;

// Replace it with our own `next`
genProto.next = function next(...args) {
    console.log(`Args:   ${JSON.stringify(args)}`);
    const result = defaultGenNext.apply(this, args);
    if (typeof result.value === "number") {
        console.log("Doubling");
        result.value *= 2;
    }
    console.log(`Result: ${JSON.stringify(result)}`);
    return result;
};

Live Example:

// Get the prototype used by built-in generators and its `next` method
const genProto = Object.getPrototypeOf(Object.getPrototypeOf((function* () {})()));
const defaultGenNext = genProto.next;

// Replace it with our own `next`
genProto.next = function next(...args) {
    console.log(`Args:   ${JSON.stringify(args)}`);
    const result = defaultGenNext.apply(this, args);
    if (typeof result.value === "number") {
        console.log("Doubling");
        result.value *= 2;
    }
    console.log(`Result: ${JSON.stringify(result)}`);
    return result;
};

// Your `range` function
function* range(start = 0, end = Infinity, step = 1) {
    // console.debug("awon", this);
    for (let i = start; Math.sign(step) * (end - i) > 0; i += step) {
        // console.debug("tawoo", this);
        i = (yield i) || i; // capricious trickery allowing us to go back or skip with next(arg)
        // console.debug("esree", this);
    }
}

// Example
for (const value of range(1, 5)) {
    console.log(value);
}
.as-console-wrapper {
    max-height: 100% !important;
}

Or with a push back to the generator if we want:

// Get the prototype used by built-in generators and its `next` method
const genProto = Object.getPrototypeOf(Object.getPrototypeOf((function* () {})()));
const defaultGenNext = genProto.next;

// Replace it with our own `next`
genProto.next = function next(...args) {
    console.log(`Args:   ${JSON.stringify(args)}`);
    const result = defaultGenNext.apply(this, args);
    if (typeof result.value === "number") {
        console.log("Doubling");
        result.value *= 2;
    }
    console.log(`Result: ${JSON.stringify(result)}`);
    return result;
};

// Your `range` function
function* range(start = 0, end = Infinity, step = 1) {
    // console.debug("awon", this);
    for (let i = start; Math.sign(step) * (end - i) > 0; i += step) {
        // console.debug("tawoo", this);
        i = (yield i) || i; // capricious trickery allowing us to go back or skip with next(arg)
        // console.debug("esree", this);
    }
}

// Example
let flag = true;
let result;
let it = range(1, 5);
while (!result?.done) {
    if (result) {
        console.log(result.value);
    }
    let param;
    if (flag && result && result.value === 4) {
        param = 1;
        flag = false;
    }
    result = it.next(param);
}
.as-console-wrapper {
    max-height: 100% !important;
}

Doubling is obviously just an example, and note that you have access to the value passed to next as well as to the result object it returns.

Again, though, I wouldn't do that, for at least two reasons:

  1. Mucking about with the built-in prototypes usually doesn't end well.
  2. Not all generators inherit from the generator prototype.

After all, generation (like iteration) is just a protocol. That means you can implement it with your own code, completely bypassing any of the built-in stuff.

What I'd do instead is write a function to wrap around the generator you want to transform. For instance, here's a simple doubler that doesn't support pushing back:

function* double(target) {
    for (const value of target) {
        yield value * 2;
    }
}

Then just use double(range(1, 5)) or whatever instead of range(1, 5).

Live Example:

// Your `range` function
function* range(start = 0, end = Infinity, step = 1) {
    // console.debug("awon", this);
    for (let i = start; Math.sign(step) * (end - i) > 0; i += step) {
        // console.debug("tawoo", this);
        i = (yield i) || i; // capricious trickery allowing us to go back or skip with next(arg)
        // console.debug("esree", this);
    }
}

function* double(target) {
    for (const value of target) {
        yield value * 2;
    }
}

// Example
for (const value of double(range(1, 5))) {
    console.log(value);
}
.as-console-wrapper {
    max-height: 100% !important;
}

Or you can write it to support the pushback:

// Your `range` function
function* range(start = 0, end = Infinity, step = 1) {
    // console.debug("awon", this);
    for (let i = start; Math.sign(step) * (end - i) > 0; i += step) {
        // console.debug("tawoo", this);
        i = (yield i) || i; // capricious trickery allowing us to go back or skip with next(arg)
        // console.debug("esree", this);
    }
}

function* double(target) {
    const it = target[Symbol.iterator]();
    let lastArg;
    let result;
    while (!(result = lastArg === undefined ? it.next() : it.next(lastArg)).done) {
        lastArg = yield result.value * 2;
    }
}

// Example
let flag = true;
let result;
let it = double(range(1, 5));
while (!result?.done) {
    if (result) {
        console.log(result.value);
    }
    let param;
    if (flag && result && result.value === 4) {
        param = 1;
        flag = false;
    }
    result = it.next(param);
}
.as-console-wrapper {
    max-height: 100% !important;
}

user_1572470
Related