undefined | Better HN

0 pointsmjw12y ago0 comments

Oh for sure, more as an exercise in curiosity than anything else. (Although I am also playing with generator- and iterator-like abstractions as part of a resource-scoped foldable stream abstraction to help process big files.)

Looks like I managed to answer the 'is it possible?' part of the question anyway -- something like this:

    (defn range
      [n]
      (let [c (chan)]
        (go
         (loop [i 0]
           (>! c i)
           (if (< i n)
             (recur (inc i))
             (close! c))))
        (fn [] (<!! c))))

0 comments

2 comments · 1 top-level

pron12y ago· 1 in thread

Yes, but I wouldn't do that. Every time you pull an item out of the channel, you actually submit a task to execute in a separate thread pool (to produce the next number), block your own thread, and wait for the task to complete on its thread and then wake your thread up. That's a lot of task-switching going on just to generate the next consecutive number :)

OTOH, you could use async's coroutine code (used to implement go blocks) to create generators, but because you have lazy sequences, that's not necessary either.

mjwOP12y ago

Yep I figured this would probably not be very efficient.

About lazy sequences: sometimes you want to avoid the allocation of lots of intermediate cons cells when mapping/filtering/etc. The reducers framework for example manages to avoid this sort of cost when the data structure supports something faster than first/rest recursion. I'm interested in extending reducers to work nicely over large files and other sequences which don't fit in memory, and some kind of iterable or generator-like abstraction could play a useful role in this.

(Actual coroutine-based generators might not be necessary, but would allow for a neat outward-facing API)

j / k navigate · click thread line to collapse