Elixir-style pipelines in Ruby (opens in new tab)

(gregnavis.com)

63 pointsgregnavis3y ago16 comments

16 comments

13 comments · 6 top-level

joeman10003y ago· 4 in thread

I've had a look at threading/piping operators in a few languages (list below). I'd say that Racket has the best one I've used. I love that you can specify a '_' for the hole which the result from the previous operation will fill. Julia's threading macro is surprisingly brittle, only letting you chain single-argument functions unless you want to use anonymous functions with one bound variable and the rest free.

+ Haskell :: https://www.schoolofhaskell.com/user/Gabriel439/Pipes%20tuto...

+ Racket :: https://docs.racket-lang.org/threading/index.html

+ Clojure :: https://clojure.org/guides/threading_macros

+ Julia :: https://syl1.gitbook.io/julia-language-a-concise-tutorial/us...

+ R :: https://r4ds.had.co.nz/pipes.html

sundarurfriend3y ago

> Julia's threading macro is surprisingly brittle, only letting you chain single-argument functions

Julia's threading/piping is surprisingly limited, but the phrasing here (combined with the link) could make things confusing to a Julia beginner. So to make things clear:

* The linked page is talking about an external Julia package which provides a macro. And that macro lets you use the `_` syntax similar to what you describe Racket as having.

* Julia's default inbuilt piping syntax is the one with the single-argument limitation, and that's an operator, not a macro.

There's been a lot of discussion about bringing the `_` syntax or something like it to the base language, but there seem to be implementation difficulties. [1]

> +R :: https://r4ds.had.co.nz/pipes.html

This page is talking about magrittr piping (which is probably still the most popular), but base R also got inbuilt piping syntax with version 4.1. It automatically passes the piped-in value as the first argument to the subsequent function. [2] [3]

[1] https://github.com/JuliaLang/julia/pull/24990

[2] https://www.r-bloggers.com/2021/05/the-new-r-pipe/

[3] https://michaelbarrowman.co.uk/post/the-new-base-pipe/

nine_k3y ago

+ F# :: https://fsharp.github.io/fsharp-core-docs/reference/fsharp-c... — only works with unary functions.

+ Hack :: https://docs.hhvm.com/hack/expressions-and-operators/pipe — works with any functions, uses $$ for the pipelined value.

OCaml also has a similar thing (@@) but I can't find a good reference page.

fells3y ago

OCaml :: https://v2.ocaml.org/api/Stdlib.html#1_Compositionoperators, it has `|>` for (unary) pipelines and `@@`.

fulafel3y ago

> I'd say that Racket has the best one I've used. I love that you can specify a '_' for the hole which the result from the previous operation will fill.

Clojure also has a placeholder variant, the as-> form. Quoting the example the Clojure doc page (https://clojure.org/guides/threading_macros):

  (as-> [:foo :bar] v
     (map name v)
     (first v)
     (.substring v 1))
  
  ;; => "oo"

zelphirkalt3y ago· 3 in thread

Do the steps of the pipeline implementation in Ruby run concurrently?

I once did an Elixir course and really liked the pipelines. I continued implementing pipelines with a Scheme macro, but not concurrently.

Fire-Dragon-DoL3y ago

Elixir pipelines are not concurrent, unless you use something that allows that (flow and probably stream behind the scenes)

woodruffw3y ago

Not naively: this kind of pipelining is partial function application, not coordinated parallelism (the way Unix pipeliens are).

AtlasBarfed3y ago

Does anyone know any documentation on how UNIX achieves parallelism in pipelines, especially modern implementations? Is each stage of a piped command just another spun off process?

2 more replies

bwilliams3y ago

The beauty (and horror) of Ruby is that you can do almost anything with it. I think this is a really interesting and clever use of the "can do anything" aspect of Ruby, although I think I'd prefer not to run into it in a production app.

Still, it's really cool to see how far we can push/mold the language to accomplish different tasks and patterns.

bonquesha993y ago

Check out this proof of concept gem to perform pipe operations in Ruby using block expressions:

https://github.com/lendinghome/pipe_operator#-pipe_operator

  "https://api.github.com/repos/ruby/ruby".pipe do
    URI.parse
    Net::HTTP.get
    JSON.parse.fetch("stargazers_count")
    yield_self { |n| "Ruby has #{n} stars" }
    Kernel.puts
  end
  #=> Ruby has 15120 stars

quechimba3y ago

I like using .then for chaining stuff, like this:

    sig { params(obj: T.untyped).void }
    def write(obj)
      obj
        .then { Array(_1) }
        .then { @wrapper.pack(_1) }
        .then { @deflate.deflate(_1, Zlib::SYNC_FLUSH) }
        .then { @body.write(_1) }
    end

ilyash3y ago

While not as thorough solution as mentioned here in comments, UFCS goes a long way in this direction. In Next Generation Shell, I've designed the methods so that the first parameter is something that is likely to come from "the pipeline". Hence mylist.filter(...).map(...) just work. Combined with multiple dispatch and the fact that methods don't belong to a particular type/class, it allows creating user-defined methods with same convention to work with any existing and new types/classes.

UFCS - https://en.m.wikipedia.org/wiki/Uniform_Function_Call_Syntax

j / k navigate · click thread line to collapse