Monads are a Class of Hard Drugs (2008) (opens in new tab)

They're not smart to do in python. Monads are an expressive, simple pattern of coding that can be type-checked. In dynamic languages it'll just look like a lot of unnecessary line noise. In HM type systems it's a great way to help you pass contexts around in a way lets you focus on your values.

But if you just want to speak Python, let me try to translate.

---

As a motivation for monads in Python, we're going to try to make "total" Python. To do so, we have to eschew execptions. This means we'll write stuff like

    def mypop(l):
      if len(l) > 0:
        return l.pop()
      else:
        return None

which, if we pass it a list of numbers, is guaranteed to return either a number or None---I've sidestepped the empty list exception. Now let's say I want to compose this function. For instance, my list of numbers is a list of bids and I have a function where I can try buying something with my bid.

    def buy(tendered):
      effective = tendered*1.2 # there's a discount!
      if market_open():
        if effective > 60:
          return True
        else:
          return False
      else:
        return None

So I believe that buy takes numbers and returns Booleans, so I might think that

    def trybuy(l): return buy(mypop(l))

takes lists of numbers and returns a boolean as to whether I've bought it. Of course, I have to be smart enough not to send in an empty list otherwise the discount in 'buy' will cause an error. I also have to be sure that if I try buying on days the market is closed to handle the 'None'. In Haskell, we'd write these constraints into the types and values using a Maybe type.

Here's a Pythonic Maybe type (kind of not pythonic though with the magic sentinel value, but we don't want to be able to sneak 'None's in).

    class Maybe(object):

      __sentinel__ = object()

      def __init__(self, obj = __sentinel__):
        self.isNothing = False
          if obj == self.__sentinel__:
            self.isNothing = True
          else:
            self.just = obj

    def just(x): return Maybe(x)
    def nothing(x): return Maybe()

Now we can have 'mypop' and 'buy' to return Maybe types to cover the cases of empty lists and closed markets.

    def mypop(l):
      if len(l) > 0:
        return just(l.pop())
      else:
        return nothing()

    def buy(tendered):
      effective = tendered*1.2 # there's a discount!
      if market_open():
        if effective > 60:
          return just(True)
        else:
          return just(False)
      else:
        return nothing()

So now we've abstracted the 'None's out a bit. 'mypop(l)' is always a "Maybe(Bool)" and you can inspect it to determine what the case is like 'mypop(l).isNothing'. Of course, this isn't any different from using None's and checking with "x == None" except it's more explicit: if you forget that 'mypop' returns Maybes then your program will throw an error on every use... instead of just the ones where you could have gotten a None without noticing for a while. (This is where the "You Probably Already Invented Monads" idea comes from, btw.)

In Haskell we'd say that mypop has type

    mypop :: [Float] -> Maybe Float

and buy has type

    buy :: Float -> Maybe Bool

but this makes it harder to make 'trybuy :: [a] -> Maybe Bool' since buy doesn't accept Maybes. It's also pretty clear, though, that if we send in an empty list we shouldn't even attempt to buy anything (our paycheck hasn't come in yet), so let's write that as a failure mode.

    def bind(maybe_something, maybe_process):
      if maybe_something.isNothing:
        return nothing()
      else:
        return maybe_process(maybe_something.just)

and now we can write trybuy such that it respects maybes with almost no additional noise

    # trybuy :: [Float] -> Maybe Bool
    def trybuy(l): return bind(mypop(l), buy)

The pattern we've captured here---the creation of lots of explicit context to help make functions more total and fail more quickly if you misuse them and the use of 'bind' to write functions that don't need to know about context on their input---is the Monad pattern. In Haskell, we recognize that many, many kinds of context are Monadic and thus can have default compositions programmed in. 'bind' is heavily overloaded and whenever you're writing code with context you use it often to compose functions without paying the complexity cost of routing that context around.

It's obviously not a really great Python pattern. This largely has to do with the fact that Python isn't terribly explicit about what's going on: you have to remember where exceptions and edge cases can fall out. In Haskell, we just use types to encode all of that into the documentation and the type checker ensures that we never mess up.

What I wish someone had explained to me earlier was that Monads are an abstraction level up from most things you ever program with. Learn specific ones, and the general picture will start to make more sense. Avoid the IO monad until you understand, in order:

+ the List monad (probably the easiest for python people, since it's very similar to list comprehensions)

+ the Maybe monad (kind of a degenerate case)

+ the State monad

And it's much easier if you do it in Haskell, trust me. "Learn You A Haskell For Great Good" is a nice easy intro to the basics.

> I'm still looking for someone who can present a cogent explanation of monads using only Python.

Personally, I don't do monads in any other language than Haskell. I find monads painful even in Ocaml. I'm pretty stupid as a programmer. And --- especially when doing things such as monad transformers --- I need lots of type checking and lots of type inference to help me over my mistakes (which are pervasive).

The cool thing is that when I've written these things called "monad transformer stacks" (see Real World Haskell for what I guess is the best explanation for those), I find that when my code gets through the type checker, it really does just work. It works, even when I've lost any sense of what my code is actually doing underneath. Without the type system, I couldn't have any confidence of this, and the runtime bugs you can potentially get with monads are so subtle that I would be terrified to use them in a language like Python (note again, I'm stupid, and YMMV).

The other reason why I wouldn't do monads without types is because, to me, monads are the type. The semantics of a monad is pretty much just that type (plus some crucial axioms). Types in Haskell are much more expressive than in most languages. The way "operator overloading" works in Haskell means that by specifying types, you are often generating runtime code (think C++ templates, but not for too long). That's why you see more type annotations in Haskell compared to (say) Ocaml. Though if you want to do the same thing in Ocaml, you'll be knee deep in the even more verbose world of these things called functors.

Consider the pattern of using None (or null) as an error. You want to compute: var x x=f(x) x=g(x) x=h(x)

however, if either f,g, or h return None, then the final value of x is None. The standard way of doing this is: def f(x): if x==None: return None ...

The monadic solution is the Maybe monad. Maybe has two constructors, Just(x) and Nothing. The above code would be equivilent to var x x=Just(x) x=x.apply(f) x=x.apply(g) x=x.apply(h)

The definition of Maybe is pretty simple. The Just object takes a value of a given type, and will run that value through a given function: Class Just(): def __init__(val): this.val = val

def apply(f); return f(x)

The Nothing object is even simpler: Class Nothing(): def apply(f): return Nothing()

Using this, we can see that in our original example, once one function returns Nothing, we skip all the other functions and end up with Nothing. When a function does return something, it needs to wrap it in Just, so we have:

def f(x): ... return Just(x)

In general, to be a Monad you need to satisfy the type class Monad(): def apply(f)

Of course, using Monads like this can end up being somewhat messy, so there is much syntactic sugar around them.

>Anyone who understands monads is on a class of hard drugs.

It's so frustrating to hear you say that. Monads are such a simple concept but they're difficult to explain because they're far more general and abstract than most programming topics. The best explanation I have is that monads are a way of storing some kind of computation with a value. When the user wants access to that value, he uses a monadic operator which forces the monad's computation to be run before returning the value.

Python it is. I've explained monads using Python many times.

http://www.dustingetz.com/2012/04/07/dustins-awesome-monad-t...

http://www.dustingetz.com/2012/10/02/reader-writer-state-mon...

(I'm not dustingetz though, he's named dustingetz on HN.)

This article definitely shows its age. These are the kinds of criticisms the Haskell community has been working quite obsessively to counter for some time. Thus the emphasis on explaining monads in a friendly matter and impressive optimization work that's been put into place.

---

And yeah, monads are much more about a particular type of composition or sequencing. Compare the common composition types (warning, scary but harmless jargon to follow)

  class Monoid m where
    e    :: m
    (<>) :: m -> m -> m

  class Functor f => Applicative f where
    pure :: a -> f a
    ap   :: f (a -> b) -> f a -> f b

  class Functor f => Monoidal f where -- isomorphic to Applicative
    unit :: f ()
    prod :: (f a, f b) -> f (a, b)

  class Applicative m => Monad m where
    return :: a       -> m a
    join   :: m (m a) -> m a

  class Applicative m => Monad' m where -- isomorphic to Monad
    return :: a       -> m a
    bind   :: m a -> (a -> m b) -> m b

  class Profunctor f => Arrow f where
    arr   :: (a -> b) -> f a b
    (>>>) :: f a b -> f b c -> f a c

    -- kind of ignore this one :)
    first :: f a b -> f (a, c) (b, c)

You can see a progression in the constraints on how things combine as you move down the list. First you have plain composition of non-container Functor types.

Then you have the Applicative/Monoidal functors (they're equivalent/isomorphic) which are probably most clearly demonstrated by the Monoidal class--it implements composition of Functors which maps to products in the contained types (or, has a "applicative" product which commutes with the functor)!

Then you have Monadic functor composition where you have "sequential" or "inward" composition via join (compare to Applicative's "horizontal" product composition) which can be implemented by the 'bind' function which maps a function that rewraps contained values.

Finally you have the Arrow types which add composition to Profunctors (which are like functors which contain both "incoming" and "outgoing" values) by composing them like a category.

---

Which is a lot of words to say that (1) Monads are just one of a whole group of "kinds" of composition and (2) they basically represent composition which allows for control over how functorial contexts get sequenced.

It's probably true that simply written Haskell code is fast enough for most uses. The trouble comes when that code isn't fast enough, or should be faster.

How do you make Haskell code faster? Apparently it's somewhat like writing fast Java code: avoid most features, use primitive data types, and write like you're writing C. Haskell code optimized for performance ends up like messy C code more often than not, except that unsafePerformIO isn't such a prickly topic in C.

Even Haskell professionals often don't know when "free" GHC optimizations will kick in, because the optimizations can be so picky, even after you remember to apply some forgotten language pragma. Writing performant Haskell is a black art.

For the first point, I think the author is aware that you can do more than just mutable state using monads:

> But mutable state covers everything that is not pure computation and that includes debug statements, input and output, system calls, network access, getting the time of the day or even throwing an exception. So the monad you are using has to provide all these services. Of course not every monad has everything you need so you have the IO monad, the Maybe monad, the STM monad, and so on. Whenever you have a C binding, you get a monad.

The second point example doesn't seem to address the author's concerns. The author is complaining about what happens when you try to compose N components that all disagree on the model of computation. OCaml has a more practical default model of computation so less such plumbing is needed.

I used to feel this way too, so I'm not trying to be coy, but really that's a property of a (some) Functor(s), not a Monad (except in that all Monads are also Functors).

Monad is just the pattern where you combine Functors. Some Functors can be pattern matched upon to "escape" their values. Functors like IO cannot. IO would still be (academically) useful if not for its Monad instance... you'd just only be allowed a single effect in the entire program!