undefined | Better HN

0 pointsdrdaeman6y ago0 comments

Coming from Perl, I used to want this badly, but then I thought that there's absolutely nothing wrong with

    m = re.match(...)
    if m is not None:
        pass

Now, I wonder what you meant saying that single-line version is less error-prone, because I don't think so. I believe they're exactly the same in this regard, except for a bizarre case when someone would bastardize the code by putting some irrelevant lines between the assignment and the comparison, obfuscating the logic.

Also, as I tend to use `if m is not None:` rather than just `if m:` (because I typically don't want to run all that subtle `__nonzero__` magic), it would - subjectively - look less pretty in the one-line version: `if (m := re.match(...)) is not None:`.

Use in the generator expressions to create aliases/shortcuts and avoid repetition is great, though. I love this use case.

0 comments

17 comments · 4 top-level

3PS6y ago· 10 in thread

The issue, in my opinion, is when you want something like this:

    while m := f(many, args):
        # do stuff with m

Now, if you're writing this in Python 3.7, you often end up with some code duplication:

    m = f(many, args)
    while m:
        # do stuff with m
        m = f(many, args) # duplicate

Or something like this:

    while True:
        m = f(many, args)
        if not m:
            break
        # do stuff with m

Personally, I consider the last version to be the most elegant, but still something of an anti-pattern. So I welcome this new operator.

hermitdev6y ago

I literally wrote your last example twice last week to hash files in 4k chunks. Those 4 lines would be reduced to 1 with the walrus operator. I welcome it, as well.

Edit: 1 line, not 2 (excluding the "do stuff")

cellularmitosis6y ago

This comes up in recursive descent implementations as well https://gist.github.com/cellularmitosis/53913ad229bad5d0a3bb...

ygrechuk6y ago

Why not something like this:

  def f_iter(many, args):
      while True:
          m = f(many, args)
          if m:
              yield m
          else:
              raise StopIteration
  ...
  for m in f_iter(many, args):
      # do stuff with m

This way you’re isolating all the initialization logic, error handling, etc. And you can focus on your domain logic in your client code.

bvrmn6y ago

Python has built-in facilities for that, there is no need to write wrapper by yourself.

   from functools import partial
   with open('mydata.db', 'rb') as f:
       for block in iter(partial(f.read, 64), b''):
           process_block(block)

ciupicri6y ago

Instead of raising StopIteration you could also just return from the function. You could also make it a bit shorter if you swap the if branches.

  def f_iter(many, args):
      while True:
          m = f(many, args)
          if not m:
              return
          yield m

1 more reply

repsilat6y ago

And less reasonably, something similar for regular expressions:

    for m in filter([re.match(...)], bool):
        ...

;-)

mlevental6y ago

so i should write a whole other function that iterates over a list instead of being happy that := exists?

1 more reply

pyrale6y ago

The last version clearly exposes that you have an infinite loop. This is something hidden by the `while(evaluate)` expression.

Also if it comes to pure form I would have preferred `while (evaluate) as x:` out of establishing a parallel with existing syntax, but that's not very important.

pdpi6y ago

On the contrary: the loop isn’t fundamentally infinite, it’s just an artefact of the language that you have to write it that way. The walrus lets you put the termination where it belongs.

1 more reply

xapata6y ago

That last one isn't an anti-pattern. That's the pattern concluded best for do-while in the discussion of PEP-315.

https://www.python.org/dev/peps/pep-0315/#notice

eesmith6y ago· 3 in thread

Think about if you match multiple patterns:

    for line in f:
      if (m:= pat1.search(line)) is not None:
        ... do stuff ..
      elif (m:= pat2.search(line)) is not None:
        ... do other stuff ..
      elif (m:= pat3.search(line)) is not None:
         ... do something else ..

In older Python that's:

    for line in f:
      m = pat1.search(line)
      if m is not None:
        ... do stuff ..
      else:
        m = pat2.search(line)
        if m is not None:
          ... do other stuff ..
        else:
           m = pat3.search(line)
           if m is not None:
             ... do something else ..

I think the newer makes it clear that it's supposed to be a simple elif chain, where all branches following the same structure.

There are other ways to structure it, but the alternatives I can think of also have their own cumbersome complexities.

drdaemanOP6y ago

If actions are different, I tend to put them in functions (independent testable pieces, yay!) and iterate over a mapping:

    LINE_ACTIONS = (
        (re.compile("pattern 1"), do_stuff),
        (re.compile("pattern 2"), do_other_stuff),
        (re.compile("pattern 3"), do_something_else),
    )
    ...
    for pattern, action in LINE_ACTIONS:
        m = pattern.search(line)
        if m is not None:
            action(m)
            break

Or even use a method registry pattern that would auto-populate LINE_ACTIONS by just declaring the fuctions:

    @action("pattern 1")
    def do_stuff(m):
       ...

Alternatively, I might just break the processing into a function:

    def _process(line):
        m = pat1.search(line)
        if m is not None:
            ... do stuff ...
            return

        m = pat2.search(line)
        if m is not None:
            ... do stuff ...
            return

        m = pat3.search(line)
        if m is not None:
            ... do stuff ...
            return

    _process(line)

Of course, this depends on the purpose. Could be completely inadequate in some situations.

eesmith6y ago

Indeed, I've also used this pattern.

I've found that I don't like using it. As you write, it's "inadequate in some situations", which I consider as part of the cumbersome complexities I mentioned.

For examples, do_stuff() and do_other_stuff() may need to share variables, and do_something_else() might need the line number to report an error while the others don't.

This can be handled with shared state/nonlocal, and by passing in more parameters to the generic handler API, but these add complexity.

Or, different parts of the file may have different line dispatch processing (eg, a header block followed by a data block followed by a footer block) where one of the handlers must indicate the transition to a different processor.

Also, function dispatch in CPython is slow. While the regex tests are also slow, it can also be important to consider the (using a hypothetical number) 5% overhead for dispatching over inline code.

kstrauser6y ago

That's... lovely. I propose making this the standard example of why you might want to use the walrus operator.

Alex39176y ago

If you only have one or two regex like that then there is no difference in readability. If you have ten in a row though then suddenly your function no longer fits on the screen and it’s more difficult to understand what’s going on.

bandrami6y ago

I mean, what you really want in this case is Perl's =~ operator, as well as the implicit $_ variable.

j / k navigate · click thread line to collapse

0 comments

17 comments · 4 top-level

3PS6y ago· 10 in thread

The issue, in my opinion, is when you want something like this:

    while m := f(many, args):
        # do stuff with m

Now, if you're writing this in Python 3.7, you often end up with some code duplication:

    m = f(many, args)
    while m:
        # do stuff with m
        m = f(many, args) # duplicate

Or something like this:

    while True:
        m = f(many, args)
        if not m:
            break
        # do stuff with m

Personally, I consider the last version to be the most elegant, but still something of an anti-pattern. So I welcome this new operator.

hermitdev6y ago

I literally wrote your last example twice last week to hash files in 4k chunks. Those 4 lines would be reduced to 1 with the walrus operator. I welcome it, as well.

Edit: 1 line, not 2 (excluding the "do stuff")

cellularmitosis6y ago

This comes up in recursive descent implementations as well https://gist.github.com/cellularmitosis/53913ad229bad5d0a3bb...

ygrechuk6y ago

Why not something like this:

  def f_iter(many, args):
      while True:
          m = f(many, args)
          if m:
              yield m
          else:
              raise StopIteration
  ...
  for m in f_iter(many, args):
      # do stuff with m

This way you’re isolating all the initialization logic, error handling, etc. And you can focus on your domain logic in your client code.

bvrmn6y ago

Python has built-in facilities for that, there is no need to write wrapper by yourself.

   from functools import partial
   with open('mydata.db', 'rb') as f:
       for block in iter(partial(f.read, 64), b''):
           process_block(block)

ciupicri6y ago

Instead of raising StopIteration you could also just return from the function. You could also make it a bit shorter if you swap the if branches.

  def f_iter(many, args):
      while True:
          m = f(many, args)
          if not m:
              return
          yield m

1 more reply

repsilat6y ago

And less reasonably, something similar for regular expressions:

    for m in filter([re.match(...)], bool):
        ...

;-)

mlevental6y ago

so i should write a whole other function that iterates over a list instead of being happy that := exists?

1 more reply

pyrale6y ago

The last version clearly exposes that you have an infinite loop. This is something hidden by the `while(evaluate)` expression.

Also if it comes to pure form I would have preferred `while (evaluate) as x:` out of establishing a parallel with existing syntax, but that's not very important.

pdpi6y ago

On the contrary: the loop isn’t fundamentally infinite, it’s just an artefact of the language that you have to write it that way. The walrus lets you put the termination where it belongs.

1 more reply

xapata6y ago

That last one isn't an anti-pattern. That's the pattern concluded best for do-while in the discussion of PEP-315.

https://www.python.org/dev/peps/pep-0315/#notice

eesmith6y ago· 3 in thread

Think about if you match multiple patterns:

    for line in f:
      if (m:= pat1.search(line)) is not None:
        ... do stuff ..
      elif (m:= pat2.search(line)) is not None:
        ... do other stuff ..
      elif (m:= pat3.search(line)) is not None:
         ... do something else ..

In older Python that's:

    for line in f:
      m = pat1.search(line)
      if m is not None:
        ... do stuff ..
      else:
        m = pat2.search(line)
        if m is not None:
          ... do other stuff ..
        else:
           m = pat3.search(line)
           if m is not None:
             ... do something else ..

I think the newer makes it clear that it's supposed to be a simple elif chain, where all branches following the same structure.

There are other ways to structure it, but the alternatives I can think of also have their own cumbersome complexities.

drdaemanOP6y ago

If actions are different, I tend to put them in functions (independent testable pieces, yay!) and iterate over a mapping:

    LINE_ACTIONS = (
        (re.compile("pattern 1"), do_stuff),
        (re.compile("pattern 2"), do_other_stuff),
        (re.compile("pattern 3"), do_something_else),
    )
    ...
    for pattern, action in LINE_ACTIONS:
        m = pattern.search(line)
        if m is not None:
            action(m)
            break

Or even use a method registry pattern that would auto-populate LINE_ACTIONS by just declaring the fuctions:

    @action("pattern 1")
    def do_stuff(m):
       ...

Alternatively, I might just break the processing into a function:

    def _process(line):
        m = pat1.search(line)
        if m is not None:
            ... do stuff ...
            return

        m = pat2.search(line)
        if m is not None:
            ... do stuff ...
            return

        m = pat3.search(line)
        if m is not None:
            ... do stuff ...
            return

    _process(line)

Of course, this depends on the purpose. Could be completely inadequate in some situations.

eesmith6y ago

Indeed, I've also used this pattern.

I've found that I don't like using it. As you write, it's "inadequate in some situations", which I consider as part of the cumbersome complexities I mentioned.

For examples, do_stuff() and do_other_stuff() may need to share variables, and do_something_else() might need the line number to report an error while the others don't.

This can be handled with shared state/nonlocal, and by passing in more parameters to the generic handler API, but these add complexity.

Also, function dispatch in CPython is slow. While the regex tests are also slow, it can also be important to consider the (using a hypothetical number) 5% overhead for dispatching over inline code.

kstrauser6y ago

That's... lovely. I propose making this the standard example of why you might want to use the walrus operator.

Alex39176y ago

bandrami6y ago

I mean, what you really want in this case is Perl's =~ operator, as well as the implicit $_ variable.

j / k navigate · click thread line to collapse