undefined | Better HN

0 pointsWithinReason3y ago0 comments

But then SUB, DIV and RELU could be an HLOP as well, no?

0 comments

We could have NEG instead of SUB, but with the constant folding it's a wash. DIV is already an HLOP with reciprocal (used to use POW, but that was slower. And what would you implement RELU in terms of?

WithinReasonOP3y ago

max(0,x)

georgehotz3y ago

That's a ReduceOp right now, more annoying to reason about than a UnaryOp. But in the limit, yea. Or add an elementwise BinaryOp for max.

Submit a PR if you can improve something!

j / k navigate · click thread line to collapse

0 comments

georgehotz3y ago

WithinReasonOP3y ago

max(0,x)

georgehotz3y ago

That's a ReduceOp right now, more annoying to reason about than a UnaryOp. But in the limit, yea. Or add an elementwise BinaryOp for max.

Submit a PR if you can improve something!

j / k navigate · click thread line to collapse