This looks pretty interesting, but I don't understand why decompilers are not allowed. If this benchmark was aimed at recreating a SASS/server based product then it might make more sense, but given the fact that decompilers are readily available in practice the "no read" restriction seems to artificially increase the challenge level.
We disallow decompilation because we want this task to answer the question of "how well can LMs program from scratch if there's no structure given", not "how well can LMs stitch together decompiled pieces".
