You'd still need to know that the number is a base 256 encoded string, and the chances you run across such a number in an innocent context (i.e. math publication) seems infinitesimal.

Someone who knowingly hosts a base-256 encoded secret should be held as accountable as if they had hosted the same information in picture form, morse code form, or plaintext.