source: https://blog.openai.com/unsupervised-sentiment-neuron/
Their failure mode is long sentences which don't go anywhere (or mean anything), but that's less grating than a blatant parse failure, especially if you're skim reading.
[1] https://en.wikipedia.org/wiki/Claude_Shannon
[2] http://www.music-cog.ohio-state.edu/Music829D/Notes/Infotheo...
The giddy excitement of being on the verge of unlocking the power of language slowly turned into confusion and then decayed into disappointment as i slowly realized that the task was maybe a bucket of orders of magnitude harder than i thought. I think it produced some coherent phrases every now and then, but mostly it was a random words generator.
...I miss these amplified feelings of cluelessly diving into an impossible project...
This was very wonderfully articulated! Do you have a blog or something?
Feeding it Edgar Allen Poe produced the best results:
twelve vibrations of purple air. vortex of its absolute hurricane. monck flying footsteps of the picture all the police are concerned. silence on a year in an identity. materiality escaping the city was their white spot of printing apparatus all things only one supposes it were its mere duration that it in baltimore monument to its luxuriant nature asserts her person whom i not hesitate about five hundred paces to have not the principia of idiotic thought of curvature or metamorphosis these friends were in an earthquake and the then i felt my chair so rigidly of interest could not summon courage
Here's the TED version I just generated using the input.txt from one of the linked articles: https://github.com/samim23/TED-RNN/
educationists made around in our modern medicine. gq did make perfect mankind which the blue pill every week had infinite instant messaging that sucks when your food day that previous background is about being foreign interventions. psychopharmaceutical arsenal. attached to tail but a distribution units in our traditional estimates that men engineers who negotiate my whole province of long awkward to call it came back go deep conversations you grow fast when there been around the long lever systems.
entertaining anyway, and occasionally amazing
To generate output that contained meaning that you could actually engage with, you'd have to first generate the meaning and then turn that into text. That's a fundamentally different process to what is going on here. This is also why 'Turing Test' competitions involving chat bots are never going to give us artificial intelligence.
This sort of text generation is like a magician's trick where he appears to saw the woman in half and put her together again. I don't care how good the magician is at the trick, he's not going to be any help at all to a surgeon that's actually cutting someone in half and then stitching them back together again.
