Ask HN: It is important”, a side effect of Transformers attention?

2 pointsslowmotarget2y ago1 comments

I constantly have to refrain GPT3 and 4 because they’re too often using expressions like “it is important” or “it is essential”.

I can’t think of a reason for the model to have been tried on texts that contain many occurrences of these expressions.

This is why I am wondering if it is a side effect of the attention mechanism built in the transformer algorithm. As the prompt and the output are recursively processed to figure out what really matters, maybe these expressions get embedded as a latent representation of the weights of the different concepts at play in the conversation context.

What do you think?

1 comments

turtleyacht2y ago

Maybe we can extrapolate Godel where a complete ruleset cannot be consistent, and a consistent ruleset cannot be complete.

Somewhere in there is the human ingenuity to adapt certain pathological patterns [1] that defeat (current) AI.

But it's not always obvious, and the composer sapiens must defy their own understanding of the rules to create a purposeful deviation (sometimes with help from other AI) [2].

The idea of sharing GPT prompts promotes the implication that results are deterministic. That helps QA, but humans love to play.

[1] Adversarial Policies Defeat Superhuman Go AIs: https://arxiv.org/abs/2211.00241

[2] Kellin Pelrine

j / k navigate · click thread line to collapse