undefined | Better HN

0 pointsbluegatty1mo ago0 comments

We don't even know what 'creativity' is, and most humans I know are unable to be creative even when compelled to be.

AI is 'creative enough' - whether we call it 'synthetic creativity' or whatever, it definitely can explore enough combinations and permutations that it's suitably novel. Maybe it won't produce 'deeply original works' - but it'll be good enough 99.99% of the time.

The reliability issue is real.

It may not be solvable at the level of LLM.

Right now everything is LLM-driven, maybe in a few years, it will be more Agentically driven, where the LLM is used as 'compute' and we can pave over the 'unreiablity'.

For example, the AI is really good when it has a lot of context and can identify a narrow issue.

It gets bad during action and context-rot.

We can overcome a lot of this with a lot more token usage.

Imagine a situation where we use 1000x more tokens, and we have 2 layers of abstraction running the LLMs.

We're running 64K computers today, things change with 1G of RAM.

But yes - limitations will remian.

0 comments

germandiago1mo ago

Maybe I do not have a good definition for it.

But what I see again and again in LLMs is a lot of combinations of possible solutions that are somewhere around internet (bc it put that data in). Nothing disruptive, nothing thought out like an experimented human in a specific topic. Besides all the mistakes/hallucinations.

bluegattyOP1mo ago

Yes, LLMs have a very aggressive regression towards the mean - that's probably an existential quality of them.

They are after all, pattern matching.

A lot of humans have difficulty with very reality that they are in fact biological machines, and most of what we do is the same thing.

The funny thing is although I think are are 'metaphysically special' in our expression, we are also 'mostly just a bag of neurons'.

It's not 'natural' for AI to be creative but if you want it to be, it's relatively easy for it to explore things if you prod it to.

germandiago1mo ago

> A lot of humans have difficulty with very reality that they are in fact biological machines, and most of what we do is the same thing.

I think we are far and ahead from this "mix and match". A human can be much, much more unpredictable than these LLMs for the thinking process if only bc looking at a much bigger context. Contexts that are even outside of the theoretical area of expertise where you are searching for a solution.

Good solutions from humans are potentially much more disruptive.

bluegattyOP1mo ago

AI has all of human knowledge and 100x more than that of just 'stuff' baked right it, in pre-train, before a single token of 'context'.

It has way more 'general inherent knowledge' than any human, just as as a starting point.

1 more reply

sigbottle1mo ago

I think the terminology is just dogshit in this area. LLMs are great semantic searchers and can reason decently well - I'm using them to self teach a lot of fields. But I inevitably reach a point where I come up with some new thoughts and it's not capable of keeping up and I start going to what real people are saying right now, today, and trust the LLM less and instead go to primary sources and real people. But I would have never had the time, money, or access to expertise without the LLM.

Constantly worrying, "is this a superset? Is this a superset?" Is exhausting. Just use the damn tool, stop arguing about if this LLM can get all possible out of distribution things that you would care about or whatever. If it sucks, don't make excuses for it, it sucks. We don't give Einstein a pass for saying dumb shit either, and the LLM ain't no Einstein

If there's one thing to learn from philosophy, it's that asking the question often smuggles in the answer. Ask "is it possible to make an unconstrained deity?" And you get arguments about God.

rimliu1mo ago

do they reason? Where was a video by AI researcher, that showed, that they do not reason but actually come with the result first and then try to invent "reasoning" to match it.

sigbottle1mo ago

I mean humans do that too, and I don't think it's very unjustified. The "we deduce from a deep base premise P down a chain of inferences" picture is extremely incomplete and has been challenged all over the place - by normal people, by analytic and continental philosophers, by science itself, etc.

Not trying to say that LLM's are equivalent to humans but that the concept of reasoning is undefined.

And the fact that their performance does increase when using test-time compute is empirical evidence that they're doing something that increases their performance on tasks that we consider would require reasoning. As to what that is, we don't know.

germandiago1mo ago

But humans verify things. AI just fools you and I would say it is the biggest problem.I have with AIs.

They give me stuff that I do not know whether to trust or not and what surprises I will find down the way later.

So now my task is to review everything, remove cruft. It starts to compete against investing my time to deep-think and do it thoughtfully from the get go and come up with something simpler, with less code and/or that I understand better.

1 more reply

j / k navigate · click thread line to collapse

0 comments

germandiago1mo ago

Maybe I do not have a good definition for it.

bluegattyOP1mo ago

Yes, LLMs have a very aggressive regression towards the mean - that's probably an existential quality of them.

They are after all, pattern matching.

A lot of humans have difficulty with very reality that they are in fact biological machines, and most of what we do is the same thing.

The funny thing is although I think are are 'metaphysically special' in our expression, we are also 'mostly just a bag of neurons'.

It's not 'natural' for AI to be creative but if you want it to be, it's relatively easy for it to explore things if you prod it to.

germandiago1mo ago

> A lot of humans have difficulty with very reality that they are in fact biological machines, and most of what we do is the same thing.

Good solutions from humans are potentially much more disruptive.

bluegattyOP1mo ago

AI has all of human knowledge and 100x more than that of just 'stuff' baked right it, in pre-train, before a single token of 'context'.

It has way more 'general inherent knowledge' than any human, just as as a starting point.

1 more reply

sigbottle1mo ago

If there's one thing to learn from philosophy, it's that asking the question often smuggles in the answer. Ask "is it possible to make an unconstrained deity?" And you get arguments about God.

rimliu1mo ago

do they reason? Where was a video by AI researcher, that showed, that they do not reason but actually come with the result first and then try to invent "reasoning" to match it.

sigbottle1mo ago

Not trying to say that LLM's are equivalent to humans but that the concept of reasoning is undefined.

germandiago1mo ago

But humans verify things. AI just fools you and I would say it is the biggest problem.I have with AIs.

They give me stuff that I do not know whether to trust or not and what surprises I will find down the way later.

1 more reply

j / k navigate · click thread line to collapse