Technology

Everything I have read about how LLM’s work suggest that you’re giving them too much credit. Their “thinking” is heavily based on studied examples to the point that they don’t seem capable of original “thought”.

For instance, there was a breakdown of the capabilities of some new imaging models the other day (one of the threads on DB0) that showed that none of the tested models were able to produce a cube balanced on a sphere because there were simply too few examples of a cubic object balancing on a spherical one in its learning model. When asked to show soldiers, the ones that could produce more accurate images could not produce accurate diversity because their improved rendering was due to it drawing from a more limited, and thus less creative, dataset. The result was that it kept looking like it had a specific soldier “in mind” rather than an understanding of soldiers in general.

These things would be trivial for even a child to do, though they may not be able to produce the “uncanny valley” effect that AI is good at. If a kid knows what a cube is, knows what a sphere is, and understands the request, they can easily draw a cube on a sphere without having seen an example of that specific thing before.

I agree that the parrot analogy isn’t correct, but neither is the idea that these things will learn from their own echo chamber in the way you have described. Maybe the idea of dreaming is more accurate—an unusual shuffling of input to make bizzaro results that don’t have any intrinsic meaning at all beyond their relation to the data that is being used.

That's not how this works. That's not how any of this works.

LLMs can't "talk to each other" as they don't think, they're more like a really complicated echo chamber. You yell your prompt into it, it bounces around and when the echo comes back you have your result. You could feed the output of one LLM into the input of another, but after a few rounds of bouncing back and forth you'd just get garbage out. Furthermore a LLM can't learn from its queries as the queries are missing all the metadata necessary to build the model.

I'd be interested in seeing an entire experimental community filled with just these AI LLM bots talking to each other all throughout the day and commenting on random news stories or posts, and seeing what sort of "AI culture" develops from it. And not in a satirical TOTALLY NOT ROBOTS sort of way, literally just restrict the community to LLMs and let them make X posts/X comments per day or something so it doesn't get out of control. Would their "culture" diverge heavily from what people were coming up with?

Otherwise, I loosely follow AI developments, but I've struggled to find any practical applications, it's more just a novelty at this point. The Image generation is fun, but the LLMs are near useless for most of what I've tried using it for, it's more hype than anything at this point.

I would only disagree on your “near future” prediction. I don’t totally disagree — maybe it will — but I’d caution that a lot of new tech gets a shit ton (metric) of hype when the “easy” problems are solved and the last 10-20% can take forever or just need too much money, power, resources, whatever to solve. (Like we see with self-driving cars where it’s tantalizingly close but edge cases are really hard.)

Technological progress doesn’t always continue in a straight line or improve exponentially. The S-curve is far more common. With A.I., your guess is as good as mine on where we are on that curve but don’t be shocked if progress stalls somewhere even if it’s temporary. (You could imagine a situation where the models are advancing faster than Nvidia or electricity infrastructure.)

Do you think an LLM can beat you in tic tac toe?

I'm on Lemmy just to poison the AI 🤣🤣.

Waiting for the AIrmaggedon..

What's the point of talking to yourself? Can you get better output by running it by yourself?

LLM's are only one kind of AI program. How smart would we be, if we only used the speech areas of our brains? It's important to be able to complement language with other kinds of thinking.

The problem with neutral network technology is the vast computational resources it requires to learn. The brain also requires enormous computing power, but brains grow organically and can efficiently run on corn and beans.

To compete, AI systems will need to become much more efficient in the way they learn and process. Venture capital only goes so far. The subscription fees for ChatGPT don't earn enough money to even cover the electricity costs of running the system.

This thread is a great example of the Dunning Kreuger effect in motion