440
this post was submitted on 19 Jul 2024
440 points (98.5% liked)
Technology
59605 readers
3403 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Not the person you're replying to, but my main hangup is that LLMs are just statistical models, they don't know anything. As such, they very often hallucinate language features and libraries that don't exist. They suggest functions that aren't real and they are effectively always going to produce average code - And average code is horrible code.
They can be useful for exploration and learning, sure. But lots of people are literally just copy-pasting code from LLMs - They just do it via an "accept copilot suggestion" button instead of actual copy paste.
I used Copilot for months and I eventually stopped because I found that the vast majority of the time its suggestions are garbage, and I was constantly pausing while I typed to await the suggestions, which broke flow state and tired me out more then it ever helped.
I'm still finding bugs it introduced months later. It's great for unit tests, but that's basically it in my case. I don't let the AI write production code anymore
Sure, people use all sorts of tools badly, that's a problem with the user not the tool (generally, I would accept poor tool design can be a factor).
I really dislike the statement of "LLMs dont know anything they are just statistical models" it's such a thought terminating cliche that is either vacuous or wrong depending on which way you mean it. If you mean they have no information content that's just factually wrong, clearly they do. If you mean they dont understand concepts in the same way as a person does, well yes but neither does google search and we have no problem using that as the start point of finding out about things. If you mean they can get answers wrong, its not like people are infallible either (who I assume you agree do know things).
You can dislike the statement all you want, but they literally do not have a way to know things. They provide a convincing illusion of knowledge through statistical likelihood of the next token occurring, but they have no internal mechanism for looking up information.
They have no fact repositories to rely on.
They do not possess the ability to know what is and is not correct.
They cannot check documentation or verify that a function or library or API endpoint exists, even though they will confidently create calls to them.
They are statistical models, calculating how likely the next token is based on transformations in a many-dimensional space in which the relationships between existing tokens are treated as vectors in a process for determining the next token.
They have their uses, but relying on them for factual information (which includes knowledge of apis and libraries) is a bad idea. They are just as likely to provide realistic answers as they are to make up fake answers and present them as real.
They are good for inspiration or a jumping off point, but should always be fact checked and validated.
They're fantastic at transforming data from one format to another, or extracting data from natural language written information. I'm even using one in a project to guess at filling in a form based on an incoming customer email.
These three are all just the same as asking a person about them, they might know or might not but they cant right there and then check. Yes LLMs due to their nature cannot access a region marked "C# methods" or whatever, but large models do have some of that information embedded in them, if they didnt they wouldnt get correct answers anywhere near as often as they do, which for large models and common languages/frameworks is most of the time. This is before getting into retrieval augmented generation where they do have access to repositories of fact.
This is what I was complaining about in the original post I replied to, no-where have I or anyone else I've seen in this thread say you should rely on these models, just that they are a useful input. Yet relying on them and using them without verification is the position you and the other poster are arguing against.
Even for unit tests it needs to be taken with a grain of salt because they should describe what should be there and at best Copilot can describe what is there.
The overlap may or may not be there but either way it's a dicey proposition to allow Copilot to second guess the intent behind the code and make that guess the reference.
Indeed. I stopped using it altogether a couple months ago.