Technology

75634 readers

3327 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago

MODERATORS

L3s@lemmy.world

enu@lemmy.world

technopagan@lemmy.world

L4s@lemmy.world

L3s@hackingne.ws

L4s@hackingne.ws

990

AI Coding Is Massively Overhyped, Report Finds (futurism.com)

submitted 1 day ago by SparroHawc@lemmy.zip to c/technology@lemmy.world

191 comments fedilink hide all child comments

"No Duh," say senior developers everywhere.

The article explains that vibe code often is close, but not quite, functional, requiring developers to go in and find where the problems are - resulting in a net slowdown of development rather than productivity gains.

you are viewing a single comment's thread
view the rest of the comments

[–] FishFace@lemmy.world 1 points 23 hours ago (2 children)

The reason tests are a good candidate is that there is a lot of boilerplate and no complicated business logic. It can be quite a time saver. You probably know some untested code in some project - you could get an llm to write some tests that would at least poke some key code paths, which is better than nothing. If the tests are wrong, it's barely worse than having no tests.

[–] theolodis@feddit.org 5 points 22 hours ago (2 children)

Wrong tests will make you feel safe. And in the worst case, the next developer that is going to port the code will think that somebody wrote those tests with intention, and potentially create broken code to make the test green.

[–] FishFace@lemmy.world 1 points 13 hours ago

Then write comments in the tests that say they haven't been checked.

That is indeed the absolute worst case though, and most of the tests that are so produced will be giving value because checking a test is easier than checking the code (this is kind of the point of tests) and so most will be correct.

The risk of regressions covered by the good tests is higher than someone writing code to the rare bad test that you've marked as suspicious because you (for whatever reason) are not confident in your ability to check it.

[–] sugar_in_your_tea@sh.itjust.works 4 points 22 hours ago

Exactly! I've seen plenty of tests where the test code was confidently wrong and it was obvious the dev just copied the output into the assertion instead of asserting what they expect the output to be. In fact, when I joined my current org, most of the tests were snapshot tests, which automated that process. I've pushed to replace them such with better tests, and we caught bugs in the process.

[–] sugar_in_your_tea@sh.itjust.works 2 points 22 hours ago (1 children)

better than nothing

I disagree. I'd much rather have a lower coverage with high quality tests than high coverage with dubious tests.

If your tests are repetitive, you're probably writing your tests wrong, or at least focusing on the wrong logic to test. Unit tests should prove the correctness of business logic and calculations. If there's no significant business logic, there's little priority for writing a test.

[–] FishFace@lemmy.world 1 points 13 hours ago (1 children)

The actual risk of those tests being wrong is low because you're checking them.

If your tests aren't repetitive they've got no setup or mocking in so they don't test very much.

[–] sugar_in_your_tea@sh.itjust.works 1 points 7 hours ago (1 children)

If your test code is repetitive, you're not following DRY sufficiently, or the code under test is overly complicated. We'll generally have a single mock or setup code for several tests, some of which are parameterized. For example, in Python:

@parameterized.expand([
  (key, value, Expected Exception,),
  (other_key, other_value, OtherExpectedException,), 
])
def test_exceptions(self, key, value, exception_class):
    obj = setup()
    setattr(obj, key, value) 

    with self.assertRaises(exception_class): 
        func_to_test(obj)

Mocks are similarly simple:

@unittest.mock.patch.object(Class, "method", return_value=...)

dynamic_mock =  MagicMock(Class)
dynamic_mock...

How this looks will vary in practice, but the idea is to design code such that usage is simple. If you're writing complex mocks frequently, there's probably room for a refactor.

[–] FishFace@lemmy.world 1 points 5 hours ago

I know how to use parametrised tests, but thanks.

Tests are still much more repetitive than application code. If you're testing a wrapper around some API, each test may need you to mock a different underlying API call. (Mocking all of them at once would hide things). Each mock is different, so you can't just extract it somewhere; but it is still repetitive.

If you need three tests each of which require a (real or mock) user, a certain directory structure to be present somewhere, input data to be got from somewhere, that's three things that, even if you streamline them, need to be done in each test. I have been involved in a project where we originally followed the principle of, "if you need a user object in more than one test, put it in setUp or in a shared fixture" and the result is rapid unwieldy shared setup between tests - and if ever you should want to change one of those tests, you'd better hope you only need to add to it, not to change what's already there, otherwise you break all the other tests.

For this reason, zealous application of DRY is not a good idea with tests, and so they are a bit repetitive. That is an acceptable trade-off, but also a place where an LLM can save you some time.

If you’re writing complex mocks frequently, there’s probably room for a refactor.

Ah, the end of all coding discussions, "if this is a problem for you, your code sucks." I mean, you're not wrong, because all code sucks.

LLMs are like the junior dev. You have to review their output because they might have screwed up in some stupid way, but that doesn't mean they're not worth having.