Technology

I'm a human being and I'm pretty sure I am already not allowed to give legal or medical advice to anybody in new york or any other state.

Chat bots should never give medical advice. Chat bots dispense basic, standalone factoids, like "aspirin is a pain reliever." But they don't know or care about dosages, comorbid conditions or whether or not you live or die, so they won't ask follow up questions.

Here's a video about a simple question and how ChatGPT both refuses the correct answer, is shown proof of the correct answer, and even gaslights the user on the wrong answer.

In the US especially, medical professionals are overworked and simply don’t have the time and energy properly diagnose. If you have a more complex, chronic issue, there’s a good chance you’ll be waiting months at a time to see various specialists who are only going to spend about 10 distracted minutes thinking about your case and might not even have any useful insights, or they might misdiagnose you and make your condition worse. You basically have to do your own research and show them studies. If you’re a person of color or a woman, etc., there’s a good chance you won’t even be taken seriously. In an ideal world, it would work like it does on TV, but in the real world, it’s all about maximizing profits and the patients be damned. Sure, LLMs are unreliable, but they do at least provide ideas to research.

1. Make laws against chatbots.
2. Demand proof you are not a chatbot.
3. Surveillance capitalism.

The real target here is population control.

The lawmakers, which take billionaire money by the ton, who HAVE NEVER given a shit, suddenly, NOW, they want to protect the vulnerable. Abso fucking lutely laughable on its face.

If implemented, that would just ban chatbots that use large language models. It's not a terrible idea.

What would actually happen is that so-called AI chatbot systems would try to detect if someone is from New York and then try to exclude them from receiving medical or legal advice, fail, and then get sued and then pay a small fine, over and over again forever.

Laws like this are great for these companies. This is how they will justify removing access to useful information and putting it behind paywalls. But oh your need a prescription so now the insurance companies are involved (spoiler: they already are) and so you don't even have access to pay out the nose for medical information.

Then when Google search has been completely replaced with AI, you won't even be able to search for medical information.

Healthcare companies aren't about to provide anything for free.

Hell yeah, let's hold them accountable for disinformation. They'll be gone completely in a matter of months.

Want to get rid of that responsibility? Direct the user to the source. Oh wait, that's just a search engine.

I think a better solution is to ban techbros from giving serious economic or cultural advice and take computers away from business majors.

Would be nice if regular legal and health advice was in any way affordable though

I don't see how you police/enforce this. The technology is out of the bag, people will find ways to access. Do we need age/location verification for this now too? What if I'm running a local agent? I don't agree with this.

You bring a regulation - can you really enforce it?

This reads as a way to protect white collar industries from the effects of AI without addressing the root problem--that AI does not actually think, and that it is little more than a meat grinder full of scraped data.

I may have become too cynical but, as is often the case when you dig deeper, this sounds like the result of lobbyists trying to protect licensing rather than people.

We can be dumb, but we’ve been doing web searches for legal and medical advice for ages because it is too damned expensive and time consuming to go to professionals for every little thing. Not to mention, doctors have so little time for you that it is hard to get them to listen to the whole story to make connections between symptoms.

The LLMs already tell you that they aren’t licensed professionals and, for many, provide citations for their sources (miles better than your typical health website).

As a personal anecdote, my son was having stomach pain but was planning to tough it out. He checked with ChatGPT and it recommended he go to the ER. He did, and if he hadn’t, he would likely be dead now. He spent 3 days in the hospital having his bowels unobstructed through a tube in his nose.

There is value in people having that kind of information at their fingertips.

Regulation is absolutely needed, but I would rather they focus on protecting us from AI being used for military purposes, mass surveillance, etc. rather than protecting citizens from ourselves.