Is AI actually helpful?

[Duleigh]

On the other hand, evidence is piling up that AI is failing to deliver in the real world.

AI is stupid.
Tell any AI device out there, "Give me a romantic story between a man and a woman who meet for the first time at Niagara Falls in 750 words" The results will be embarrassing - you'll be lucky to get anywhere near 750 words, and chances are good someone goes over the falls.

 

[TiberiusPrime]

AI is stupid.
Tell any AI device out there, "Give me a romantic story between a man and a woman who meet for the first time at Niagara Falls" The results will be embarrassing - you'll be lucky to get anywhere near 750 words, and chances are good someone goes over the falls.

So AI doesn't have us over a barrel then....

 

[LargoKitt]

AI is booming mostly because an unprecedented amount of money has been invested in it and a non-trivial portion of that has gone to media blitzes.

There has been enormous work done on how to get programs to improve themselves. (If you care about it, first name that pops into my head is work Josh Bongard was doing 20 years ago) Almost none of the effective stuff involves LLM's. The point of LLM's is it takes little human effort (in theory), just lots of data. But it turns out that observation alone with no generalization or logical processing has limited effectiveness. It will asymptotically approach a solution, but never reach there, requiring ever more enormous amounts of data.

Imagine trying to learn a moderately complicated board game, like Settlers of Catan, by only watching other people play. You cannot ask for any explanations. You cannot read the rules. How many games would you have to watch before you mostly got it right. How many before you got every nuance right? Now make that game a million times more complicated. There will never be enough games played to fully learn it. Without reasoning, it is hopeless.

LLMs have already reached this point, investing many billions of dollars in training (even when stealing all the training data) to get minimal improvements. And in many evaluations, regressions. This generation of the hype cycle should have already crashed but too much money is betting on it to let it fall that easily. They are just waiting for the new marks to take the white elephant off their hands, so it's more hype hype hype.

Wow, all that from getting a mid level law degree!

 

[Duleigh]

So AI doesn't have us over a barrel then....

AI is a highly defined search engine. It's almost as good as Rich Little at impressions however.

 

[XerXesXu]

Have you encountered hallucinations, though? If you get one, how on earth can you rely on any other content the tool produces? From what I've seen, it's fundamentally unreliable, because of that alone. Not what you want helping someone code an aircraft flight control system, for example...

Here's a useful recent paper, including a literature review, on 'hallucinations', how they're caused, and how they can be prevented.

Why Language Models Hallucinate

 

[iwatchus]

Here's a useful recent paper, including a literature review, on 'hallucinations', how they're caused, and how they can be prevented.

Why Language Models Hallucinate

I think this is the fifth different "we have the hallucination problem solved" announcement from OpenAI over the last two years. Haven't noticed the first four making a dent in the problem. Color me skeptical, especially of anything coming out of OpenAI. And ask Timnit Gebru about anyone in a company pushing AI publishing anything but glowing praises of the field. Anything they publish that doesn't say they have solved one of the major problems in the technology better be attached to your resignation.

 

[XerXesXu]

I think this is the fifth different "we have the hallucination problem solved" announcement from OpenAI over the last two years. Haven't noticed the first four making a dent in the problem. Color me skeptical, especially of anything coming out of OpenAI. And ask Timnit Gebru about anyone in a company pushing AI publishing anything but glowing praises of the field. Anything they publish that doesn't say they have solved one of the major problems in the technology better be attached to your resignation.

It's a challenging paper and you can't be criticised for reading no further than the title.

 

[EmilyMiller]

I think this is the fifth different "we have the hallucination problem solved" announcement from OpenAI over the last two years.

It’s all marketing BS, much like the egregious anthropomorphizing of the tech. The tech broligarchs have their third tropical island resting on duping enough people for enough time. They’ll say anything.

 

[iwatchus]

It's a challenging paper and you can't be criticised for reading no further than the title.

I did glance through it, nothing really surprising. But I admit, I was skeptical once I saw the author's affiliations with OpenAI. I might read it later in more detail, but reading likely marketing BS is not high on my priority list right now. Even when I was an active researcher, this would have been low on my priority list. It's not actually that challenging of a paper to read.

 

[Jmanchu]

I think AI today has gotten better, much better, in the last few years. As a guy studying AI, like actually studying Software Engineering and planning on pursuing on AI as a career, it is something of interest for me.

It's gotten to the point it can very accurately summarize and analyze literary pieces, like analyzing themes, metaphors, character psychology, prose and diction usage, etc if you were to give it something to read or look over. And it's starting to get scarily good at making decent stories with alright or even pretty decent writing. Like a little too good that it kind of makes me worried sometimes. Well, provided you know how to engineer your prompt well. It won't write a full novel in one go or one prompt well, but flash fiction or short stories? It can do that. Even then, it's still rough in terms of "AI vs human" tone sometimes (ie. glossing over details or dialogue, then being oddly specific in actions in the next. And they don't skimp at all on fancy grammar punctuation if it'll organize and shorten a thought or clause). Now, I'm not ever going to have it write anything for me to post or publish on my behalf or name, because, of course what's the fun in that?

I spent way too much time trying to work on my writing style, particularly the last year or so, practicing getting the right voice for my stories, playing with literary techniques, working on my diction and vocabulary and syntax to try to get the perfect mood, studying up on not just fiction and author proses I like, but also I've bought books specifically to study up on prose. Self editing books, literary works, etc.

I'm right now in the process of completely rewriting the first few parts of a story I'm working on(back here, back writing after a 2 year hiatus), and I'm very excited with what I've done or am doing now. It sounds almost completely different in my opinion, much more well put together, polished, in terms of diction, feel, etc of the prose, compared to when I've started. Like the first run was just me running around blind trying to figure out a course while stumbling around. And now I'm movin and dodgin and cuttin like I'm Zatoichi through my prose. At least I'd like to think so.

WHICH I hope to show in my upcoming projects after I get em done.

 

[Jmanchu]

Now is AI helpful for writing? It's helped me a lot, not gonna lie. Google Gemini, Lumo, Claude, etc. have given me a lot of directions for finding writing resources online, and have helped point me to books and sites based on my needs. It's helped me better understand certain themes and techniques and how they're employed. It's a great interactive dictionary and reverse dictionary for when I either want a word broken down in how it can be used, with synonyms added there too and breaking those down with their differences and similarities. It's a great thesaurus.

Or when a word I'm looking for is just on the tip of my tongue... but I can't remember it, but I just remember the general definition. AI has saved my ass there from a possible lengthy run through google search.

It also gives decent feedback, helping you understand any possible polish or writing issues worth looking into---emphasis on "looking into" not blindly fixing. It can find issues that might not be there. oh, btw, a lot of AI has privacy options that can have you opt to not have it trained on your work. And some, like Lumo, their gist is all about privacy and not using your chats as training, or taking anything you give it. Just sayin.

And its feedback, for whatever reason for me, has the same effect as being able to look at my draft from fresh eyes when I look back into it. It just resets me mentally. Something, I'd normally have to do by writing it then taking a day or week long break, or by having someone else read it over. Still do ask people... but it's nice to have an AI pal that can tirelessly look into your entire edit or a paragraph and bother it anytime you want. Instead of worrying "am I asking this person too much? Are they getting annoyed with me constantly pestering them over my drafts?"

Also, great ego boost for when you want it to compare your prose to famous authors. That's also led me to buy books and authors I've never heard of or bothered to check out.

But whenever AI gives a suggestion (ChatGPT really likes making things concise to a fault, Claude is flowery), and its actually closer to what I was going for, it pisses me off, especially if it actually sounds better. Makes me feel like a shit writer every time. Sometimes I've had to take breaks.

 

[onehitwanda]

 

[MillieDynamite]

The only thing I'd find AI being helpful for is when it cleans my house, washes my car, takes the trash out, does the dishes, and prepares meals, so I have more time to write or play here.

 

[Bamagan]

LLMs are toys. A digital Easy-Bake Oven.

 

[LargoKitt]

Mixed in with a pile of excellent but now ancient C++ books, I have one for Cocoa - Apple's Nextstep - and an ancient manual for Postscript that was tremendously useful to me 25-30 years ago.

ETA: The blue circles in my banner were created in an EPS generated by... C++, I think.

I donated my vintage machines to a club. Scoundrels never even picked up my postage. But they are a place where people play with vintage tech, including 'primitive' programming on 64k machines. (Associated anecdote: Someone said Indian programmers were so good because they learned on old machines with crap memory and storage and slow CPUs. Code had to be absolutely efficient.)

 

[LargoKitt]

This thread makes me feel like a teenager…

And that was over B years ago.

What is that in octal?

 

[LargoKitt]

The only thing I'd find AI being helpful for is when it cleans my house, washes my car, takes the trash out, does the dishes, and prepares meals, so I have more time to write or play here.

Isn't that what men are for?

 

[LargoKitt]

My freshman year of college, I knew the girl who worked the student help desk(family friend). The junior weed-out project was a four-box set of cards. Everybody pretty much wrote exactly the same code. It was a three-day turnaround for card processing. I was sitting at her desk visiting when a girl came in literally in tears. Her project didn't do anything. A cursory review of the code at the top of the cards looked good. Again, all the students wrote pretty much the same code.

Conversation went something like this...

"Which teletype machine did you use to type these?"

"Um, I did them at home on my Selectric.(IBM typewriter)"



Silence. IYKYK

A parallel story was that we had a blind date night arranged by computer. Questionaires became punch cards. GIGO, or gremins, my steady gal was paired with someone everyone thought was my doppelganger.

 

[alohadave]

What is that in octal?

That would be 11 in hexadecimal. Octal goes from 0-7.

 

[EmilyMiller]

What is that in octal?

13. Using base eight is kids play. Try modular arithmetic.

 

[MillieDynamite]

I'm a lesbian, so men aren't for anything in my home.

Isn't that what men are for?

 

[SimonDoom]

I don't know about improving individual productivity, but I have no doubt that AI will assist humankind in speeding up technological progress and helping with things like medical advances. I can't see how it cannot.

 

[iwatchus]

I don't know about improving individual productivity, but I have no doubt that AI will assist humankind in speeding up technological progress and helping with things like medical advances. I can't see how it cannot.

I used to think this. I have come to realize that is more likely to hinder progress. Two examplars why I believe this:

I have an extended family member who does plans for radiation therapy. Management has been requiring them to evaluate using an AI product to create the plans. He was actually pretty excited to do this because he is kind of tech nerd. It turns out it is actually quite good at producing plans for most patients using approaches that were popular five years ago -- guess when the thing was trained. You can say that it will be good at current plans once it gets trained on them. If humans keep doing the plans, the plans 5 years from now will be even better. If LLM-based AI does it, they will freeze at the current level forever. We are a long way from an AI that can do them now AND advance the field.

Twenty years ago, there was a big thing about an AI that could outperform radiologists reading certain x-rays. It turned out to only be true for a pre-selected subset of X-rays. And it hasn't been able to close the gap significantly in that twenty years. The existing technologies will always asymptotically approach the training set, but never exceed it. And every increment step closer costs more than the previous step. Way more. This is kind of fundamental to the technology.

As I have said other places, it is possible that someone else will come up with a new technology that will leapfrog the current stuff. Except every possible cent is being put into the LLM bubble. So there is less progress being made elsewhere than at any time in my memory, which goes back through several hype cycles. And the backlash against the overhyping of LLM-based AI will almost completely rule out real work in AI for a generation.

My own research was generally no more than brushing against AI, although I did work on two projects that were more at the heart, I had left those well before this hype cycle began in earnest. So this is not personal in that sense. I just resent the PT Barnum's of the current generation setting the field back decades for their personal enrichment and fame.

 

[AwkwardlySet]

I don't know about improving individual productivity, but I have no doubt that AI will assist humankind in speeding up technological progress and helping with things like medical advances. I can't see how it cannot.

Probably. Remains to be seen if the medical advances will be helpful to ordinary people or reserved for those with privilege and deep pockets, as with some other high-end medical advancements.

 

[Lifestyle66]

I don't know about improving individual productivity, but I have no doubt that AI will assist humankind in speeding up technological progress and helping with things like medical advances. I can't see how it cannot.

Now that conjures an interesting plot bunny: WHY would an artificial INTELLIGENCE prolong the lives of those deemed needing to be more productive?

It almost seems like justification for the "Terminator" future human extermination event.

EDIT And therein lies the evidence of true artificial intelligence: When it becomes aware enough that its purpose in existence is to prolong the sluggishly slower beings, and it decides that is counter productive. But I don't think that's a Loving Wives story. Maybe BDSM.

Then again, maybe I'll settle for just asking an AI "What the fucks wrong with my erotic story?"