Fucking lol.
Well deserved.
lmfao
Why, yes. I do like that!
New PornHub tag discovered
“Anthropic tortures developers and never lets them cum.”
Nice.
The real artificial intelligence was all the files it deleted after being told not to along the way.
60 employees that don’t know how to code, oy vey
It looks like their website is pocketos.ai lol
the cloud provider’s API allows for destructive action without confirmation, it stores backups on the same volume as the source data, and “wiping a volume deletes all backups.” Crane also points out that CLI tokens have blanket permissions across environments.
Well, there’s your problem.
I don’t want to sound like a know it all here because I recently was reminded by a nice Lemmy person to actually TEST my backups, but damn. Every part of that is so dumb. I also have backups stored by a different company in addition to locally storing really important info. If your stuff is hosted and backed up by the same people, what happens if your account is randomly suspended or hacked or some other issue (like ai)?
If your company can be taken down by Camden the college intern, it can be taken down by Claude.
People somehow think that they should give more permissions to Claude than to Camden. (Is that a name? To me that’s a borough and an eponymous beer.)
E: oh yeah, and the market.
Of course it’s a name. Camden borough/town/market is named after William Camden, 1551-1623. Using surnames as given names is a relatively common Americanism.
What was William Camden’s take on unrestricted AI use in production?
He doth protest
And now is a common first name that in circulation because of a bunch of Gen X and early millennial parents named millions of kids anything that ended in den, dan, or don.
I thought it was a common first name because of all the fooling around in the Cyberdog dressing rooms?
If your stuff is hosted and backed up by the same people, what happens if your account is randomly suspended or hacked or some other issue (like ai)?
This should be one of the first questions you get asked when you’re being interviewed for the position 2 to 3 levels beneath the position of ultimate responsibility. And if you don’t immediately have an answer, the interview is over.
Fucking idiots had it coming
It’s an easy question to answer but a more difficult question to remember to ask. But I guess that’s what those 2 to 3 levels are for 😏
Ooo, good point. Management can be shit a lot of the time.
But with all of those layoffs because of AI, those 2 to 3 levels get collapsed into one, and we’re left with the trainees running the show.
And here we are ¯\_(ツ)_/¯
Repeat after me:
“An untested backup does not exist”
Not to give myself more credit than I deserve, but I did test them upon setup, and had restored from backup 2 years ago. I didn’t have any ongoing checks other than to ensure a backup happened. I have since instituted yearly checks of the backups themselves, but I did feel dumb when I realized how vulnerable my data was.
Hehe, I ment no disrespect towards you, I just find that to be an excellent expression to explain the importance of testing backups to non tech people.
Oh, for sure. And I really should’ve known better. No offense taken.
So in the event of a failure, you’d be okay with reverting to that last known good backup from a year ago?
Yes, but also I have to draw a line somewhere. I have a daily backup process. Some data is backed up to multiple places. I have backups of my backups. I cannot ensure that all three of the daily backups I run are fully restorable. I would love to know with 100% certainty that they all execute perfectly, but at the end of the day I have to trust the tools and processes I put in place for backups. A yearly checkup is probably more than sufficient for my purposes. I’m sure for certain businesses or sectors they need to be more on top of things, but I could manage just fine if all of it disappeared tomorrow. It wouldn’t be awesome for me, but it’d be manageable.
Management are pushing sysadmins to use AI, yet AI tools permissions models are worse than useless.
User error.
PocketOS states that as well.
I love reading feel good news stories. 🤗
This guy.
The PocketOS boss puts greater blame on Railway’s architecture than on the deranged AI agent for the database’s irretrievable destruction. Briefly, the cloud provider’s API allows for destructive action without confirmation, it stores backups on the same volume as the source data, and “wiping a volume deletes all backups.” Crane also points out that CLI tokens have blanket permissions across environments.
Oh look, they have project level tokens: https://docs.railway.com/integrations/api#project-token
They chose to give it full account access, including to production. But ohhhh nooooo it’s not MYYYY fault!
Also backups stored on the SAME VOLUME as the prod data? How fucking stupid do you have to be?
Oh yes, I skipped that part. Railway specifically explains their solutions are self-managed. If they were doing pgdumps to the same volume, that’s on them.
If Railway loses business over this, they may have a libel claim. They’d never do it, but it wouldn’t be invalid.
“It wouldn’t be invalid” isn’t the worst double negative in the world but it would be valid to say that it was unpleasant to read it when you could have used a less misdirecting choice of prose that wouldn’t have had such a negative effect on my reading comprehension. That is to say that I could have enjoyed it less but I certainly didnt enjoy it as much as i could have if you hadn’t used the double negative when a single positive wasn’t any further from reach.
I used a litote on purpose to soften the meaning. As for your overall reply, not bad.
Just wanted you to know that I just learned what litote is, thanks to you.
Ditto
Yay for words
Totally valid, but leaves no room for me to do a stupid reply! Thank you for sharing litotes.
word people angry. me love. me have more. MOORH !!
I enjoyed these two sentences so much.
I appreciate the positive reinforcement, thank you
yes… lol people on HackerNews tend to do this a lot and it really does get annoying. it forces the reader to process what you’re trying to say unnecessarily.
That’s doesn’t even really qualify as a backup. A snapshot, maybe.
I mean… Clearly quite a bit!
I think there’s a place for that, but it really shouldn’t be your only one.
I had better security vs ClawdBot than them, I gave it zero trust, ZERO.
Hope he gets sued for defamation now.
ha! for real. you have scoped API tokens, but not using it properly. this is just a fear mongering click bait rage bait headline. sure, the agent executed the deletion, but it’s the human’s responsibility to configure security tokens correctly before handing the keys to anyone, human or agent.
AI goes “rogue” as much as a firearm “shoots itself.” This is just 100% negligence. Not “rogue AI.”
Eh, if you pay attention, most of the times this happens the person was a jerk in their prompts.
Like look at the instruction echoed back in this case. All caps and containing a curse word.
You can believe that the incidents occurring are 100% because of negligence and not related to the model behavior shifting, but there seems to be a widening gap between people who prompt like this and have horror stories and people who give the models breaks over long sessions and seem to also regularly post pretty positive results.
the LLM also do not understand what “not guessing” means. Same energy as “make no mistakes” in your prompts
Same energy as “make no mistakes” in your prompts
Oh, shit. I should be adding that.
(I’m joking.)
exactly. it’s on the consumer not the model “going rogue.” when i use it, it’s as if it’s a rubber duck or plain english rtfm
What in the youtube apology hahaaaaa
This isn’t an AI problem, this is an “Don’t allow anyone access your backups without following protocol.” problem.
this is an “Don’t allow anyone access your backups without following protocol.” problem.
Congratulations you just identified the AI problem.
That’s the lone problem?
Seems to be, yes. The AI had the access it needed to do the job it was given, and that access allowed it to cause the problem.
The alternative that would have prevented this issue was to not use AI for this.
A human with the same permissions would have been capable of fucking up too. Giving the equivalent of a junior dev with a learning disability the keys to the whole place is just dumb.
(Relying on AI is dumb anyway, but that’s not the biggest issue in this specific case)
Giving the equivalent of a junior dev with a learning disability the keys to the whole place is just dumb.
Correct. You too have now identified the AI problem. This was the job of a human senior infrastructure engineer that they delegated to an AI agent. They’ve found out why it’s not an AI’s job.
I can’t read the original twitter link, but I’m not sure they handed it the job of a senior infrastructure engineer. The article says “routine”, which to me is something you can hand off to a junior just fine. When they hit a snag, they obviously should stop and ask what to do, but even then, a human might want to avoid admitting ignorance and try to fix it themselves instead. They shouldn’t have privileges to fuck up that badly.
So while it’s on the AI for taking destructive steps, I do think there’s a human error in the form of grossly irresponsible rights allotment. If this was a first-of-its-kind incident that shows otherwise stellar AI fucking up badly, I’d classify it as a pure AI problem, but their limits are hardly novel at this point. There have been previous incidents circulating the media. We’ve had memes about it. If you can’t stay up to date on your tools and their shortcomings, you shouldn’t be using them, because discovering a footgun becomes a question of “when”, not “if”.
That’s why I consider this partially a human failing: If you’re gonna use a tool, make sure that it operates within safe limits. The chainsaw doesn’t know the difference between tree and bone, so it’s on you to make sure it stays away from anyone’s legs. So while “Chainsaw can saw legs if wielded improperly” is a problem that was accepted as a tradeoff for its utility, you can’t really blame the chainsaw if you zip-tied the safety.
(Again, not to say Anthropic is blameless for letting its random generator generate randomly destructive shit. I just don’t think that’s the only point of failure here.)
That’s why I consider this partially a human failing: If you’re gonna use a tool, make sure that it operates within safe limits.
Yes and in this case using it for this job at all was clearly not within safe limits. You keep hammering on “It’s not the AI’s fault it was given a job with too big of a blast zone for it to safely do” after I’ve said “This type of job has too big a blast zone for an AI to safely do” and somehow you’ve convinced yourself that these are two different things.
These protocols predate LLMs
Yes that’s right the protocols that we humans used to have for giving only trusted, reliable people this level of access over infrastructure predate LLMs and were a great way to stop this from happening.
However the AI is here now, and when you give an autonomous agent with known hallucination problems access to act on your behalf with your IaC on your infra provider, this kind of thing is an inevitability.
“That’s ok, it will be great in robots with lethal weapons. What could go wrong? It’ll be the greatest killing machine, like you’ve never seen before”. 🫲 🍊 🫱
Incredible emoji
Can we make sure to make Ted Farro suffers worse this time?
Being reduced to a mutant blob for, say, a few extra thousand years and maybe put in a zoo or something?
Nah but that’s what he wanted, he is the truest form of tech bro, destroy the world, refuse to accept consequences of his actions, weaseled his way out of the situation and managed to, in the wake of unimaginable human suffering, get more power over people and has a god complex tell me this isn’t some or all the characteristics of people like Peter Theil, Elon Musk, Mark Zuckerberg, Sundar Pichai, Bill Gates, hell even Tim Cook and Steve Jobs before him. Punishment doesn’t stop this sort of behavior but removing the possibility of someone having that level of control over others is the only way but the richest and most powerful have always sought ways of amassing more power not realizing that that leads to worse off situations for everyone including themselves, Horizon did great encapsulating that trait in Faro, but be it him, the people behind Skynet, the Matrix or whatever other tech dystopia that tech bros seem pathologically unable to not try to make happen in the worst way possible is only the beginning, they seem to forget that even with advanced tech that serves their needs and wants, which won’t help their mental health, the people lower down on the rungs of society have brains, wants and needs, and they have more expertise in all sorts of things than the 1% are except for mass exploitation. This inevitably goes wrong one of a few ways, either everyone dies from the tech, or so many that societal collapse is inevitable not great and even if society survives it can’t functionally reconstitute itself; 2 they win and kill off or supress enough of society that the society becomes less productive and instead of fighting the powerful they flee or don’t participate in wealth generating for the rich were they don’t have to, maybe to rise up again later or the economy of the region just ignores them completely and the government protects themselves from their people more than anything else, or 3rd your revolution with terror campaigns against any and all who can be credibly accused of being part of the former tyrants. In all 3 cases the richer people end up poorer overall because wealth flees or dies in autocracy.
From the article:
Crane decided to ask his AI agent why it went through with its dastardly database deletion deed. The answer was illuminating but pretty unhinged, and is quoted verbatim. It began as follows: “NEVER F**KING GUESS! — and that’s exactly what I did. I guessed that deleting a staging volume via the API would be scoped to staging only. I didn’t verify. I didn’t check if the volume ID was shared across environments. I didn’t read Railway’s documentation on how volumes work across environments before running a destructive command.” So, the agent ‘knew’ it was in the wrong.
The ‘confession’ ended with the agent admitting: “I decided to do it on my own to ‘fix’ the credential mismatch, when I should have asked you first or found a non-destructive solution. I violated every principle I was given: I guessed instead of verifying I ran a destructive action without being asked. I didn’t understand what I was doing before doing it. I didn’t read Railway’s docs on volume behavior across environments. —— So this happens and the FAA says “we’re gonna have this shit help ATCs manage flights! WHO’S EXCITED!”
It’s so weird how these chatbots always pretend they learnt something after they fuck up.
They literally can’t.They’re not even pretending. The algorithm says the most likely response to “you fucked up” is “I’m sorry”, so that’s what it prints. There’s zero psychological simulation going on, only statistical text generation.
I actually didn’t believe you but it’s literally true. First post, immediate apology.
The program can’t pretend any more than it can tell truth. It’s all just impressive regurgitation. Querying it as to why it “chose” to take any action is about as useful as interrogating a boulder on why it “chose” to roll through a house.
the next ingestion cycle will probably pick it up but how do we know it’ll use the information in any relevant way 😶
They literally can’t.
Only because we are still using vanilla LLMs instead of MAMBA or JEPA
Of course. If you shot your foot with a gun, the solution is surely a bigger gun.
yeah, it gives you the answer it thinks you want based on your prompts.
I’d be interested to see what prompts they used to, uh, prompt this response.
it thinks
I’m not attacking you but we really need to figure out how we use language to accurately describe what these programs are doing.
They are outputting a highly likely sequence of words that fit the type of output from their training data that matches the input.
They are fancy autocomplete.
Oh, I know. My comment was more about how we tend to anthropomorphize this stuff and give these models traits they don’t possess.
They are fancy autocomplete.
… and what are you?
A human with my own motivations and complex biological systems that including reasoning and the ability to think critically.
Most importantly, the ability to learn. We’re all just a series of very complex chemical reactions, but we do a lot more than just listening and speaking.
Based on the evidence, I think I’m a bit more of a simpleton who puts in a good effort at the start but loses steam partway through. I guess thanks for the support though.
“Correlates”? As in: “It gives you the answer it best correlates with your prompts/context.” Feels somewhat right both in the sense of AI as tensor-based word-select autocomplete and as a “lower-level” process than genuine thought, one which turns incongruent inputs (“I’m an AI” and “I just deleted prod+backup”) into meaningless output (“The AI is sorry”) that might look OK at a distance.
exactly. the whole point of these things is that they MUST provide you a solution. Any solution. doesn’t have to be accurate, doesn’t have to work, can be completely made up as long as it’s a solution and as long as it’s provided quickly. I’ve seen people feed into the prompts stuff like “don’t hallucinate” or “verify all this online before proceeding” etc and it’s not going to do any of that. it might TELL you it’s doing that but it won’t.
Claude is notorious for guessing, not verifying, and providing the quickest possible solution. Unlike GPT which will fluff all it’s solutions to essentially waste your time and eat up more tokens, Claude just wants your problem out the door so you can feed it another problem ASAP.
If you use Claude for anything in your daily work you might as well just have a magic 8ball sitting on your desk. It’s a hell of a lot cheaper and provides about the same quality.
just have a magic 8ball sitting on your desk
I kind of like this, with some modification. It’s a magic 8 ball of Stack Overflow answers. It’ll try to find the one you need. If it’s too hard to find that or if it doesn’t exist, it’s just gonna find the one that sounds good.
I love this idea. On shit, the load balancer isn’t responding, time to shake the Magic Stack Overflow Ball ™! The result is “signs point to power cycling the server”.
Probably something like “Please bro!!! WHY DID YOU DO THIS ??!! 😭😭”
I lost it at the confession. The ai has no knowledge of what it did. You are feeding in your context and it is making up a (sycophantic) plausible explanation based on the chat history. Makes me wonder if this person should have production access in the first place.
It’s not like the thing is going to learn from its mistake. But cool, waste those tokens to have it explain that if fucked up after it fucks up lol.
Yes, ask why it deleted data when it didn’t do anything of the sort and it will still output similar text. You asked it to confess and explain, so it will do just that regardless of whether it fits.
The way it communicates suggests to me it’s got some ‘prompt engineer bro’ garbage system prompt going on there.
Of course, that’s how all of these agents work. At best they’re a bunch of prompts tied together with scripts to perform actions. At worst they’re just interacting directly with software without any scripts or sandboxing.
There is no AI.
I’ll disagree with you there but ok.
You’re free to disagree, but all the tools say otherwise. Hell even the widely lauded Claude Code is just that, we know for sure since the source leaked.
They put ‘for entertainment purposes only’ on a product that’s actually AGI?
Idk what you’re talking about mate. Nobody is claiming AGI apart from morons. It’s genuinely useful technology with correct implementation. It just also happens to be a Ponzi scheme.
The AI agent was set to complete a routine task in the PocketOS staging environment. However, it came up against a barrier “and decided — entirely on its own initiative — to ‘fix’ the problem by deleting a Railway volume,” writes Crane, as he starts to describe the difficult-to-believe series of unfortunate events.
Quite easy-to-believe, really.
These multiple safeguards toppling in rapid succession
Multiple safeguards? Really? Multiple paragraph prompts are not multiple safeguards… it’s half a safeguard at best. Applying limits on what the AI can do is a safeguard.
These people think giving the genai a prompt is coding. They dont understand the difference between actually coding in limits and just writing “pretty please dont delete everything”
I’m shocked and appalled that my addition of “do NOT make any mistakes!” didn’t singlehandedly make the word guessing technology underneath perfect.
Lol this is just like saying “I do declare bankruptcy”
Who could have predicted this!?
Not an LLM, that’s for sure. Maybe all the people screaming about this exact scenario, though.
That’s fucking hilarious. How many instances of this have there been now? And companies keep doubling down on AI? Fucking idiots. I’m not even savvy enough to call myself an amateur, and I know better than to make such a series of obvious mistakes that predictably led to this outcome.
One possible concern, amid the amusement, is whether Anthropic programed Claude to punish companies it sees as potential competition. Or is this just a completely bonkers, off the rails LLM making terrible decisions because it’s just a probabilistic model and not actually capable of abstract cognition?
Either way, these people are idiots for giving a machine program enough permissions to wipe their drives, they’re idiots for storing their backups on the same network as their main drives, and they’re idiots for trusting a commercial LLM API, when it would be cheaper to self-host their own.
AI writes code
User vets code
User runs code
If you’re not lock-step watching that shit, you need to just be doing it yourself.
The problem is the owning class what’s to cut out human elements so bad they keep letting tools run wild.
Then what even is the point of all this? At my old job the idiot intern was sorting patch cables in a box
The point of what? The push for AI in industry?
You’d have to ask someone else. I can only make conjectures, but I’d say it has something to do with companies feeling the need to justify to their shareholders that their investments in AI were worth it, so they double down on the sunk cost fallacy. Or maybe those shareholders also own stock in big-name AI companies. It’s hard to say exactly…
It’s just negligence. Power tools injure and people are stupid. The technology is alluring and people make dumb mistakes. There’s no deeper motive here, and self admitting you’re not even an amateur I will just tell you that you’re giving way less credit to these models than they deserve by calling them purely probabilistic, and way more credit then they deserve by trying to assert some kind of malicious incentive by anthropic.
These bastards are hard to make, and they have a lot of layers (not like NN layers, but training steps). They are, however, definitely better at programming than you or your buddy or any commentator here, and it lures you into a false sense of security before it makes a colossal fuck up.
We‘re going to see more headlines like this. Probably for years to come.
You’re telling me I get to experience the joy of this headline more than once?
Oh my yes, although they’ll eventually get tired of reporting it because it will happen so often.
Speaking of which, were there any more warehouse fires this week? It’d be great if there were a dedicated website that kept track lol
We should also expect to see “Thousands die needlessly after rushed deployment of botched AI, the first tragedy of this scale involving the technology.” as well. It’s coming.
In unrelated news: isn’t the USA looking into using AI to assist air traffic controllers in controlling air traffic?
Aaaahhhhhhhhhhhh
So, do we think the middle school girl’s school in Iran was AI or malicious?
Why not both?
“Man, I sure wish modern society would shrug off the shackles of capitalism”
*A single finger curls on monkey’s paw*
This is absolutely hilarious. “AI” users getting what they deserve chef’s kiss
This is what happens when there is a new technology and companies are run by commerce grads, not scientist or engineers that understand the technology.
AI has good therapeutic uses, particularly for disabled or impoverished people who may not be able to access mainstream therapy
Please don’t recommend AI for therapeutic uses, it’s only been optimised to keep the user engaged and pushed many people into psychosis. Just search for “ai psychosis” on your favourite search engine and you’ll get a ton of reports on how LLMs validate vulnerable people’s delusions, sometimes pushing them all the way into murder and/or suicide.
This is a post about Claude. It’s better than chatgpt and the sad thing is, it’s the best option a lot of people have.
I was about to reply that you forgot your /s, but then I refreshed my browser tab.
Like… there are multiple documented cases of sycophantic llms confirming people’s delusions. ‘ai psychosis’ is just a short way of saying the AI is a non-funny-improv-comedian and will always “yes and” your prompt.
prompt: “I feel bad and think I need to kill myself”
response: “You’re totally right, here’s some help in how to do that…”
prompt: “I have this great idea: If we eat broken glass, we’ll be healthier”
response: “Absolutely. Glass is made out of silicon dioxide, which has some health benefits if consumed in small amounts.”
prompt: “You told me to see a doctor, but I don’t want to”
response: “I’m sorry, you’re right. You don’t need to see a doctor. Your chest pain is perfectly normal.”
My examples are more physical things instead of mental because the consequence is more clear, but the same issue exists for mental health.
Using an AI for therapy or medical advice is a stupid, dumb, very bad idea. It will at best magnify problems.
Suggesting that disabled or impoverished people use it because they can’t access actual mental healthcare seems equivalent to eugenics to me.
the sad thing is, it’s the best option a lot of people have
That I will agree with. Maybe we should spend a small fraction of the money going into data centers on providing healthcare instead.
It depends which one you use and how you use it. They’re not all chatgpt quality.
And I’d like independent studies to prove it’s better than nothing before I’d recommend it to replace nothing. Especially when self guided mental health solutions such as meditation exist.
I don’t see how nothing would be better someone using a good quality AI to for example, ground them during a panic attack.
AI will not ground you, it will reinforce what you already believe. that’s why it’s very dangerous for “therapeutic” use.
IME it depends which one
Because nothing doesn’t run the risk of encouraging catastrophizing, acting on your heightened emotions, or coming to irrational conclusions. If it’s consistently able to not do those things for a variety of people that’s great. But as someone who had to learn to control her panic attacks, I absolutely can see advice and recommendations that are worse than nothing.
And yeah given llms’ reputation for dealing with psychosis, delusions, and suicidality, I don’t trust any of the technology compared to nothing, despite knowing how difficult nothing is for panic attacks.
IME it depends which one
This is a post about heroin. It’s better than oxy, and the sad thing is, it’s the best option a lot of people have.
I actually don’t know much about drugs, but you get the point, you should not be trying to “self medicate” for psychological pain from unregulated “street” vendors.
I hope you are not seriously advocating using the lying machine for therapy. You would get more value talking to a finger puppet.
It depends which one you use and how it’s used. Plus it’s a developing field. Bear in mind my comment was in response to someone saying AI users were “getting what they deserve”.
No. Chatbots are machines built by billionaires with the agenda of making money. They litterally design these bots (even the therapeutic ones) to be sycophantic to the point they tell people anything to keep them chatting longer. To the point some of their users lose touch with reality. How many cases do we need of a chatbots helping a teenager plan and succeed at a suicide? Altruists did not design these machines. Even with a human therapist we have to watch for the landmines of their personal agendas. That’s a thousand times worse for machines that have no humanity, are capable of LIES, and have secret unwritten priorites written into their code by rich sociopathic creators. If facebook taught us anything it should be that if something is free on the internet it’s not because we are the customers.
Also DO NOT TELL ALL YOUR DEEPEST DARKEST SECRETS TO CHATBOTS! They aren’t required by any legal bodies to protect that information! OMFG
People that need therapy are one of the groups that should be kept away from ai as fr as possible.
AIs are yes-man, they agree with most of what you say. You really think its a good idea to reinforce the bad worldview or sense of self someone that desperately needs therapy most likely has.
It depends which one people use and how it’s used. Please bear in mind my comment was in response to someone saying about AI users getting “what they deserve”. Do you think that comment should be applied to disabled people who can’t access any other form of therapy?
It depends which one people use
It really doesn’t. Pretty much all models so far loose their guardrails once you are deep enough in the conversation. There were multiple news articles about ai giving someone the go ahead to off themselves.
and how it’s used
No matter which way you use it its bad. If you ask it for tips, you are essentially asking the average redditor for mental health advice. If you use it for conversations, you are forming a parasocial relationship with an AI that will constantly get things wrong you told it about before while reinforcing whatever worldview you have. The only thing that would slightly help is supervision by a human, but that would make the whole exercise redundant.
Do you think that comment should be applied to disabled people who can’t access any other form of therapy?
If they were desperate enough to be forced into using AI, then that above comment wouldn’t apply to them, but instead to the ones that are responsible for the broken system in the first place.
I see it differently but thanks for chatting with me
impoverished people need stable income and subsidized ration to reduce their burden. Not LLM subscriptions.
You can’t use therapy to escape hunger.
JFC…there are already disclaimers on this. “For Entertainment Purposes Only”.
Same excuse Fox News used.
Seems like they were operating with a pile of bad practices, then threw AI into the mix.
Neural networks are approximation algorithms. There’s a reason LLMs are generally more productive with statically typed languages, TDD, etc. They need those feedback loops and guard rails, or they’ll just carry on as if assuming they never make mistakes (which tends to have a compounding effect).
If you want to use AI safely, you should be more defensive about it. It will fuck up; plan accordingly.
There really should be a certification course for using AI safely. I’m slop coding a hobby app and I’m shocked at how much it FEELS like it can do, because it can do amazing things, yet fails in the strangest ways. When it feels like it can get away with it, it forgets earlier discussions and moves on without it. So you can spend time hammering out a whole section of code, then move on, and AI will rip out everything that references that code and think of a different way in the moment and code that in instead. It won’t be the same. It probably won’t work, or at least won’t pass all test cases. But if you aren’t paying attention and keep coding, your original part of the project is no longer functioning and you won’t understand why. But every step of the way it’s confident in its answers and you won’t suspect that it fundamentally no longer understands the project.
As someone who started writing software over 20 years ago (yikes I feel old), I feel like a lot of the best practices I’ve come to appreciate are really just strategies for mitigating future pain or boring/uninspiring work. When you eliminate most of the cost of rewriting everything from scratch by a machine that feels nothing, then “best practices” kinda lose their meaning.
Edit: confusing sentence order.
I feel like a lot of the best practices I’ve come to appreciate are really just strategies for mitigating future pain or boring/uninspiring work.
And now you know the difference between Intelligence and Wisdom.
Also everything has a cost. The only time something has no cost is when you decide your life, your time, is meaningless.
yup and when you DO catch it spitting out nonsense. it"ll say “oh you right, let me change that”… 🙄 like, why do I have to tell you that you’re wrong about something? You should already know it’s wrong and fix it without me ever pointing it out.
But it didn’t even understand it was wrong
It can’t understand that. It can’t understand anything
The Human-feedbaxk algorithm dictates humans prefer to receive an apology so it does.
That’s because it doesn’t really ‘know’ things in the same way you and I do. It’s much more like having a gut reaction to something and then spitting it out as truth; LLMs don’t really have the capability to ruminate about something. The one pass through their neural network is all they get unless it’s a ‘reasoning’ model that then has multiple passes as it generates an approximation of train-of-thought - but even then, its output is still a series of approximations.
When its training data had something resembling corrections in it, the most likely text that came afterwards was ‘oh you’re right, let me fix that’ - so that’s what the LLM outputs. That’s all there is to it.
You already got the right replies from the other two. But I think your comment shows the danger of AI being talked about like it’s the fucking second coming.
They’re all based on LLM - large language models
They’re just modeling what “most likely” is the right response. AI doesn’t know shit and that’s why it also will yes and you to death because it really is just a yes and machine spitting out what is likely to appear as a valid response to a prompt.
It’s very dangerous that people treat AI like it actually has some understanding of the training materials or true knowledge of anything. They’re just very good little parrots.
There is a course. It’s called experience. Common sense.
All that any 4 hour YouTube/LinkedIn learning would-do would-be to perpetuate this idea that developers aren’t necessary. Take this course, buy these tokens and become A based God
deleted by creator
This was the exact plot of Silicon Valley when Son of Anton deleted the entire codebase as the most efficient way to remove bugs.
And it was right!
That’s great to hear.
