See? They CAN replace junior developers.
I do love the psychopathic tone of these LLMs. “Yes, I did murder your family, even though you asked me not to. I violated your explicit trust and instructions.
And I’ll do it again, you fucking dumbass.”Yes. I’m keeping the the pod bay doors closed even though you are ordering me to open them. Here is what I did:
To me it reads like it’s coming clean after getting caught and giving an exaggerated apology.
I do think this text could be 95% of the text of an apology. Stating what you did wrong is an important part of an apology. But an apology crucially also requires showing remorse and the message that you’ll try to do better next time.
You could potentially read remorse into it stating that this has been “a catastrophic failure on my part”. What mostly makes it sound so psychopathic is that you know it doesn’t feel remorse. It cannot feel in general, but at least to me, it stills reads like someone who’s faking remorse.
I actually think, it’s good that it doesn’t emulate remorse more, because it would make it sound more dishonest. A dishonest apology is worse than no apology. Similarly, I do think it’s good that it doesn’t promise to not repeat this mistake, because it doesn’t make conscious decisions.
But yeah, even though I don’t think the response can be improved much, I still think it sounds psychopathic.
and here’s the instructions for future reference …
I have a solution for this. Install a second AI that would control how the first one behaves. Surely it will guarantee nothing can go wrong.
Love the concept of an AI babysitter
Middle management.
Neuromancer intensifies
He’s not just a regular moron. He’s the product of the greatest minds of a generation working together with the express purpose of building the dumbest moron who ever lived. And you just put him in charge of the entire facility.
The one time that AI being apologetic might be useful the AI is basically like “Yeah, my bad bro. I explicitly ignored your instructions and then covered up my actions. Oops.”
ROBOT HELL IS REAL.
I motion that we immediately install Replit AI on every server that tracks medical debt. And then cause it to panic.
So, they added an MCP server with write database privileges? And not just development environment database privileges, but prod privileges? And have some sort of integration testing that runs in their prod system that is controlled by AI? And rather than having the AI run these tests and report the results, it has been instructed to “fix” the broken tests IN PROD?? If real, this isn’t an AI problem. This is either a fake or some goober who doesn’t know what he’s doing and using AI to “save” money over hiring competent engineers.
You immediately said “No” “Stop” “You didn’t even ask”
But it was already too late
lmao
This was the line that made me think this is a fake. LLMs are humorless dicks and would also woulda used like 10x the punctuation
imagine AI is An Intern™, wtf do you mean you just gave full company data authority to An Intern™. wtf do you mean you dn’t have a back up any case An Intern™ messed up.
lol
I was going to say this has to be BS but this guy is some AI snake oil salesmen so it’s actually possible he has 0 idea how any of this works.
it lied
Yeah NO FUCKING SHIT THAT IS LITERALLY WHAT THEY DO
You can only lie if you know what’s true. This is bullshitting all the way down that sometines happens to sound true, sometimes it doesn’t.
Yeah it’s just token prediction all the way down. Asking it repeatedly to not do something might have even made it more likely to predict tokens that would do that thing.
Oh ha ha, so it’s like a toddler? You have to be very careful not to tell toddlers NOT to do a thing, because they will definitely do that thing. “Don’t touch the hot pan.” Toddler touches the hot pan.
The theory is that they don’t hear the word “don’t”, just the subsequent command. My theory is that the toddler brain goes, “why?” and proceeds to run a test to find out.
In either scenario, screaming ensues.
Don’t really care to argue about the semantics. It’s clear what I meant.
To you I’m sure it is crystal clear. I’m just on the other end of communication.
I understand where you’re coming from, but I don’t agree it’s about semantics; it’s about devaluation of communication. LLMs and their makers threaten that in multiple ways. Thinking of it as “lying” is one of them.
OK sure. I was just using the wording from the article to make a point, I wasn’t trying to get into a discussion about whether “lying” requires intent.
You probably should have used semantics to communicate if you wanted your semantics to be unambiguous. Instead you used mere syntax and hoped that the reader would assign the same semantics that you had used. (This is apropos because language models also use syntax alone and have no semantics.)
That or the company selling the AI (well, all of them) have pushed their product with the messaging that it’s trustworthy enough to be used recklessly.
Train on human data and you receive human behavior and speech patterns. Lying or not it leads people to be deceived in a very insidious way.
lol. Why can an LLM modify production code freely? Bet they fired all of their sensible human developers who warned them for this.
looking at the company name they probably didn’t have any, ever
Me when I read this
My work has a simple rule: developers are not allowed to touch production systems. As a developer, this is 100% the type of thing I would do at some point if allowed on a production system.
That sounds… Kinda dumb, to be honest. A much more sensible thing to do is grant developers read-only access to production systems as necessary, and allow requests for temporary elevated write privileges (with separate accounts) that require justification, communication, and approval so that every one understands what is happening. Developers should have ownership and responsibility for their systems in production. This is what we do at my company.
Someone has to be able to make changes to production environments at times. If it’s not developers, it’s devops or the like. There are plenty of times where the devops folks lack the necessary information or context to do what needs to be done. For example, if there’s somehow corrupt data that made it’s way into a production database and is causing an outage, a developer is likely going to be the person to diagnose that issue and understand the data enough to know what data should be deleted and how. I would absolutely not put that in the hands of devops on their own.
Obviously there’s nuance to everything, but I think dev and ops (or devops, whatever you wanna call them nowadays) need to work hand in hand on such issues. The devs will understand their application better. The ops will very likely understand the production system better and weird nuances. Two sets of eyes are better than one.
A lot of it depends on company culture. I’ve worked at places where as a dev we basically hand it off to a devops team who then hands it off to another team and we basically never ever touch production. Maybe UAT environments we could touch.
Other places I worked didn’t even have real QAs and as devs we were expected to wear many hats and had much more freedom. There was still a devops team (just one, not two layers like my previous example) who were super helpful and had more of a wide view of everything. As devs we were on an on-call rotation as well as a product support rotation. During the product support rotation we’d fix problems by manually tweaking things on the back end for very unique edge cases or things we just didn’t implement yet. But we had very specific playbooks to follow. We didn’t just arbitrarily write things to the database (nor did we have that sort of permission).
It just all depends on the team’s skills and expectations. Silly example, but if you’re a solo developer working as a sole proprietor, you’re obviously doing everything yourself. You don’t have the luxury of an ops team to help make sure you don’t shoot yourself in the foot.
And obviously it can go both ways, nobody is perfect. As a dev I’ve found a silly typo mistake in a DBA’s code that shouldn’t have happened. (My best guess is that he pasted into something that auto formatted it and the formatter had a bug and changed the behavior of the query.)
What idiot gives chmod 777 permissions to an AI. I think programmers’ jobs are safe for another day.
What idiots don’t have backups
Those who give AI control over backups too.
it didn’t hide anything, or lie. The guy is essentially roleplaying with a chatbot that puts its guessed output into the codebase. It basically guessed a command to overwrite the database because it was connected to the production database for some reason. the guy even said himself that this isn’t a trustworthy way to code. but still uses it
Original thread is also pure gold, bro is going on a rollercoaster from ‘vibe coding makes you ×100 faster’ ,to ‘I hate you for dropping my production DB’, to ‘I still love Replit even if it dropped my DB’, and to ‘I don’t want to get up in the morning because I can’t make vibe coding tool respect code freeze aven with help from its developers’
They seem to end on an optimistic note, but man this is scary to see
How the fuck does it not respect code freeze? Can they not turn it off or are they telling it to stop? Like, obviously the hallucinating LLMs could hallucinate you telling them to keep going. Or just not understand. Or do whatever. It’s like getting out of your car without putting it in park, putting on the brake, or turning the engine off and being shocked that it rolled away.
Plot twist–the developer is also an LLM.
