Oops. Now that users are being to made to pay something closer to the true cost of AI inference, no one will like using it anymore. Could this be what ultimately sets off the bubble *collapse?
Users are paying way more than the cost of inference. Look up inference prices of high end open weight models vs claude or gpt. Cheaper by an order of magnitude.
It’s the constant training of new models that’s losing them money. New version is out every month.
To clarify, AI companies charging the cost that would make the inference profitable for them, against the operating costs and financing costs on new capital expenditures (new data centres, new compute and new model training*), is more than what most people appear to be willing to pay. That cost is indeed more than just the cost of inference incurred by the AI company.
*(I’m being generous and including model training as capex for the sake of argument, even if I personally think to continue the hypetrain, continuous model improvements are core to AI companies’ operation.)
To be clear, We don’t really know the architecture or model size of claude or chatgpt. If opus is a 1 trillion parameter dense model, then yes of course it’s going to be way more expensive to run than deepseak which is an 862 billion parameter MoE model. American models have focused on being the best regardless of cost where chinese model makers were forced to focus on efficiency because of lack of access to chips. Which will probably give them an advantage as the free investor money runs out for the american companies.
The constant training is simply how AI works and will continue to work chinese or american. That cost will never go away. Anytime you need your model to learn new skills or gain new knowledge, it needs to be trained.
The best advancements in usability of AI haven’t all come from training larger models. Tool usage is super useful and doesn’t require new training for new tools, etc.
Deepseek for example doesn’t have a monthly release schedule, they’ve only released one version so far this year.
Plus for new knowledge, there’s web search. It’s no longer strictly true that AI output is restricted to information available before the cutoff date.
At this point you only really need to train a new model if you’re trying out architectural changes, you don’t need to crank out constant updates.
The 2 remaining devs using Copilot will leave it.
It’s rare to see such a clear example of first-mover disadvantage as GitHub Copilot.Honestly, only Tesla comes to mind. Maybe bleeding edge tech frontier is the one place where first movers RND cost is heavy enough to make it irrecoverable.
It’s interesting to contrast this w/ the field of medicine; RND cost can be recovered by squeezing the folks in the domestic market.
The company I work for had a pilot project for a while but recently they opened up for all users to order Github Copilot. Of course it is Github Copilot since everything else at the company is from Microsoft.
You mean you guys don’t rotate between 10 free accounts and use their monthly quotas?
That many free accounts is against their user agreement though hehe
What are they gonna do, close your accounts and make you sign up for 10 more? lol
Limit it per household/IP and block VPNs. You people really have no imagination.
Residential VPNs are available, plus home IPs change constantly on most providers.
Their best option is probably requiring a phone number verification to make an account.
"For basically nothing’… If it’s basically nothing then use your damn brain to do it.
Best I can do is spin up 20 agents to barely get some basic functionality written.
The slop is now prohibitively expensive. Good.
Every single person who mentions using AI for anything- any reason at all- all I can do is imagine what face they would make if I spat in it.
Actually it can be a very useful tool, if you know what you’re doing. Expensive in real terms, so only for tasks which are worth it. So perhaps not for making annoying cat videos.
rumbles throat
Said like someone who doesn’t understand that AI is a tool that can greatly improve your work life and productivity.
If I was you I’d do some research into it, especially around GitHub copilot, and get learning - or you’re going to be left behind. If you’re a dev then you are signing your careers death warrant by having that attitude.
I wouldn’t even like what I do if I had to use this shit.
Stanley Parable button pusher life for real.
If it is a sooo big productivity improvement just pay for it. Or maybe you mean it was a worthwhile productivity improvement when it was almost free?
People will, and are. Business licenses still get included usage amounts btw.
Local AI it is then. Not that I’m using all that much now anyway…
Where I work, Chinese models are banned due to legal concerns. Not just in production, on any company-owned machine. That basically eliminates all the decent open weight models. I’m imagining this type of policy will be more widespread. I suppose it’s because of the potential for legal woes if systems and people are dependent on these models and then federal or state laws impose harsh penalties with little time to react.
I meant for private use.
As for work, we do use AI quite a lot and I don’t have a say over what’s available.
Copilot? Copilot.
deleted by creator
That’s the biggest baddest model out there. There are models that get you 90% of the way there with a significantly smaller parameter count and thanks to MoE offloading you don’t need the entire model active at once.
A 5090 and a beefy CPU and tons of RAM won’t be cheap or even affordable to most, but you could run very big models and have a beefy PC for other activities. But even a 16 GB card could do plenty.
Yeah, I’m aware. I have realistic expectations and I’m looking into running something simpler and less demanding.
Codeberg accounts incoming!
I don’t think they’re moving from GitHub, just GitHub CoPilot.
they should probably move away from plagiarismhub
That’s a non-existent distinction. GitHub is under Microsoft’s AI division.
https://www.techspot.com/news/109040-microsoft-ai-push-tightens-grip-github-after-ceo.html
What I want to know, is that are they just charging closer to the real cost or the actual real cost.
Chances are they would want to slowly increase the price à la boiling frog method.
And once that happens, then they have to increase it again to make profit, AND that has to measure up against regular ways of making money so it can’t just be barely profitable.
It’s a long road ahead for them
I was still using my copilot account, figured I’d see how the new pricing worked. I blew through 60% of my max+ limit in 1 day on absurdly light usage, promptly cancelled rather than upgrade from the $39 / month plan to the $100 /month plan.
I do think there’s a productivity help from AI but vibe coding everything is miserable and gives awful results. Targeted AI usage makes sense and I’ll refine my local AI usage and tooling for that.
Claude was already like that months ago, I would blow through my weekly usage 2 or 3 days into the week.
Me too, and same. I canceled immediately.
between anthropic going ipo for cash and this, I think we’re seeing the edge of the bubble
Single prompt takes 25% of the monthly usage in like 30 seconds even though I have Pro haha
Another thing that greatly improves productivity is not relying solely on AI to do your job for you.
Good devs don’t rely on it to do their job, but use it to do their job better and more productively.
As a dev, copilot is amazing.
My friend who uses Claude daily says it saves him a hell of a lot of time doing all the routine work, which he verifies. It’s not AI’s fault if devs are sloppy, or if non-devs think they can type “Create software” and go get coffee. I still remember some people around 1990 who similarly misunderstood object-oriented programming - like it meant drawing a box labeled “DoAccounting” and magic would happen.
It certainly saves time because of course plagiarism saves time. Does it mean people are doing their job better or more productively? Depends on how carefully they review that AI output, but I have a feeling that people aren’t always so vigilant. That’s where the technical debt creeps in.
Well good luck with the next few years of sprint planning. Shouldn’t be long before the climate impacts really catches up to us.
They’ll move to another service, the service’s expenses will spike, and then that service will switch to usage based billing, and then they’ll have to look for another service again, and repeat the loop until they can run a good enough model locally.
Why would anyone use Copilot when you have so many other options. Cursor just released Composer 2.5 and it’s actually decent. I should have a job right now doing this.
From the enterprise admin of Copilot, Cursor & Claude…enterprise controls.
MicrosoftGitHub understands what companies need to run their products securely and successfully (at scale™)./edit to add. Our Copilot bill went down due to change in billing type, mostly due to organization pooling of credits.
That just sounds like bad management, and a huge opportunity to cut costs by switching to Linux and something like cursor or other alternatives.
I don’t think you understood my sentence, we (the company) already offer plenty of choices. In fact we offer OpenAI (Codex, ChatGPT) as well as Gemini. I just admin Cursor, Copilot and Claude (and at one time we offered Windsurf, but that didn’t have enough users to rectify the costs/time spent administrating it). What I am trying to tell you is that the fairly new companies, OpenAI, Cursor, Anthropic/Claude absolutely suck at making their product usable by the enterprise while Gemini and Copilot completely understand enterprise needs/requirements.
We already have Linux laptops (with enterprise controls). /edit Apple machines are primarily used though. We even offer Windoze if you really need it! (Some folks do.)
That’s fair, as long as it’s recognized that it’s that way for now, and rapidly changing.
Because subsidized plans can’t be used at enterprise. Enterprise pays expensive api rates
