inari@piefed.zip to People Twitter@sh.itjust.worksEnglish · 2 months agoManagersmedia.piefed.zipimagemessage-square166linkfedilinkarrow-up1919
arrow-up1919imageManagersmedia.piefed.zipinari@piefed.zip to People Twitter@sh.itjust.worksEnglish · 2 months agomessage-square166linkfedilink
minus-squareKaligalis@lemmy.worldlinkfedilinkarrow-up15·2 months agoIt might not be as impossible as it sounds. Some of the “open” models are rumored to be able to code. The real problem is that you likely need something with 128 GiB VRAM to run them with a reasonably large context window.
minus-squareIratePirate@feddit.orglinkfedilinkarrow-up6·2 months agoAn Nvidia B200 (192 Gigs of RAM) sells somewhere between 30-50k a pop. That’s feasible for a company.
minus-squaremindbleach@sh.itjust.workslinkfedilinkarrow-up3·2 months agoQwen’s 27B model from April outperforms its 397B model from February. Local and small were always going to win.
minus-squareDiurnambule@jlai.lulinkfedilinkarrow-up1·2 months agoQwen 3.6 ? It is unstable though. It go awry more often than the 3.5 of the same size.
It might not be as impossible as it sounds. Some of the “open” models are rumored to be able to code. The real problem is that you likely need something with 128 GiB VRAM to run them with a reasonably large context window.
An Nvidia B200 (192 Gigs of RAM) sells somewhere between 30-50k a pop. That’s feasible for a company.
Qwen’s 27B model from April outperforms its 397B model from February.
Local and small were always going to win.
Qwen 3.6 ? It is unstable though. It go awry more often than the 3.5 of the same size.