Glm 5, which i think is near frontier model capabilities, takes around 1.5 Tb of vram un-quantized.

DeepSeek’s latest version is 508GB, and even that is wasteful, compared to the wild difference in sizes with some of the quantized versions. The huge amount of parameters and weight sizes are completely unoptimized in terms of sizes, and there’s all kinds of diminishing returns with the haphazardous way these LLM corps are applying new model versions.

Messy brownfield code requires a lot more meta-analysis from previous sessions, just to get all of the spaghetti code into a manageable set of rules and documentation.

Don’t just send the agent blindly into the code. Every session, the agent is just going in fresh without any context on how the code works. Break the codebase down to a 10,000 foot view as a summary, and then get more documentation and agent rules for each of its major sections. Good tests help a great deal, too, and if there aren’t tests, start building them. Start creating skills just for the codebase, or any common tasks you’re asked to do against it.

Also remember that you don’t have to write any of this stuff. Have it write the docs, rules, skills, tests, and just review it to make sure it looks right.

A majority of software runs on OSS. Your CIO is an idiot.

That seems like the worse of both worlds. About 95% of my code nowadays is agent-generated, but it’s based on my input, because I’m the fucking expert. Outsourcing the work just turns it into the same “garbage in, garbage out” process that outsourcing always does.

I really wanted to spin up my own home LLM server, but the prices around that are still scary right now.

Good for you. Hope that works out!

This is what happens when you trust some random “news” outlet that nobody has heard of.

Okay, but that brand is Elon Musk, and nobody should buy anything even remotely owned by him.

The headline literally names the brand.

Who the fuck is a “spyware firm”? Marketing is one thing, but a company dedicated to pushing malware should be shunned and thrown in prison.

Sounds like a company that’s one cybersecurity event from bankruptcy. Good luck!

For that price, I could create my own GPU farm of local models, instead of wasting money on tokens.

Actually, the model itself is called Prometheus, but Microsoft has slapped their Copilot label on so many things that it’s hard to know what they are talking about.

Real developer who has been programming daily since the 80s. The quality of the models used to vary pretty wildly, but here in the past year or two, everything has gotten way better and most models are pretty capable. Claude, Gemini, GLM, Kimi, DeepSeek. The prices haven’t, but my advice is to ignore the bullshit and just pick the models that work for you. Claude isn’t 25x better than DeepSeek, so it doesn’t deserve to be 25x the cost.

I think Claude and OpenAI have started to realize that nobody’s buying their overpriced token costs. Although, frankly, none of these prices even make sense compared the cost of compute for an average cloud-based GPU cluster. I can’t see how any of these companies are making a profit, just riding the private equality wave until the money runs out. Maybe Amazon, Microsoft, Google, since they own their own hardware, but not OpenAI or Anthropic. China’s going to run them out of business in a few years, anyway.

And of course, if you have the option, use local models. VRAM size is still a bit of a problem, but quantization techniques have gone a long way. You can run a decent model on a personal PC with a bit of tweaking, for home programming tasks, or just a personal model that doesn’t immediately sell your data to Peter Thiel.

Holy fucking shit… rational conversations when talking about AI tool usage on Lemmy? WTF?!

Why are we writing articles on what the biggest megacorps are doing with the technology? Meta is one of the most braindead corporations I’ve ever seen, wasting billions and billions of dollars on failed tech after failed tech. Microsoft can’t stop pushing their shitty Copilot, which isn’t even on anybody’s LLM radar. Anything Elon Musk leads consistently makes dumbass decisions that scare us all.

If anything, it just shows how stupid these megacorps are.

And I’m disgusted at colleagues just copy pasting our IP into random cloud-based LLMs, makes me want to bang my head on the wall.

If you’re talking about “intellectual property” and not an IP address, then that sounds like a security violation. At the very least, you need some vendor that can sign an NDA as one layer of protection, and a policy that dictates that nobody shares passwords, tokens, PII through these agents. Then firewalls, strict guidelines on where to use these things, make production access forbidden.

The problem is rushing into anything is a bad idea. Slow down, figure out how this shit works, implement policies, use what works, throw away what doesn’t, don’t radically change everything because of new shiny bullshit.

The people who jumped in headfirst at the first sign of LLMs and fired half of their employees? They all crashed hard and fucked themselves. That’s the lesson in what not to do.

They treat their citizens and women like trash. They treat migrants even worse. Barely one step above slavery.

Where is the story here?

“AI bad. AI do bad thing. Let’s point and laugh at AI doing bad thing to delegitimize the technology in the public’s eyes.”

Meanwhile, at work, I interact with coding agents on a regular basis, and the results are pretty good. Then it goes through a local blind review by a more powerful agent, get pushed out to the company’s code review agent, which is designed to be as critical as possible, check the ticket details, make sure everything matches what the design plan says. Depending on the repo, it might even go through an extra security review.

Then, because we aren’t fucking stupid, it goes through the standard code review process by real developers. Sometimes they still find something, sometimes the agent code reviews found way more than a human can. Overall, I trust the current process over our previous one, where some guy spends barely two minutes looking at the code, rubberstamps it, and then QA discovers a few bugs that need fixing, because the damn code review wasn’t as through and critical as it should have been.

And he had Hillary secretly working with Obama on the underground tunnel that was supposed to seal the leaks between computers and the clever fuckers who programmed the computers.

Go on. I want to see where this goes.