← Feed
@berkes@mastodon.nl
2026-08-05 12:00 UTC
@Xavier@infosec.exchange My point is twofold.
But certainly not to tell you that you're doing it wrong, which is the feeling I get from your reply. Sorry if that came across.
First, a hosted LLM, purely financially, is cheaper than the equivalent ran by yourself. Reversed: running your own models will be more expensive for the foreseeable future than renting/pay-per-use-ing it.
Second, the quality of self hosted may very well be "good enough", which is all you need. But it isn't the quality of what you get from hosted versions. Where quality *only* looks at output an omits vectors such as privacy or freedom.
Reasons, as stated earlier, are that the usage of hosted LLMs is heavily subsidized by venture capital. No such subsidies exist when self-hosting. And in hosting in general, but this area in particular, has a very strong "economy of scale" discount.
Open weight models are quite good and improving (and may be "good enough"), but nowhere near the quality performance of top tier models like Opus. Esp in areas that take large context windows or that benefit from strong reasoning, tool usage, etc., the hosted versions are noticably "better".
Or, in short: while it may work for you, and be economically viable for your use-case, it is almost always cheaper and with better quality, to pay for llm usage. Even if that's hosted open-weight models like those from mistral.
Replies (1)
-
@berkes@mastodon.nl I see your point. It may be true that the best value for the money is to use the commercial models. But those are heavily subsidized right now. So the TOE will be ballooning soon.
The comparison reminds me of the lease vs. buy argument with cars in the US. You can get a brand new car every 2-3 years if you lease. It's the right call from a money perspective.
But all new cars have surveillance technology that I can't stomach. And I don't need a new car every two years. "Good enough" is a 2002 PT Cruiser and a 1995 Mazda Miata.
Same goes for commercial models. My local AI has access to my email, banking records, all my chats, all my files. No way am I give that away to the tech bros in Silicon Valley. Also, these companies are awful. While I enjoy the technology, I despise the way those companies have done business.
I don't want to support those activities, and I don't want t hand over my private data. Luckily, local models are also getting better every day. I'm currently doing some A+B testing with some Chinese models and getting positive results.
Open ##5422611