Elektrine lite

← Feed

@Xavier@infosec.exchange

2026-08-05 14:21 UTC

@berkes@mastodon.nl I see your point. It may be true that the best value for the money is to use the commercial models. But those are heavily subsidized right now. So the TOE will be ballooning soon. The comparison reminds me of the lease vs. buy argument with cars in the US. You can get a brand new car every 2-3 years if you lease. It's the right call from a money perspective. But all new cars have surveillance technology that I can't stomach. And I don't need a new car every two years. "Good enough" is a 2002 PT Cruiser and a 1995 Mazda Miata. Same goes for commercial models. My local AI has access to my email, banking records, all my chats, all my files. No way am I give that away to the tech bros in Silicon Valley. Also, these companies are awful. While I enjoy the technology, I despise the way those companies have done business. I don't want to support those activities, and I don't want t hand over my private data. Luckily, local models are also getting better every day. I'm currently doing some A+B testing with some Chinese models and getting positive results.

Replies (1)

  • @berkes@mastodon.nl 2026-08-06 10:00

    @Xavier@infosec.exchange I get you. The parameters you choose on are good ones, imo. I'd even add two: If you selfhost open models, you support (the ecosystem around) development of such models. Validating the business case, (government) subsidies or venture capital investing in further R&D of such models. And if you self-host, your technical expertise will make this more accessible for people with less expertise. Provided you contribute back in any way. I wasn't, by any means, trying to invalidate your choices. I was merely pointing out that purely financially, so ignoring any of the other parameters, self-hosting models isn't viable. Though I do want to make a point of the economy of scale. When you run e.g. Devstral2 on own hardware it will, on average, use more energy (ie emit more CO2), cause more e-waste than renting it from Mistral (or another centralised hoster) itself. This is true for all decentralised tech: it comes with a penalty in efficiency. Reasons are threefold. Centralisation allows overallocating (a hosting company I built and owned overallocated up to 3.5 times). It allows cheaper sourcing (Mistral will buy the GPUs at steep discounts. OpenAI gets them almost for free. All get very cheap energy) and operation is spread (one engineer for thousands of servers, vs one engineer on one server). Again, that doesn't make your choice of self-hosting a bad one. You weighed the pros and cons and came on this. Point is that in order to make this choice,we do need to consider a fair and clear picture. Which is a complex one.

    Open ##5422612