2026-08-05 14:21 UTC
Replies (1)
-
@berkes@mastodon.nl 2026-08-06 10:00
@Xavier@infosec.exchange I get you. The parameters you choose on are good ones, imo. I'd even add two: If you selfhost open models, you support (the ecosystem around) development of such models. Validating the business case, (government) subsidies or venture capital investing in further R&D of such models. And if you self-host, your technical expertise will make this more accessible for people with less expertise. Provided you contribute back in any way. I wasn't, by any means, trying to invalidate your choices. I was merely pointing out that purely financially, so ignoring any of the other parameters, self-hosting models isn't viable. Though I do want to make a point of the economy of scale. When you run e.g. Devstral2 on own hardware it will, on average, use more energy (ie emit more CO2), cause more e-waste than renting it from Mistral (or another centralised hoster) itself. This is true for all decentralised tech: it comes with a penalty in efficiency. Reasons are threefold. Centralisation allows overallocating (a hosting company I built and owned overallocated up to 3.5 times). It allows cheaper sourcing (Mistral will buy the GPUs at steep discounts. OpenAI gets them almost for free. All get very cheap energy) and operation is spread (one engineer for thousands of servers, vs one engineer on one server). Again, that doesn't make your choice of self-hosting a bad one. You weighed the pros and cons and came on this. Point is that in order to make this choice,we do need to consider a fair and clear picture. Which is a complex one.