OpenAI/Anthropic is incentivized to prevent this.

They are also big enough and unregulated enough that they could use their power & political/industry relationships to drive up the price of local AI ownership (RAM, GPUs, etc)

It will be once the bubble pops. Small local tuned models for specific tasks that the user powers are much less expensive for the tech companies than tech companies powering and watering datacenters.

Right now the tech bros genuinely think people will be cool paying hundreds of dollars a month to rent a GPU for all their Internet tasks. AI fatigue is already setting in.

The tech bros' investors will pull funding once they realize how asinine that is long-term. Probably already starting to, with the likes of Zuck trying to use green charity money to fund his LLMs.

…Without cash, though?

We’ve had an obvious, somewhat proven path to uber fast local inference (bitnet), but no one has taken it. No one is willing to roll the dice with a few multi-million dollar training runs, apparently, and this is true of dozens of other incredible papers.

It seems like organization around local model tinkering is hanging by a thread, too. Per usual, client business will barely lift a finger to support it.

So while I’m a local acolyte, through and through, I’m a bit… disillusioned. It doesn’t feel like anyone is coming to save us.

Also, if y'all are interested, run local models!

It’s not theoretical.

The cost of hybrid inference is very low; You can squeeze Qwen 35B on a 16GB RAM machine as long as it has some GPU. Check out ik_llama.cpp and ubergarm's quants in particular:

huggingface.co/ubergarm/models…

But if you aren’t willing to even try, I think that’s another bad omen for local models. Like the Fediverse, it won’t be served to you on a silver platter, you gotta go out and find it.

ubergarm (John Leimgruber III)

Open LLMs and Astrophotography image processing.

^{huggingface.co}

I'd argue it's inevitable for the simple reason that the whole AI as a service business model is a catch 22. Current frontier models aren't profitable, and all the current service providers live off VC funding. And if models become cheap enough to be profitable, then they're cheap enough to run locally too. And there's little reason to expect that models aren't going to continue being optimized going forward, so we are going to hit an inflection point where local becomes the dominant paradigm.

We've seen the pendulum swing between mainframe and personal computer many times before. I expect this will be no different.

Actually, I agree. And so far, small local models are really solid, and can punch above its weight even when compared to frontier models.

I believe what I meant when I said I doubted it was since these AI corpos seemingly give no indication that local is an option, so most people would think they can only access an LLM through the web. This would bolster the SaaS ecosystem dominating over local AI, although local will keep increasingly growing as a more favourable option.

Although I do agree that the industry will shift from being server based to PC based inference as well, I don’t see that shift being large enough to make these companies change their training paradigms to include telemetry from local AI, but I’m sure some will.