If you (or your employer) can afford one of the more expensive Apple silicon computers, then you know that the machines can run LLM's well enough. You just need a lot of that unified RAM. More and more people are experimenting with this, and having various degrees of success. This is the foreshadowing that I've been talking about when it comes to an AI bubble. The music analogy is that you no longer need a recording studio if you have a laptop. The AI data centers are the recording studio.
Now Microsoft announced the Surface Laptop Ultra, and the desktop Dev Box. These use Nvidia RTX Spark silicon, an approximate analog of Apple's M-series. This means that the massive power found in the Macs of the last few years will compare to something similar in Windows. The ubiquity of local LLM running ability is coming.
The availability of this hardware, along with increasing end user costs for AI agents, will converge in a way that I think will legitimately harm the big AI companies. I don't know how it will shake out, but the only way it doesn't happen is if open source models can't compete with the "big AI" models. They're already "good enough," if you have the hardware to run them on. That hardware will get cheaper over time.
Agentic coding is certainly just one use of AI, but apparently it's the segment that actually makes money (if not profit) right now. Search and advertising also benefits from this, but Google and Meta own their own infrastructure. Support bots and the like often run on the cloud providers, not AI firms' hardware. While I admire the advancements of Anthropic, OpenAI and such, it's not clear what their long-term play is. Developing great models that eat insane amounts of compute isn't sustainable. And that's not even getting into the pushback from everyone about building data centers.
Don't get me wrong, I think being able to run a local model is a future that we as software people can get into. I've messed with it, and it kind of works with less robust models that will fit in my rig. I need a computer anyway, so if it can do this, why would I pay someone else? Exciting stuff. I don't think it'll be so great for my retirement accounts in the near-term.
No comments yet.