Just to make this clear up front: I was always very skeptical of LLMs (I’ll never call them AI) and still see them as fundamentally limited and unreliable. See my post history if you don’t believe me.
Having said that: I work in business software and the models have progressed to the point where typing code and writing classes and functions by hand has become uneconomical. We review the output closely to avoid the idiocy of “vibe-coding”, but the quality has improved to the point where we rarely have to outright reject solutions. The discussions have moved to high-level architectural points, where we just request better implementations than the first-shot. We get them, review again and move on. Maybe there are deeper, latent problems beeing embedded in the code-base through this process, but we can’t spot any although we are looking intently.
And just like that, my profession has died. Nobody in my team writes code anymore. Things that would have taken days are now done in minutes. We are all just considering design and architecture. We are changing our processes since they are not fit for the new way of doing things. Scrum? What’s even the point anymore? We can work through a sprint-backlog that would’ve taken two weeks in two days now. We do reviews 90% of the time. When we request changes they come back in minutes.
I have a feeling like the ground has been taken from below me. My son wanted to become a developer, that isn’t a real job anymore. Everyone around me feels the same way, including our management. We are a small team and a small company, everyone is effective and our jobs are not in danger as long as our product is succesful. But my profession is essentially dead. It’s a craft I honed for many, many years and the craft itself is now automated. I wanted to improve my typing skills to code faster, this is now a ridiculous endeavour. We don’t know what comes next, but I now only have my analytical skills and knowledge of good architecture to fall back on.
And it all changed so fast. I don’t think this will transfer to other professions so easily, but for programming LLMs ended it all in the span of the last four months. I thought I knew what my future holds as a senior dev, now I can barely see past christmas.This is a nightmare.
I don’t think the dotcom bubble is comparable. Back then, once it was figured out how to make an income at all, it was mostly a matter of scaling, and the technology scales very well. If anything, the bubble popping helped the surviving companies by giving them the space to grow.
But the problem right now is making enough money per customer to make up for the cost of servicing each customer, with no way of making up for it with scale. So even if some companies survive the bubble popping, they’ll have a hard time selling anything at a price most people would be willing to pay, unless they get very lucky and someone discovers a very effective optimisation or some new technology becomes mature enough, that makes running the model considerably less costly.
Locally-run models won’t be unaffected either, because open or not, training a model is expensive and the funding for that will dry up too. And it’s not enough to just get it to a state where it can successfully delude people into thinking it provides value to them and call it done, it has to be constantly updated to stay relevant.
A lot of local models are already at a place where we don’t need them to be better, they are doing 90% of what’s needed to significantly change the landscape. Continual growth is not required. This is delusional to think otherwise.
Local AI HW seems to be roughly 2x perf in 3 years. In 10 years, we should be running at roughly a magnitude more, either same model faster or larger models at same rate.
DeepSeek has shown that with the right inference machinery, they make more than 35$/hour/gpu, so even non local models can be profitable since renting a GPU is cheaper than this. The main moat is the inference kernels: more efficient= more profit.
Inference HW seems to be improving roughly 2x in 2 years or faster. In 10 years, that’s be 32x faster, roughly the gap between a rack vs a single GPU.
Training isn’t that expensive for smaller models. For frontier model, Xiaomi live trained one in a couple million dollars. That’s expensive but not impossible for a company to do one frontier model a year, or a few smaller ones per year. Beyond all this LLM hype, the older CNN are still living a good life as experts in their small domain.
What’s not profitable is doing frontier models every month (like a dick measuring contest), or giving inference for free (via free plan or subsidized subscriptions) or spending on marketing and Superbowl ads or booking gpus to scale by 10x each year or quarter (spot prices are roughly 2x more than pre booked at this scale).
It’s not about growth, it’s about adapting to changes. The rest of the world is not going to stop moving forward just because a model has.
That $35/hour/GPU metric doesn’t really mean anything. How many users are being serviced by a single GPU in an hour? How much are they paying so that it ends up generating over $35 in revenue in that hour? By how much does the overhead increase as the number of users increases? The linear scaling of providing LLMs as a service means that their costs will grow faster than the number of users, as the number of users increases (on average, it would be a staircase graph, not a smooth one).
Also, all your extrapolations are dependent on exponential curves not turning out to be S-curves (they always are) and continued investment (which the bubble popping will put a damper on)
A calculator hasn’t changed much since 1990s. The world has moved on, but it still does the one thing cheaply and reliably. We had hiccups in beginning (expense, limited precision, etc.) and now we have phones and computers to eclipse it, but we still have calculators being produced, sold and used. Once you found a model to fit a use case, it can freeze till you can justify the expense of upgrading the processes around it.
Wrt your questions: I love that you are asking the right ones. You can delve into deep infer for more details on diff tradeoffs between batching and latency for the specific DeepSeek case. In general, more users allow larger batches and higher efficiency as well as better amortization.
S curves are genuine problems, though engineers are pretty good at finding something else to offset the slowdown. Computer chips, batteries, solar panels, fiber optics have been pretty good examples of scaling in hardware space. Signalling (Mobile LTE, WiFi, optical trancievers), games, etc. have shown scaling in mixed hw and SW space.
Scaling might stop for one thing, but then people focus on a diff item to still continue the forward progress. What would that look for models? I don’t have a strong idea, but we have already seen MoE, KV Cache quantization, RL Post Training, etc. as major knobs beyond just scaling model size, with each new knob showcasing the same logistics curve behavior. From HW front, there has been incessant march since 70s, where we have seen multiple plateaus and the engineers still moving the frontier ahead. Freq Wall was hit at 2GHz and we have 6 GHz chips now, at a slower pace but still. Memory Wall was hit and yet HAMR and multi level cells have given us densities like never before.
Calculators don’t need to change because arithmetic doesn’t change. They don’t have a moving target. But with code, languages gain new features over time and new languages get developed altogether. Same is true for libraries, frameworks, platforms etc. New CPU designs change what code is optimal, ways of designing software go in and out of favour and so on. And you can only take having the model read documentation or explicitly instructing it to do things in a certain way so far. So a model that doesn’t get updated will definitely become less relevant over time, and these days that would be pretty quickly.
I don’t think CPUs and RAM managing to move beyond what were thought to be plateaus is a very meaningful precedent. For those, the rules that govern what sequence of inputs lead to what outputs are defined externally from the implementation. As long as it enforces those rules, whatever Rube Goldberg machine is between the inputs and outputs is only really bound by physics (…and the IC manufacturing process), so there’s a lot of flexibility in how they can be designed.
But for a neural network, those rules are encoded in the model itself, i.e. the expensive bit. So you can’t really fundamentally restructure it without having to do the training all over again, and there are only so many ways you can represent a given model in memory and optimisations are pretty limited in what they can avoid doing. Like yeah, given the sheer size of these models, minimising the amount of memory reads through batching will absolutely have a meaningful effect on performance, but you’re pretty limited by memory how big those batches can be (especially given how large context windows get with “agentic” use) and you’re ultimately still doing the same number of calculations. These affect the incline of the curve more than they do the shape.
And again, advancement in this area depends on continuing investment.
Once a model knows python 3.14 there’s little need to update it until 3.14 is EOL. Most professional settings pin dependencies and runtimes and only move them for security issues or planned migrations.
Several embedded platform only support C++14 as the latest target. That’s a lag of 10 or more years. This is similar to how the calculator chips are redone once a decade or so depending on new fab process giving enough benefits. Once something gets used for years, a large investment makes more sense than the current senseless frontier models every month or two.
isnt thats why they are peddling it hard for mass surveillance with AI, they know govt are easier to fool enough to have contracts.
More than mass surveillance, they are pedalling for decision making via AI. Running detection via vision models 24x7 is super cheap, most of them can run on the camera chip itself.
It’s the larger models that are supposed to turn govt into the biggest sucker of SaaS (surveillance as a subscription :( ). They lower the morality bar (by hiding details. Just ask a model and it’ll find data so you dont feel guilty of trawling through people lives) as well as intelligence bar (fewer vocal employees who could raise a voice)