• 0 Posts
  • 36 Comments
Joined 3 years ago
cake
Cake day: June 12th, 2023

help-circle


  • Most white bread sold in supermarkets is preservative full, even in Europe and Japan. It’s required because the supply chain (from baking to on shelf) itself costs at least half a day or a full day depending on the setup.

    A similar bread baked at home has a shelf life of 2-3 days before it is not fluffy soft anymore. This puts a big constraint as the feedback for the next batch is literally time critical. A longer shelf life enables bulk distribution per store which reduces logistics cost and improves per unit cost



  • So… The bread in supermarket is high in preservative chemicals, which is why it can stay ‘fresh’ for days. The one from a local bakery (boulangerie or pattisere or whatever) will have a shorter shelf life since they don’t have a long supply chain to manage

    There’s not much evidence that the preservatives are bad or the supermarket bread is inferior, but I do miss the fresh smell and crack of the bread when I end up eating the (imo inferior) supermarket bread




  • Once a model knows python 3.14 there’s little need to update it until 3.14 is EOL. Most professional settings pin dependencies and runtimes and only move them for security issues or planned migrations.

    Several embedded platform only support C++14 as the latest target. That’s a lag of 10 or more years. This is similar to how the calculator chips are redone once a decade or so depending on new fab process giving enough benefits. Once something gets used for years, a large investment makes more sense than the current senseless frontier models every month or two.





  • A calculator hasn’t changed much since 1990s. The world has moved on, but it still does the one thing cheaply and reliably. We had hiccups in beginning (expense, limited precision, etc.) and now we have phones and computers to eclipse it, but we still have calculators being produced, sold and used. Once you found a model to fit a use case, it can freeze till you can justify the expense of upgrading the processes around it.

    Wrt your questions: I love that you are asking the right ones. You can delve into deep infer for more details on diff tradeoffs between batching and latency for the specific DeepSeek case. In general, more users allow larger batches and higher efficiency as well as better amortization.

    S curves are genuine problems, though engineers are pretty good at finding something else to offset the slowdown. Computer chips, batteries, solar panels, fiber optics have been pretty good examples of scaling in hardware space. Signalling (Mobile LTE, WiFi, optical trancievers), games, etc. have shown scaling in mixed hw and SW space.

    Scaling might stop for one thing, but then people focus on a diff item to still continue the forward progress. What would that look for models? I don’t have a strong idea, but we have already seen MoE, KV Cache quantization, RL Post Training, etc. as major knobs beyond just scaling model size, with each new knob showcasing the same logistics curve behavior. From HW front, there has been incessant march since 70s, where we have seen multiple plateaus and the engineers still moving the frontier ahead. Freq Wall was hit at 2GHz and we have 6 GHz chips now, at a slower pace but still. Memory Wall was hit and yet HAMR and multi level cells have given us densities like never before.


  • More than mass surveillance, they are pedalling for decision making via AI. Running detection via vision models 24x7 is super cheap, most of them can run on the camera chip itself.

    It’s the larger models that are supposed to turn govt into the biggest sucker of SaaS (surveillance as a subscription :( ). They lower the morality bar (by hiding details. Just ask a model and it’ll find data so you dont feel guilty of trawling through people lives) as well as intelligence bar (fewer vocal employees who could raise a voice)


  • A lot of local models are already at a place where we don’t need them to be better, they are doing 90% of what’s needed to significantly change the landscape. Continual growth is not required. This is delusional to think otherwise.

    Local AI HW seems to be roughly 2x perf in 3 years. In 10 years, we should be running at roughly a magnitude more, either same model faster or larger models at same rate.

    DeepSeek has shown that with the right inference machinery, they make more than 35$/hour/gpu, so even non local models can be profitable since renting a GPU is cheaper than this. The main moat is the inference kernels: more efficient= more profit.

    Inference HW seems to be improving roughly 2x in 2 years or faster. In 10 years, that’s be 32x faster, roughly the gap between a rack vs a single GPU.

    Training isn’t that expensive for smaller models. For frontier model, Xiaomi live trained one in a couple million dollars. That’s expensive but not impossible for a company to do one frontier model a year, or a few smaller ones per year. Beyond all this LLM hype, the older CNN are still living a good life as experts in their small domain.

    What’s not profitable is doing frontier models every month (like a dick measuring contest), or giving inference for free (via free plan or subsidized subscriptions) or spending on marketing and Superbowl ads or booking gpus to scale by 10x each year or quarter (spot prices are roughly 2x more than pre booked at this scale).