• 1 Post
  • 141 Comments
Joined 3 years ago
cake
Cake day: June 22nd, 2023

help-circle







  • I’m not an expert either. I mostly use Opus at work and I don’t pay for it - the company does. I’ve noticed Fable almost seems like a waste for a lot of tasks, so mostly I don’t use it. I’ve not tried dialing it down to Sonnet or even Haiku to see how things fare, but keep meaning to try that out.

    If you look at leaderboards like:

    https://llm-stats.com/leaderboards/llm-leaderboard

    You’ll see Fable 5.2 is at the very top with 61.1. Even Opus is 58.7.

    Here, you can see the very top is 54.9 from Kimi and I’m not sure which of these you can host yourself. My understanding is that just because something is open weight doesn’t mean you can download it and in some instances, even when you can, the hardware to run it is not something exactly consumer-class (but a company might choose to build or rent that hardware themselves).

    https://llm-stats.com/leaderboards/open-llm-leaderboard

    When you remember how people went gaga over this in 2022…imagine using a model that “lags” some of the top-tier stuff but applied to more mundane things - it’s not like the capabilities from 2022 or whatever evaporated. Suppose you assume you can run something at decent enough speeds, but it “lags” the others by 6 months to a year?

    At some point, it just seems to me there will be diminishing returns to keep paying for the most expensive option. My company offers me Fable now and I don’t find that I need it for day to day things. For all I know, Haiku might be enough for some things.





  • I mean, that was the joke for Amazon for years, but…look where they are now.

    But I just don’t understand the massive hype around LLMs. The cost per 1M tokens has dropped, significantly, since 2023 or so. And I don’t think there is any reason to expect it not to continue. If you host it yourself, it’s basically free other than hardware and electricity.

    Sure, with agents like claude code, you may use a bit more tokens or use it more often, etc, and the best models are more expensive. But not every task needs the best model or even a paid model at all. And people seem to be finding ways to fan out things to prefer local models for simple stuff, escalate to something paid, etc.

    I just don’t get the business model at all, but maybe I’m just too stupid to understand. It’s also possible we have an Enron-style situation, but spanning multiple companies and/or industries…