• 1 Post
  • 42 Comments
Joined 3 years ago
cake
Cake day: July 1st, 2023

help-circle





  • I think people assume the AI code goes first, but it’s not really like that.

    I just busted out a half day of coding. Really impressive stuff for my standard and was just in flow with little optimisation breakthroughs popping in my head as I made it through. Output looks great, all working as expected, documented and commented my new things in case I forget them later or want to explain them to the team.

    But later, I’ll probably run it through a model real quick and expect at least one change, usually around tidying things up or an opportunity to optimise a block here or there a bit more. Basically tighten up any loose screws or be pointed to big obvious things I somehow missed.

    Really, I think that’s how most devs are making use of it. It can be frightening the shit LLMs come up with starting for scratch (that goes with anything), or even 60-70%. Doesn’t matter how good the prompts are, at the end of the day, it’s relying on Frankencode sourced from all over the place to apply solutions. It has no consistent and familiar, style, logic, or “accent” in its language. But if it’s got an established and detailed base, concept, and direction (such as a mostly completed chunk of code), frequently enough it’ll assist well with finding dumb shit.

    It’s like a final phase thing, to get you beyond, “It seems work work well; good enough.” to “It’s definitely working well. This is done for now.”

    I like to think that’s what’s going on here because anyone with even moderate developing knowledge/skill/experience totally sees just how bad LLMs are at coding.












  • “It’s a simple question for any enterprise. Would you like something that’s 10 percent of the cost if it’s 90 percent as good as the best, or do you want to pay that 10 times higher cost?”

    That really depends…

    90% is disasterous. It may sound good because it’s “just” 10% off 100%, but it’s awful. Two iterations reduces that to ~0.01%< but that’s still pretty bad. Obviously the more complex, the higher the risk and the more demand and time per iteration. The machine also has no concept of criticality, so how much is being saved with three or even four iterations to assure failure is basically zero?

    “Then the question is: how many tasks require that? I would tell you some do, but the vast majority, summarizing an email, examining a contract, answering a customer call, composing an email, all of those can be done as well with the much cheaper answer.”

    Imagine 10,000 emails an hour through your enterprise and 1,000 times It fucks up. Anything from missing info to misinterpreting to just straight up modifying/deleting. No concept of criticality per email. 8,000 in a 9–5 office. 24,000 a day for a global enterprise. That’s what 90% looks like.

    Then you got “examining contracts” and all the other stuff he mentioned…