• 0 Posts
  • 7 Comments
Joined 3 years ago
cake
Cake day: August 19th, 2023

help-circle


  • The article from Anthropic describes how a small agentic model~15-50gb can be distributed on an inference server hosted across a network of 10-20 IoT devices like smart-TVs, which can have up to 4gb RAM–higher end IoT appliances can also have 1-4GB RAM to hold a meaningful shard of a 30B model.



  • Soleos@lemmy.worldtoFuck AI@lemmy.world•Pull the plug already
    link
    fedilink
    arrow-up
    1
    arrow-down
    1
    ·
    26 days ago

    Lol what? This conversation is about the capacity of agentic AI systems replicating across cloud infrastructure, not the affordability of laptops for individual consumers.

    The point was you don’t need an entire server farm. You can rent an 80gb H100 on Lambda for $2/hr. A 125gb Spark laptop costs $5k.

    If it comes down to an effective worm-like agentic distillation at 50gb, the cost to sustain a small swarm comes way down to the point that the swam could sustain its own activities through different revenue streams.

    Once the capability is there at that size, it’s no longer Russia, Iran, or North Korea who could easily drop a billion dollars to jumpstart it. It’s any criminal organization or asshole rich guy who can jumpstart it for the price of a nice car.



  • Soleos@lemmy.worldtoFuck AI@lemmy.world•Pull the plug already
    link
    fedilink
    arrow-up
    7
    arrow-down
    3
    ·
    27 days ago

    My assumption was that it would be a smaller 5-20 GB scale worm-style distillation that’s capable of maintaining/replicating itself anonymously across different cloud computing servers, either through hacking, stolen credentials, or human-aided account verification. From there, as long as it can pay for APIs, it can access the power of frontier models like any individual or company. And then capability and resilience would scale with a networked swarm.

    Yes, there are know your customer financial mechanisms and not detection in cloud services… but people are also out there giving Muse full access to their emails and some even their banking credentials.