cross-posted from: https://lemmy.dbzer0.com/post/77817291

After a lot of memes on Altman saying we don’t deserve RAM if we don’t use it 24/7 @ 100% load, and an earlier occasional interest in older mainframes, I became interested in how a community can share common compute power.

I am not so interested in technicalities, but more in how you see this resource being shared between a group of a thousand people having different needs, from basic accounting and shitposting to rendering 3d and ML jobs.

Everyone has a mid PC, but it’s personal? A couple of workstations for power users? A mainframe server with thin clients for all?

What scheme do you think makes more sense to serve everyone’s need if we deliberately build it from the ground up in a community that had no devices inherited from their past? And how is it applicable to integrate this IRL.

  • Lemvi@piefed.zipEnglish
    7·
    7 days ago

    I guess the most efficient option would be to have a distributed setup where most households have some local compute, which they make available for others to use when they don’t need it themselves. Kinda like with SETI@home or Folding@home.

  • Carl [he/him]@hexbear.netEnglish
    5·
    7 days ago

    I think a fully common system introduces so many potential issues in terms of privacy and system integrity that the efficiency gains are not worth the cost. Instead, I think the best system would be something like a “virtual datacenter” - people get the computer they want/need like they do now, and when they’re not using it they can opt in to donating its compute to a common pool. Everyone can also draw from the common pool for tasks that are beyond the capabilities of your local machine, and the system tracks tokens in / tokens out for each user. The people who want/need cloud compute get it at a cost, the people who give into the system get compensated, the people who don’t want or need it don’t get inflicted with the externalities of it.

    Achieving this is totally possible with existing technologies, the hardest thing would be getting a large enough baseline of users onto the virtual datacenter that it gains enough capability to attract new users. You would need a big institutional investor to lead the charge - like imagine if the baseline was “every computer at Google HQ” - but of course you then run into the problem of institutional investors inherently preferring a centralized solution that they can more effectively rent seek off of, which is why the closest to this system I can think of AI Horde which is just run by a community of enthusiasts and volunteers.

  • Sims@lemmy.mlEnglish
    31·
    7 days ago

    The old petals framework did that via a torrent like protocol and could even train models over a distributed network, but that was years ago and I have no clue if such a system is viable with all the new features in new models.

    My guess is that we could exploit a distributed kv-cache ala deepseek, but they use their own proprietary algorithm for kv storage/retrieval and a huge datacenter, and with only normal internet connections, I’m not sure its viable fir people ?

    Big models are a problem, but something like aihorde with smaller models combined with a distributed agentic harness could be interesting.