A custom fork for frontier-scale mixture-of-experts inference on an RTX 3060 with 12 GB VRAM, 16 GB system RAM and local NVMe

github.com

A custom fork for frontier-scale mixture-of-experts inference on an RTX 3060 with 12 GB VRAM, 16 GB system RAM and local NVMe

github.com

cm0002@no.lastname.nz to

AI - Artificial intelligence@programming.devEnglish · 10 days ago

GitHub - Betelgeuse-AI/Betelgeuse: A custom fork for frontier-scale mixture-of-experts inference on an RTX 3060 with 12 GB VRAM, 16 GB system RAM and local NVMe.** Betelgeuse keeps dense computation on the GPU and brings routed experts into a bounded VRAM pool only when needed.

github.com

A custom fork for frontier-scale mixture-of-experts inference on an RTX 3060 with 12 GB VRAM, 16 GB system RAM and local NVMe.** Betelgeuse keeps dense computation on the GPU and brings routed exp...

You must log in or register to comment.

Chat

AI - Artificial intelligence@programming.dev

Aii@programming.dev

Create a post

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]

AI related news and articles.

Rules:

No Videos.
No self promotion: Don’t post links to your articles.

Visibility: Public

This community can be federated to other instances and be posted/commented in by their users.

2 users / day
31 users / week
136 users / month
153 users / 6 months
1 local subscriber
408 subscribers
124 Posts
14 Comments
Modlog