True, but currently my prefill speed sucks: some 300-400 t/s. I guess going 8x 8x on the cpu lane will improve that, hopefully by quite some… Losing it again in favor of 24gb extra vram?

That’s the question …

Guess need to wait for the raiser board and test actual speed improvement for prefill.

True… The advantage in vram probably outweighs the rest …

I guess a test is the only answer.

Need to wait for the raiser board to arrive and test …

Yes, carefully check AI output is a must, always.

But still tests even if not perfect are better than no tests, so…

I have relaxed the rate limiter on the proxy, try now

It seems it’s being overwhelmed somehow… Refresh does help.

Yes, my use case seems to be accepted indeed.

Can you elaborate on using obsidian as a memory store?

You are welcome, Indeed my original wall of text was typical of AI bots…

Well, rules are important and knowing them is too. Ignoring them can then be a choice, otherwise it’s just ignorance…

But yes I fully understand your point.

Ram is overkill. 32gb maybe even 16gb would be enough. The CPU too, I used to have an Intel i7 from 10 years ago and it worked. Upgraded because that was the cheapest upgrade I could do.

The GPUs are mandatory, at least to get 30t/s…

Thank you! These days it’s good not to master English enough to be mistaken for an ai bot

My mother language is Italian, not English. But I spent some time living in the US so maybe there is that at my advantage.

LazyNVR was started without any AI assistance… Both use of ai and the rule came afterward.

I am not native English speaker, thanks for the correction.

Smartpants, you wouldn’t have read it either way…

And I wrote in the first sentences that no ai has been used to write it’s it’s all my personal salt

Should I edit the original post? I can delete the text and leave the link

Is that an Ampere card?

My data is a raid5 (Linux MD raid) on top of which I have btrfs partition. They are 4 dad’s.

I have another raid5 on USB mechanical disks again MD raid5 + btrfs, which is my 1st level backup.

Os, Gentoo Linux, is on an nvme ext4, going to migrate to btrfs when I have time.

Server is a Ryzen 9 with 64gb ram that I also use for AI inference thanks to two Nvidia RTX A4000. Currently one on PCIe 16x and one on pcie 4x. I am waiting for a pci-e bifurcation card to move both cards on 8x 8x pcie bus… Splitting my CPU bound 16x lanes in two…

Edit: I host lost of stuff… Mostly on bare metal or podman rootless…

Check out my wiki at https://wiki.gardiol.org/ where I keep track of what i install and how.

I use btrfs on main data disks and backup disks. Recently it caught a data corruption due to a wrong write (neutrino? Luck? Halloween approaching?).

Then it occurred to me that similar corruption would have been undiscovered for years until I needed that file on ext4…

So, that’s a good reason to migrate…

I need to replace my server root filesystem from ext4 to btrfs…

And get rid of a leftover MD/raid1 as well …

I have no idea what mas is, and I never did element calls. You should try it or check online.