If I had to guess, teleco buildings already have strong internet connectivity and backup power or secondary power utilities which malls don’t have because they don’t have the same needs.

Ignoring the other comments around complexity. If anybody is considering running LiteLLM in their lab. I strongly recommend looking at GOModel instead. LiteLLM was nothing but bugs, slow page loads, and security issues for me. GOModel just worked and was simple.

Spammers and other nefarious actors.

There’s a lot of trust built into federation.

Back when I had a Wordpress blog before I switched to a static website, when I got these spam comments, I’d just edit the positive sounding ones to remove the spam link and post them. It made me like people were reading my blog and caring.

It means they release the model so you can run it yourself with enough hardware. Kind of like open source. Governments and large companies should be self-hosting this kind of stuff because of security, availability, and IP-leak risks anyway.

With LLMs, there is some risk of hidden malicious actions embedded in the weights that you can see until you ask it a magic question and it says “activate skynet.”

Whether or not the government is doing this because of fears of hidden actions in the weights or just because OpenAI/Anthropic CEOs told the government to give them more money is unclear.

Yes, I run Ollama on a NVIDIA 1080 Ti at home. I run granite4:3b for my Home Assistant assistant for my voice assistant, paperless-ngx, OpenWebUI for chat, and a few other things. For speech to text, I run a custom-built hybrid speech-to-phrase + parakeetTTS (requests that match speech-to-phrase take priority over parakeet.) I run GOModel to route requests to my local Ollama instance, or DeepInfra (cloud-based for larger models), or Tinfoil (cloud-based for encrypted LLM chats.)

As much as I love to use local-everything, only my smarthome+basic chats go local. Trying to fit both the STT and LLM models on the same GPU at the same time while also using this computer as a media console is challenging. I’ve recently started looking at the Hailo m.2 accelerator board to see if I can shift the STT to it.

That doesn’t check out when considering model size. Unless that’s what you’re trying to say that self-hosted models force people to use smaller models. Just from an operations perspective a data center had economies of scale and the ability to batch up multiple independent user’s requests which should reduce the per token kWh along with the fact that your utilization will be higher vs somebody’s GPU at home would be idling.

That said I do believe there are negative environmental impacts from AI like always using giant frontier models for everything instead of smaller fine tuned models.

EDIT: I see a bunch of replies on the other comment, which all raise valid points, but that’s definitely not the way most readers are going to interpret the original post and moves the goal posts.

The illustrator seems to be referenced at the bottom of the blog post Owen Yeconiel, but I don’t know how much it would cost. An artist friend suggested it might be in the few thousands when considering layering and various illustrations.

IMO this is more PR than anything. The lava lamps are far away from the camera that the variance in pixel data from the lava lamps is low. Think about the percentage of pixels changing compared to the total camera space and the range of pixel data.

I’ve heard about pointing a camera at a black surface, then massively upping the gain which causes the camera to oversaturate the noise in the sensor measuring process to get random seed data.