plenipotentprotogod

joined 1 year ago
[–] plenipotentprotogod@lemmy.world 42 points 5 months ago (3 children)

If you were actually hoping to buy one but the rounded corners are a dealbreaker, then you may be interested to know that the DIY edition lets you mix and match the older display with the newer motherboards. Looks like opting for the older display even saves you $130 on the purchase price.

[–] plenipotentprotogod@lemmy.world 3 points 9 months ago (2 children)

Out of curiosity, what software is normally being run on your clusters? Based on my reading, it seems like some companies run clusters for business purposes. E.g. an engineering company might use it for structural analysis of their designs, or a pharmaceutical company might simulate the interactions of new drugs. I assume in those cases they've bought a license for some kind of high-end software that's been specifically written to run in a distributed environment. I also found references to some software libraries that are meant to support writing programs in this environment. I assume those are used more by academics who have a very specific question they want to answer (and may not have funding for commercial software) so they write their own code that's hyper focused on their area of study.

Is that basically how it works, or have I misunderstood?

This actually came up in my research. Folding@Home is considered a "grid computer" According to Wikipedia:

Grid computing is distinguished from ... cluster computing in that grid computers have each node set to perform a different task/application. Grid computers also tend to be more heterogeneous and geographically dispersed (thus not physically coupled) than cluster computers.

The primary performance disadvantage is that the various processors and local storage areas do not have high-speed connections. This arrangement is thus well-suited to applications in which multiple parallel computations can take place independently, without the need to communicate intermediate results between processors.

[–] plenipotentprotogod@lemmy.world 2 points 9 months ago (1 children)

I'll have to look a little more into the AI stuff. It was actually my first thought, but I wasn't sure how far I'd get without GPUs. I think they're pretty much required for Stablediffusion. I'm pretty sure even LLMs are trained on GPUs, but maybe response generation can be done without one.

[–] plenipotentprotogod@lemmy.world 3 points 9 months ago (4 children)

I’m not sure what you’d want to run in a homelab that would use even 10 machines, but it could be fun to find out.

Oh yeah, this is absolutely a solution in search of a problem. It all started with the discovery that these old (but not ancient, most of them are intel 7th gen) computers were being auctioned off for like $20 a piece. From there I started trying to work backwards towards something I could do with them.

[–] plenipotentprotogod@lemmy.world 5 points 9 months ago (1 children)

I was looking at HP mini PCs. The ones that were for sale used 7th gen i5s with a 35W TDP. They're sold with a 65W power brick so presumably the whole system would never draw more than that. I could run a 16 node cluster flat out on a little over a kW, which is within the rating of a single residential circuit breaker. I certainly wouldn't want to keep it running all the time, but it's not like I'd have to get my electric system upgraded if I wanted to set one up and run it for a couple of hours as an experiment.

 

A university near me must be going through a hardware refresh, because they've recently been auctioning off a bunch of ~5 year old desktops at extremely low prices. The only problem is that you can't buy just one or two. All the auction lots are batches of 10-30 units.

It got me wondering if I could buy a bunch of machines and set them up as a distributed computing cluster, sort of a poor man's version of the way modern supercomputers are built. A little research revealed that this is far from a new idea. The first ever really successful distributed computing cluster (called Beowulf) was built by a team at NASA in 1994 using off the shelf PCs instead of the expensive custom hardware being used by other super computing projects at the time. It was also a watershed moment for Linux, then only a few yeas old, which was used to run Beowulf.

Unfortunately, a cluster like this seems less practical for a homelab than I had hoped. I initially imagined that there would be some kind of abstraction layer allowing any application to run across all computers on the cluster in the same way that it might scale to consume as many threads and cores as are available on a CPU. After some more research I've concluded that this is not the case. The only programs that can really take advantage of distributed computing seem to be ones specifically designed for it. Most of these fall broadly into two categories: expensive enterprise software licensed to large companies, and bespoke programs written by academics for their own research.

So I'm curious what everyone else thinks about this. Have any of you built or admind a Beowulf cluster? Are there any useful applications that would make it worth building for the average user?

[–] plenipotentprotogod@lemmy.world 8 points 10 months ago

Searx is a search aggregator. It masks your identity from the search providers, but under the hood it's still just a middle man for google/bing results. I don't see how this helps if the results themselves are getting worse.

 

I've been aware of pi-hole for a while now, but never bothered with it because I do most web browsing on a laptop where browser extensions like uBlock origin are good enough. However, with multiple streaming services starting to insert adds into my paid subscriptions, I'm looking to upgrade to a network blocker that will also cover the apps on my smart TV.

I run most of my self hosted services on a proxmox server, so I'd like something that'll run as an LXC container or a VM. I'm also vaguely aware that various competing applications have come out since pi-hole first gained popularity. Is pi-hole still the best thing going, or are there better options?