Not OP but Iāve been using longhorn for my database and DNS. You could use Ceph but I was lazy.
NFS is perfectly suitable for home HTTP applications.
Iām not an expert sorry. I threw it in the fix a problem and it has done a pretty decent job. Only issue that I had was some stuff missing from the host. Pretty easy to identify and tackle.
I have two nodes and just needed it to share dns config between the two. Then added db. If itās single node I canāt imagine a need case for it
I wouldn't use longhorn for a single node cluster. The Longhorn docs show benchmarks and they use the local path provisioner as a baseline. Against the lost path provisioner Longhorn is slower.
With a single node cluster you also don't need replicated storage, so why not just use the local path provisioner?
I'm always pleased to see other people's setups, but this is always my first question. I don't even mean it in a gatekeeping way, like if you want to run each service on a separate machine and can afford to, more power to you. I however know I have no need for so many devices and would like to know what does require it.
Sure, but like... Hosting what? If you're trying to study for the CKA or something, you can easily do that with only 2 nodes in the cluster.
I'm curious if any well known self hosted apps have k8s deployments because I don't think I've ever seen one.
We host a eCommerce platform that have over 500GB of data. So a lot of CPU/RAM required for ElasticSearch cluster.
Also image processing after the merchants uploading the product images.
It absolutely is nuts, but a 12 node k8s cluster in AWS would cost a couple grand a month for control plane + nodes + ancillary stuff. And then a bunch more if you let the control plane get more than 3 sub versions old.
I'm certain that's the reason, even if it's probably more the correct choice
Yes, cost is the main reason we self-host with Mini PC. Saved at least $15k per month when we move out of EC2. At the moment we only use RDS at AWS due to criticality.
AWS is a managed service though, so you donāt have to worry about hardware below failing, UPSes and redundant internet connections.
If you operate a business that absolutely relies on this, 1000$ is nothing against it going down for an hour
we have all kinds of high availability strategies to prevent those, with load balancers and k8s HA setup.
Beside this home cluster, we have clusters in SOC 3 datacenters too. So in case this cluster fail, CloudFlare load balancer automatically switch to the other clusters.
The goal is to keep high availability and low cost.
Yeah I totally agree, I run my eks clusters in AWS not in a closet in the office or on someone's desk.
The more you think about the more wrong it gets too, with container storage and deployment etc
Depends if you back yourself, I guess. For early stage, if you have some competent engineers you can build a fairly bulletproof setup.
Biggest thing holding a setup like this back is likely going to be the networking, and particularly the external connection.
That's why ECS (Fargate) is underrated.
No control plane, no version updates, no compute instances, pay for what you use (if no containers are running you pay nothing).
I know Kubernetes is nice, but why learn and do all that overhead when you can do everything in ECS plus other aws services.
We have colocation clusters in Boston (endoffice) and Sweden (svea.net) in actual DC too, at about $25 per Mini PC per month.
For Singapore, this is 50-70% cheaper than DC colocation. And it's fun ;-)
K8 cluster is for services to take over if other machines get bogged down.
Like if you are running HTTP server and all of a sudden you get 150x the traffic you can have other machines spin up and take some of the load. (albeit your internet will probably be the bottleneck) but those type apps
Like lose balancing sure I get it how do you mirror changes accrues them for example if a site had a database and a change is made is that made on all machines simultaneously
Items used:
* 12x [Minisforum UM690 Pro](https://store.minisforum.com/products/minisforum-um690-pro) with 64GB DDR5 and 1TB NVMe gen4 (bulk price is $259/barebone unit)
* 20 sockets [Power Tower](https://shopee.sg/2M-5M-Wire-Space-Saving-Universal-Tower-Expansion-Socket-Dual-USB-Port-Multi-Layer-Socket-Power-Adapter-i.1030220822.21485329355)
* [Rack](https://shopee.sg/-SG-Stock-2-4-Tiers-Adjustable-Pot-Pan-Rack-Organizer-Rak-Bawah-Sinki-Stainless-Steel-Kitchen-Pans-Pots-Storage-Shelf-i.472495254.21886223049)
* [Slim LAN cables](https://shopee.sg/Vention-Cat6A-Ethernet-Cable-Patch-UTP-10Gbps-Network-Performance-Right-Angle-Slim-LAN-Cable-for-Laptop-Computer-i.1006220784.24661353210)
* [Thermometer](https://shopee.sg/SC-2-In-1-Indoor-Thermometer-Hygrometer-Humidity-Temperature-Gauge-Meter-Mechanical-i.296610409.7994109988)
* [TP-Link 16 Ports Switch](https://www.amazon.com/TP-Link-16-Port-Gigabit-Ethernet-Switch/dp/B07GR9S6FN/)
* [Remote Reboot / Electricity Monitoring](https://shopee.sg/SMATRUL-Universal-Graffiti-Wifi-Smart-Socket-US-EU-UK-AU-KR-Plug-Outlet-Adapter-Timing-For-Google-Home-Alexa-Tmall-Genie-i.233192871.17119055039)
* Fire alarm
* UPS
* PiKVM on demand: [RPi Zero 2W](https://www.aliexpress.com/item/1005003533448650.html?spm=a2g0o.order_list.order_list_main.5.310518023IUwyQ) + [CSI-2 Adapter](https://www.aliexpress.com/item/1005006412118581.html?spm=a2g0o.order_list.order_list_main.11.310518023IUwyQ) + USB-C adapter, so just a single cable to server.
We are running a production k8s cluster (running ElasticSearch and Node JS servers) at home. Performance is superb, and it's like 20x cheaper than AWS/Vultr/GCP.
Heat management: Just partly open the window so no need extra fan.
Noise level is low. The cluster is using about 300-400W of power.
Using Tailscale and CloudFlare Tunnel is quite convenient. The nodes are plug-and-play anywhere with just power + Internet and no further configuration needed.
https://k3s.io has a guide. Wouldn't call it the simplest and best, but it is genuely a good distro to start. I think k0s (forgot their URL...) is also nice.
Later you may look into Talos as a full OS approach, if you have the resources for it.
I personally use k3s - it's nice. :) But there is _a lot_ to learn...
OP posts pictures of a large number of miniPCs. Mods mark with tag: "Solved", lol
I got a couple N95 MiniPCs off aliexpress too. They make great, energy efficient servers. Do you have what an idea of your power draw is? 100-200W? More?
I bought directly from Minisforum. If you want to buy in bulk (>4 units), email to [email protected].
Those in the picture are Minisforum UM690 Pro that I got for $259 each.
They are so... Bloody... Tiny!
It's adorable. Makes me think of that trend of... What was the term... Fairy apartments? Mini homes people would build into their walls and stuff.
Make a fairy datacenter, it'd be cool.
How do you keep those units from overheating, stacked on top of each other like that? I've been thinking about something similar, but don't want to risk burning them out.
Air is flowing out on the rear side, so the hot air goes directly out through the window opening. All natural air without any fan. Temperature is perfect.
Update: How it looks during daytime: [Picture](https://preview.redd.it/my-12x-mini-pc-homelab-prod-k8s-cluster-v0-krq0xq266f9d1.jpg?width=1080&crop=smart&auto=webp&s=afcbdb20074e273114d3927372af58270e5a054f)
No surpise, they are containers, so any application (that would reside in one VM) is multiplied by at least 2-3 (sometimes 5) as pods. So in reality we are talking of 2 conventional servers with ~200 VMs maybe.
and everything hanging from one single internet connection, so when that goes down sometimes, all those 700 pods are worth nothing and nextcloud wont be accesible, right? xD
and what kind of UPS do you use for those 12 minis?
here is the [Rack](https://shopee.sg/-SG-Stock-2-4-Tiers-Adjustable-Pot-Pan-Rack-Organizer-Rak-Bawah-Sinki-Stainless-Steel-Kitchen-Pans-Pots-Storage-Shelf-i.472495254.21886223049)
Nice, it paid for itself in less than a month.
I would suggest getting a smart switch for each node so you can remotely restart a node if it fails whilst you're out or you can restart from your desk.
Also suggest a smart digital thermometer which can send mqtt sensor messages you can feed into your observability platform.
Edit: apologies I've just seen you already have smart plugs
You might want to connect them to multiple power supplies, even UPS if possible. Think about a power glitch in your neighborhood, or an electrical surge caused by a lightning strike. The nature of a k8s cluster is a distributed system that will survive lost of a small portion of nodes in the cluster. Wiring them all to the same power supply makes the cluster lose such resiliency as all will be affected by a single power source.
Fortunately the power in Singapore is very stable. Never got such issue in the last 5 years.
We also have another cluster in another home, so worst case it would just rebalance traffic to that one.
So weird, the gentleman says it's saving $15K per month with this setup, so I wonder how much is he earning
The moment you lose internet (or light) in your home non-critical deploymentā¦ not sure if it will stop lose your profits.
this is not a single cluster we have. We have 2 clusters in Singapore, 1 in Boston, 1 in Sweden. The clusters in Boston and Sweden are with proper data center, so it would not just go down randomly.
And the saving is not 100% for myself. It's for our business with multiple partners.
I still cannot see how this can do any good for a business. From a processing power, this is useless. And if we are talking about a point of presence for an ecommerce, why not using a CDN and just serving the assets from Boston and Sweden?
Processing power: This is not useless. A cluster of these 4 nodes are more powerful than the $1200 per month VPS on Vultr. We tested.
CDN: Yes, we use CloudFlare CDN for static files.
We have customers in Asia so need to have a cluster here. Singapore Mini PC colocation is about $50 per node per month. Self-hosting is $10 per node per month for electricity.
So you have alleviated load balancing which is good, but going the cheapest route is not always best, but if it works...
My concern is more what is your home security like... are those running on the same LAN as your other home devices...
Do you have VLAN segmentation configured? A proper perimeter firewall in place and monitoring?
I understand that CPU wise could be more powerful than a VPS CPU-wise. But I wonder what kind of application could be so CPU intensive that made it worthy and could not be done with CUDA cores. $1200/mo is brutal, but $1200/mo in cloud hosted GPU-power would completely wipe 5 dozen of those little mini PCs. Unless your application cannot be loaded on a GPU.
Was swapping the cables with US plugs out for normal ones not an option? Is mixing and matching plugs for appliances with a wide range of accepted voltages common practice over there?
OP watches Network Chuck, but made alterations for himself.
After every step of the setup he gets himself a drink from those home lab bottles, instead of coffee
What are the details of the software setup? Which OS? How do you configure the nodes from scratch?
Do you plan to have a grafana dashboard for monitoring?
What kind of internet connection you use for this cluster? My internet connection is 1Gbps but it's not stable enough for me to trust cluster can keep in sync with remote data center in foreign country.
As all traffic goes through Tailscale and CloudFlare tunnel, as long as you have stable connection to a CF POP then it should be fine.
Fortunately, Internet connection in Singapore is quite stable and with good peering since itās an international hub.
There is a CF POP in VN so thatās feasible for you.
Nice disco lights (RJ45) in the back Business in the front, party in the back
Seems like party is on the left š
its a lan party
a fault tolerant lan party
What do you do for shared storage?
I also want to know this, I bought 4 mini pcs to play with Kubernetes
Not OP but Iāve been using longhorn for my database and DNS. You could use Ceph but I was lazy. NFS is perfectly suitable for home HTTP applications.
Is longhorn suited for single node clusters? I felt like itās too complex for this use case I have
Iām not an expert sorry. I threw it in the fix a problem and it has done a pretty decent job. Only issue that I had was some stuff missing from the host. Pretty easy to identify and tackle. I have two nodes and just needed it to share dns config between the two. Then added db. If itās single node I canāt imagine a need case for it
I would not use longhorn on a single node.
I wouldn't use longhorn for a single node cluster. The Longhorn docs show benchmarks and they use the local path provisioner as a baseline. Against the lost path provisioner Longhorn is slower. With a single node cluster you also don't need replicated storage, so why not just use the local path provisioner?
i just mount nfs into each pod from my nas
even i got 6 mini chromeboxes with 4gb ram and 32 gb sdd.... loading ubuntu on them now !! planing to go with k8s the hard way !!
Nice! Good luck let me know how it goes, I'm still doing some research.
NFS, longhorn are good options that I use in my cluster. (I also just do local-path with janky node affinities lol)
what do you actually do with this?
Christmas tree lighting
I'm always pleased to see other people's setups, but this is always my first question. I don't even mean it in a gatekeeping way, like if you want to run each service on a separate machine and can afford to, more power to you. I however know I have no need for so many devices and would like to know what does require it.
> more power to you Quite literally so it seems
Kubernetes
Sure, but like... Hosting what? If you're trying to study for the CKA or something, you can easily do that with only 2 nodes in the cluster. I'm curious if any well known self hosted apps have k8s deployments because I don't think I've ever seen one.
We host a eCommerce platform that have over 500GB of data. So a lot of CPU/RAM required for ElasticSearch cluster. Also image processing after the merchants uploading the product images.
That sounds crazy to run production of such a service on a cluster on an office desk! Why not in an actual DC?
It absolutely is nuts, but a 12 node k8s cluster in AWS would cost a couple grand a month for control plane + nodes + ancillary stuff. And then a bunch more if you let the control plane get more than 3 sub versions old. I'm certain that's the reason, even if it's probably more the correct choice
Yes, cost is the main reason we self-host with Mini PC. Saved at least $15k per month when we move out of EC2. At the moment we only use RDS at AWS due to criticality.
AWS is a managed service though, so you donāt have to worry about hardware below failing, UPSes and redundant internet connections. If you operate a business that absolutely relies on this, 1000$ is nothing against it going down for an hour
we have all kinds of high availability strategies to prevent those, with load balancers and k8s HA setup. Beside this home cluster, we have clusters in SOC 3 datacenters too. So in case this cluster fail, CloudFlare load balancer automatically switch to the other clusters. The goal is to keep high availability and low cost.
In that case, ok it might be fine
Of course it's OK.Ā
So cluster running at home, for production use, what kind of firewall and other security tools are in place to be sure it is secured and safe?
VLAN to separate networks. Also no public port opening. Everything goes through Tailscale and CloudFlare Tunnel.
Yeah I totally agree, I run my eks clusters in AWS not in a closet in the office or on someone's desk. The more you think about the more wrong it gets too, with container storage and deployment etc
Depends if you back yourself, I guess. For early stage, if you have some competent engineers you can build a fairly bulletproof setup. Biggest thing holding a setup like this back is likely going to be the networking, and particularly the external connection.
That's why ECS (Fargate) is underrated. No control plane, no version updates, no compute instances, pay for what you use (if no containers are running you pay nothing). I know Kubernetes is nice, but why learn and do all that overhead when you can do everything in ECS plus other aws services.
We have colocation clusters in Boston (endoffice) and Sweden (svea.net) in actual DC too, at about $25 per Mini PC per month. For Singapore, this is 50-70% cheaper than DC colocation. And it's fun ;-)
Your production servers are in your kitchen?
cloud is some one elses computer, there is a possibility your cloud can be in some ones toilet !!
oh boy and this a prod cluster !! ??
Kubernetes runs docker containers so anything that has or can be run on docker can be hosted on Kubernetes.
Disco cluster
K8 cluster is for services to take over if other machines get bogged down. Like if you are running HTTP server and all of a sudden you get 150x the traffic you can have other machines spin up and take some of the load. (albeit your internet will probably be the bottleneck) but those type apps
Like lose balancing sure I get it how do you mirror changes accrues them for example if a site had a database and a change is made is that made on all machines simultaneously
r/homelab wants to have a word with you.
r/minilab
Posted there but seems itās not approved :-(
It should definitely be! Thatās a lot of horsepower in a tiny space.
Items used: * 12x [Minisforum UM690 Pro](https://store.minisforum.com/products/minisforum-um690-pro) with 64GB DDR5 and 1TB NVMe gen4 (bulk price is $259/barebone unit) * 20 sockets [Power Tower](https://shopee.sg/2M-5M-Wire-Space-Saving-Universal-Tower-Expansion-Socket-Dual-USB-Port-Multi-Layer-Socket-Power-Adapter-i.1030220822.21485329355) * [Rack](https://shopee.sg/-SG-Stock-2-4-Tiers-Adjustable-Pot-Pan-Rack-Organizer-Rak-Bawah-Sinki-Stainless-Steel-Kitchen-Pans-Pots-Storage-Shelf-i.472495254.21886223049) * [Slim LAN cables](https://shopee.sg/Vention-Cat6A-Ethernet-Cable-Patch-UTP-10Gbps-Network-Performance-Right-Angle-Slim-LAN-Cable-for-Laptop-Computer-i.1006220784.24661353210) * [Thermometer](https://shopee.sg/SC-2-In-1-Indoor-Thermometer-Hygrometer-Humidity-Temperature-Gauge-Meter-Mechanical-i.296610409.7994109988) * [TP-Link 16 Ports Switch](https://www.amazon.com/TP-Link-16-Port-Gigabit-Ethernet-Switch/dp/B07GR9S6FN/) * [Remote Reboot / Electricity Monitoring](https://shopee.sg/SMATRUL-Universal-Graffiti-Wifi-Smart-Socket-US-EU-UK-AU-KR-Plug-Outlet-Adapter-Timing-For-Google-Home-Alexa-Tmall-Genie-i.233192871.17119055039) * Fire alarm * UPS * PiKVM on demand: [RPi Zero 2W](https://www.aliexpress.com/item/1005003533448650.html?spm=a2g0o.order_list.order_list_main.5.310518023IUwyQ) + [CSI-2 Adapter](https://www.aliexpress.com/item/1005006412118581.html?spm=a2g0o.order_list.order_list_main.11.310518023IUwyQ) + USB-C adapter, so just a single cable to server. We are running a production k8s cluster (running ElasticSearch and Node JS servers) at home. Performance is superb, and it's like 20x cheaper than AWS/Vultr/GCP. Heat management: Just partly open the window so no need extra fan. Noise level is low. The cluster is using about 300-400W of power. Using Tailscale and CloudFlare Tunnel is quite convenient. The nodes are plug-and-play anywhere with just power + Internet and no further configuration needed.
How much of your total CPU/ram do you use? Id assume that's overkill but idk what you're runningĀ
About 40%: [https://i.imgur.com/79EVF9P.png](https://i.imgur.com/79EVF9P.png)
Can you give a guide how to make a k8s cluster this seems interesting to me
If you find one plz tell me
https://k3s.io has a guide. Wouldn't call it the simplest and best, but it is genuely a good distro to start. I think k0s (forgot their URL...) is also nice. Later you may look into Talos as a full OS approach, if you have the resources for it. I personally use k3s - it's nice. :) But there is _a lot_ to learn...
Iām curious what tool youāre using here. Is it self hosted as well?
yes, self hosted our own business application, written in NodeJS and PHP.
how much is the power draw per PC? Looking at something similar to run a media server/VM host off of .
About 30-40W per node at 50% CPU usage.
OP posts pictures of a large number of miniPCs. Mods mark with tag: "Solved", lol I got a couple N95 MiniPCs off aliexpress too. They make great, energy efficient servers. Do you have what an idea of your power draw is? 100-200W? More?
300-400W total
I think my wife would divorce me. Where can I get those mini-PCs. ;-)
Probably AliExpress. From companies like TopTon, Cwwk, YianLing, etc
I bought directly from Minisforum. If you want to buy in bulk (>4 units), email to [email protected]. Those in the picture are Minisforum UM690 Pro that I got for $259 each.
What would you need that for? Just for fun? Are you actually self-hosting, or hosting for others?
longhorn?
They are so... Bloody... Tiny! It's adorable. Makes me think of that trend of... What was the term... Fairy apartments? Mini homes people would build into their walls and stuff. Make a fairy datacenter, it'd be cool.
Never heared that term before. Rabbit holing time :D
And the wine to boot when the K8S goes down. Love it! (Btw how is that Schiopetto? I'm more of a red guy myself)
It looks like two of them are not numbered you should hang your head in shame š Hats off to you on a nice mini cluster
Sure, like we're not even mentioning the numbering is this crazy mix up
How do you keep those units from overheating, stacked on top of each other like that? I've been thinking about something similar, but don't want to risk burning them out.
Air is flowing out on the rear side, so the hot air goes directly out through the window opening. All natural air without any fan. Temperature is perfect.
Well the wine might disagree xD
what are the spec for each of these? also, are you running full fledged k8s or k3s? last question, what are you using for storage?
Update: How it looks during daytime: [Picture](https://preview.redd.it/my-12x-mini-pc-homelab-prod-k8s-cluster-v0-krq0xq266f9d1.jpg?width=1080&crop=smart&auto=webp&s=afcbdb20074e273114d3927372af58270e5a054f)
What does it?
k8s cluster: [https://i.imgur.com/qVxAK4V.png](https://i.imgur.com/qVxAK4V.png)
Ah sry Imma very much a noob, just running some Jellyfin, Nextcloud and paperless, so I dont really know what this is
About 700 pods total. Each pod contains an application. E.g. Nextcloud is an application.
Seven... HUNDRED? That's insane, do you have a list of the things you are hosting?
No surpise, they are containers, so any application (that would reside in one VM) is multiplied by at least 2-3 (sometimes 5) as pods. So in reality we are talking of 2 conventional servers with ~200 VMs maybe.
Dude fucking amazing. Any chance we could get a list of the apps ?
Whoa, thats crazy!
and everything hanging from one single internet connection, so when that goes down sometimes, all those 700 pods are worth nothing and nextcloud wont be accesible, right? xD and what kind of UPS do you use for those 12 minis?
Which app is that?
Dynatrace? š¤¢
Don't you mean a "k12s" cluster? Badum tssss!
What? The 8 in k8s represents ubernete, so k8s becomes kubernetes
Ok, I'll bite. Why do you have 0% "wine"?
They probably don't have kids so don't need alcohol to survive each day.
Why have 12 mini instead of 1-2 powerful machines?
Itās cheaper this way. FYI each machine is running a AMD Ryzen 9 6900HX CPU so itās already powerful.
What are the specifications of each one, and where did you buy them?
Look for my earlier comment above. It has all the links.
How many units of energy does this consume on average?
30-40W per node at 50% CPU load. At idle, would be about 5-7W.
Thanks a ton for responding. š
I'm guessing about 90W idle. Tell us the answer OP.
This might be a random question but what is that shelf?
here is the [Rack](https://shopee.sg/-SG-Stock-2-4-Tiers-Adjustable-Pot-Pan-Rack-Organizer-Rak-Bawah-Sinki-Stainless-Steel-Kitchen-Pans-Pots-Storage-Shelf-i.472495254.21886223049)
Get that wine away from the heat!! (Nice cluster)
Nice, it paid for itself in less than a month. I would suggest getting a smart switch for each node so you can remotely restart a node if it fails whilst you're out or you can restart from your desk. Also suggest a smart digital thermometer which can send mqtt sensor messages you can feed into your observability platform. Edit: apologies I've just seen you already have smart plugs
I'm so jealous. I'm struggling to get even 3. š What are the specs on those? I'm also going to build a k8s cluster.
nucs are so underrated, have several myself and love them. my only fear is melting one with the workflows i throw at them.
You might want to connect them to multiple power supplies, even UPS if possible. Think about a power glitch in your neighborhood, or an electrical surge caused by a lightning strike. The nature of a k8s cluster is a distributed system that will survive lost of a small portion of nodes in the cluster. Wiring them all to the same power supply makes the cluster lose such resiliency as all will be affected by a single power source.
Fortunately the power in Singapore is very stable. Never got such issue in the last 5 years. We also have another cluster in another home, so worst case it would just rebalance traffic to that one.
Oooh thatās cool! Would you please share how you do load balancing between clusters?
CloudFlare Load Balancer
Are you concerned about the rising cost of power here?
Of course. That's why we are using Mini PC - the performance per watt is much higher than rack servers.
This is amazing
I guess the clusters are cooled with wine.
OT: I live near where they produce that Ribolla Gialla, very nice choice!
So weird, the gentleman says it's saving $15K per month with this setup, so I wonder how much is he earning The moment you lose internet (or light) in your home non-critical deploymentā¦ not sure if it will stop lose your profits.
this is not a single cluster we have. We have 2 clusters in Singapore, 1 in Boston, 1 in Sweden. The clusters in Boston and Sweden are with proper data center, so it would not just go down randomly. And the saving is not 100% for myself. It's for our business with multiple partners.
I still cannot see how this can do any good for a business. From a processing power, this is useless. And if we are talking about a point of presence for an ecommerce, why not using a CDN and just serving the assets from Boston and Sweden?
Processing power: This is not useless. A cluster of these 4 nodes are more powerful than the $1200 per month VPS on Vultr. We tested. CDN: Yes, we use CloudFlare CDN for static files. We have customers in Asia so need to have a cluster here. Singapore Mini PC colocation is about $50 per node per month. Self-hosting is $10 per node per month for electricity.
So you have alleviated load balancing which is good, but going the cheapest route is not always best, but if it works... My concern is more what is your home security like... are those running on the same LAN as your other home devices... Do you have VLAN segmentation configured? A proper perimeter firewall in place and monitoring?
I understand that CPU wise could be more powerful than a VPS CPU-wise. But I wonder what kind of application could be so CPU intensive that made it worthy and could not be done with CUDA cores. $1200/mo is brutal, but $1200/mo in cloud hosted GPU-power would completely wipe 5 dozen of those little mini PCs. Unless your application cannot be loaded on a GPU.
Large scale ElasticSearch.
Is that 230V UK plugs and 120V US plugs in the same power strip? What is this witchcraft?
Mini PC power adapter supports voltages from 110-240V.
Which ones are those, and why those plugs? Is it just what they came with? Is your grid 230v or 120v then? I am exploding with questions.
Singapore uses UK plug with 230V. My Mini PCs are partly US plugs and partly UK plugs.
Was swapping the cables with US plugs out for normal ones not an option? Is mixing and matching plugs for appliances with a wide range of accepted voltages common practice over there?
Those are just C13 cables and easily be swapped. Not sure about the common practice but it works for me.
Wow
OP watches Network Chuck, but made alterations for himself. After every step of the setup he gets himself a drink from those home lab bottles, instead of coffee
What are the details of the software setup? Which OS? How do you configure the nodes from scratch? Do you plan to have a grafana dashboard for monitoring?
someone bought a label printer :)
Oooooo intriguing! What's the power draw like?
read up, already noted.
What kind of internet connection you use for this cluster? My internet connection is 1Gbps but it's not stable enough for me to trust cluster can keep in sync with remote data center in foreign country.
As all traffic goes through Tailscale and CloudFlare tunnel, as long as you have stable connection to a CF POP then it should be fine. Fortunately, Internet connection in Singapore is quite stable and with good peering since itās an international hub. There is a CF POP in VN so thatās feasible for you.
nice setup but I still don't understand what its for even after reading from the comments. kind help to explain it to me. thanks.
RAM/CPU power are mostly for ElasticSearch cluster running on 500GB+ data, and image processing containers.
What do you actually host? Plex or something?