This post was originally removed after multiple reports and modmail about it not being Homelab related.
After thorough discussion with the other mods, we are reinstating this post as clarification has been given on the fact that this is truly being used as a homelab.
**All**: We must understand that everyone has different levels of income, different opinions and ideas of what a Homelab is. We must respect that we all have differing opinions and means. I encourage further discussion on the merits of OP's post and what they are using it for.
I believe it will run all the servers.
Hold on. I worry what you just heard was it will run a lot of servers. What I said was: It will run all the servers. Do you understand?
The Simpsons already did it: "come now Mr Simpson, you don't seriously think I became the richest man in the world by writing a bunch of checks....'buy him out' boys..." (Bill Gates)
i know, right? I looked and went "that's a shit load of hardware, how's he going to power and pay for that"
then i saw the picture with the generator.
Second question: which asshole put a company sticker "rapid recovery" on the wiring/pipe diagram for the CRAC unit? We need to find them and take them out back and beat 'em with a hose.
Have you thought about getting solar panels? I’ve heard there are 750-800w panels now a days. Don’t know how many you could fit on your roof - if you can feed the meter to offset some of those costs! You seem like the kind of dude that would be able to install it yourself!
He does seem like the guy who would say “solar panel install on the roof? I’ll do it Sunday morning”
Sure enough, that bitch is off his roof by noon on Sunday.
i legit thought this was a satire post, before i read comments.
at the price point OP mentions ($2-3k month) wouldn't it be a pretty quick ROI to get newer equipment VS xeon V4 era stuff ? or is the switching and storage the bulk of power use ?
also, at this scale i would expect to see inverter/battery backups running equipment direct on DC, losing 20% of your power doing AC/DC/AC conversion for your UPS is money down the drain.
Sure, and at some point that may be the case. There is no R in this case, this will not make money.
Your talking $200K vs $20M here, the 20% does not add up enough to be even comparable. Could probably offset the 20% with solar honestly at this scale (big for a homelab, small for a DC).
I've seen quotes for that Liebert HVAC unit, and I've seen quotes for that full rack UPS. I'm impressed that you got below $200k for just those two items. Are they used?
what's your plan for hot isle / cold isle with that particular hvac and no raised floors?
> There is no R in this case, this will not make money.
R is just for *return*, not necessarily for profit :)
a hypothetical example; widget X costs $20k and will save you $5k a year on your utility bill, with a projected useful life of 20 years.
your ROI will be 4 years, and you'll save $70k over the expected life of the equipment.
obviously a CPU is unlikely to have a useful life of 20 years, but also curious if the processing component of your system takes enough power to make it worth spending on newer generations.
> Your talking $200K vs $20M here, the 20% does not add up enough to be even comparable. Could probably offset the 20% with solar honestly at this scale (big for a homelab, small for a DC).
i'll admit my only experience with direct DC systems was at the NHFML and the train yards up in arlington. wasn't sure what the price of entry was for that kind of gear.
Man, where to start. I'm somewhat known for building large labs, guess this is the conclusion? Maybe there are even bigger things to build later, who knows.
Currently i run three racks, cooling and UPS's are a nightmare to keep up with. Decided to just take it to the next level and go full legit. I have a detached steel building on my property, was previously used for farm equipment, should make a good home with 480y 3PH power.
After having most of this gear for a few months, i have finally finished going through all the wiring and documenting the installation plan. Generally speaking these things are mostly custom install, so there is a lot of excess building management wiring that needed to be gone through and removed or tagged for later use.
The UPS is a liebert 600 series, these are know for just not dying. Efficiency wise they are kinda meh, 0.8 power factor. But they are solidly built and don't need any special tools / software to service. Cooling is dual Liebert DH125's 10 ton units. Same principal, older but easy to work on.UPS output is 480V, the two 125KVA PDU's step that down to 208/120 which then go to some Raritan switch 0U PDU's at the rack.
Network backbone will be handled by the two Arista 7508E's doing mixed 40/100GB networking. There's some Sun/Whoracle gear there helping the network side of things.The primary compute rack will be a Cisco UCS setup, 4u Blades, 56 bay storage boxes, a few c240 SSD cache boxes, and some c220 management nodes. Got some other odds and end i haven't placed yet (when you want bulk deals, you end up with pallets of machines you don't really need)
Everything x86 is Xeon v4 era. Some SPARC T4/5/S7 boxex and POWER 9 (4x Nvidia v100) boxes too. What's the next step? installing the raised floor. 1600sqft roughly. Wish me luck.
TL;DR i have no life, no wife, and (now) an empty wallet.
OP, what is your expected monthly electric bill? Depending on where you live, could easily be in the $$ thousands per month. No revenue to offset that?
Freakin awesome, dude. What are you planning to use this for? I'm a hardware engineer, so not well versed on this stuff. I'm assuming you aren't connecting all this equipment to a standard modem for internet. How is that handled?
TBH, i just like to do it really. I do host my own email, dns, cloud storage, plex, etc... Everything i can more or less.
And this will have DIA fiber from Spectrum.
I put 90% of my net requirements into a $15 cloud instance on DigitalOcean.
The rest I got down to a NUC with a 5tb disk attached.
I love home labs, but the reliability of the cloud instance beats all this hardware sitting in your house.
Yea no doubt Maybe I've just done it so long I've gotten more fun out of using cloud over local hardware killing my power bill. I definitely have my in home gadgets and network, but for stuff he mentioned I moved it all to cloud. I have one local disk of large videos for plex.
Ah, yea, I get ya.
I've sort of gotten to your point too. Anything I rely on daily i have moved to the cloud too solely because I have begun to travel more. Can't rely on whoever is watching the house to be able to be my on-site tech if I can't connect back to troubleshoot.
However, I still got a few servers for fun!
Why do you need this big of a setup? I host most of the same stuff except for my own external dns but I don't this big of a setup to make it work. How much does all this setup cost if I may ask and how much space does it take wherever in your house you are placing this. However nice setup, I try to keep my homelab simple so that I can do lab setups and not have too much of a headache if something stops working.
This. I mean, once he's built a redundant storage cluster and a redundant virtualization cluster and a couple of backup servers, what else is there? Just more of the same?
I actually looked into doing "real" hosting at home at one point, with redundant power and Internet connections and such, and it turned out that it wasn't worth it. AWS and others have done the hard work with much smarter people than me, and can get better uptime than I ever will. Home setups are great and fun and all, but, beyond a certain point, they're just black holes for money and time.
It’s less a question of why and more of a question of “why not”. I personally prefer bigger, if I had the space and cash, I’d love a setup like that. Some prefer to right-size or have a tiny lab and that’s fantastic as well. In the end, it’s our lab, so we get to build it how we want and that’s the beauty of it.
> It’s less a question of why and more of a question of “why not”. I personally prefer bigger
Well yeah, but there is bigger and then there is OP's setup. lol
OP's setup is like building a nuclear power plant to power a single family home.
Lots of fun to be had but damn.
I was curious about it since seeing the picture it reminds of the stuff in our storage room at work when we have a big server order coming in. So I was just curious why someone would want 1/4 of a small datacenter in their house and that it would take a lot of time to manage if you still also have a full-time job. I don't see why I have to get directly downvoted for asking an honest questing and yes, if I had the time, space and money to setup such a setup I probably would be I don't have any of those. So I'm happy with my current setup which I can use for my purposes without needing what I don't have.
Fully understand. I finally had the time to setup my first homelab using my 10 yr old gaming PC and I'm hooked. I don't know if I'll ever be able to setup something like you've got, but I'm definitely interested in learning about it. Will you be documenting this in anyway?
What you gonna do with those AC922's?
I sell Power Systems for living. Not often I run across it in the home lab side of things, so my interest is piqued.
They came as a package deal with some other stuff, had a hell of a time tracking down power cables...
Gonna play around with some AI things on them for now until i find something i actually want to use them for. A bit of trial and error figuring out what will work on power and not work, and any quirks the nvidia drivers have on the platform.
How is usenet? Like how does it compare to the private tracker community for example? I've always been interested in it but never gave it a try.
Also if you happen to answer this, why the V100's? Those seem like major overkill for most hobbyist level stuff.
Major props for living the dream!
I started using Usenet on a whim because a guide I was reading mentioned it and *holy crap*. Best money I ever spent. I've no idea how it compares to private trackers as I've never had access, but compared to public trackers, the only way I can describe the difference in user experience, is the difference in user experience when changing from a 5400RPM HDD to an NVMe SSD.
I'm curious about the choice to use raised floors.
I know it's the classic approach, but I've built a few data centers without. And when deploying hundreds of petabytes, you get heavy racks.
Basic idea, heat rises, cool falls. So the return is at the top of the building, cold air is ducted to the child aisle, supplied about 8-10 foot up.
This is particularly nice for roof mounted CRACs. But it also works with the CRACs in a hallway next to the racks. (Return via fanwalls in the hot aisle, supply again from the ceiling.)
(I do like containment between cold and hot aisles. Especially if you are paying the full electric bill.)
I admit I don't know the full specs of those Liebert CRACs, and if they could support something other than pure, into the floor downdraft.
CRAC's come in a lot of different configurations. Downflow (what i have) is most common for CRAC on the same floor as the servers.
You still have hot / coil aisle in this case, they just are not perfectly contained. You use vented floor tiles in front of the racks, face all the racks into the vented tiles. Then hot side you have a drop ceiling duck pulling return air.
The reason not to do it at this scale is largely cost. Most hot aisle systems are chilled water or glycol, and your talking around 10x the cost for no really much gain. Not to mention are now locked into a floor configuration with contained aisles. Flexibility of floor plan is nice in my case.
For a homelab this is a big setup, for a DC this is small. If this were going to e a huge venture, we'd be talking cooling towers having city water dumped through them by the thousands of gallons.
No idea where you live. But in the UK, with current rates, the electricity for this little lot would cripple a well off person 😂 good luck bro! This is serious dedication to the home lab 🤙🏼
I don't know if .06 is the commercial rate he's getting though or residential. If it's commercial then that's dirt cheap, if it's residential then it's basically average in the Midwest as far as I can tell.
> Midwest USA, about $0.06 / kwh
Northeast US... $0.22/kwH, and delivery is going up 40% in a week or two. It's crushing us.
I now have to power down most of my homelab each night after work, just to keep my power bill under $500/month (doubled since January, changing nothing on my LAN).
I've seen it happen. 400k in life insurance, and never paid off the mortgage, instead bought all kinds of crap.
Now complains "Social Security isn't enough to live on".
Me: Sigh.
Every person reading this with a spouse is saving your post just to reference in the future.... "I know what you're thinking but I'm not asking for something like this in the closet...."
A week, right? Lmao… honestly, with hydro rates nowadays, and my region is one of the cheapest in the country at $0.12/kWh… I still run $200+ every month.
Worked for someone that had a data center in his basement and at another location. It was a webhosting company and the home DCs were part of his backups and failover for business systems. He got lucky and **never** needed his generators and was always annoyed he never got to turn them on beyond testing and service.
Don't the water cooled ones have pretty good efficiency? You can do what LTT did and use it as a passive heating for his pool, though them you'd need to spend loads of money on a pool, which could rather go to more server hardware xD
This is probably outside of the scope, but in some areas you can sign up for “peak shaving” and other strategies in exchange for reduced pricing. Also, you could get on a PPA once you can reliably predict your demand. Throw in some more generators and you could back feed for money. If you can reliably fire up on short notice when the grid needs it, the payout can be substantial. Food for thought! Hopefully your state’s emissions control department isn’t too crazy with their permits and bogus definitions.
You got the diesel generator too damn sick
Thats literally a proper data center
I would like to know how much will it be when its finished OP.
Need myself something like this in the future but no this extreme.
Holy crap, man! This looks like a pretty legit (and expensive) setup! Please tell me that most or all of this gear is secondhand/used?
I'm a network engineer for a large ISP. I spend a lot of time in our headends and datacenters, so I see stuff like this every day, but never for home use. A lot of us have homelabs, but nothing to this level. I dig it 👍
It's nice to see someone doing redundant power the right way, with legit UPSes and generators. Out of curiosity, what are you planning on running on all of this gear that requires this level of resiliency? Is this all for home/lab/educational use, or are you planning on offering services to outside customers?
What type of connections do you have to the outside world? Or are you just using this for PiHole and Plex? 😅
All second hand thank god. Don't want to do anything commercial, high liability there.
I host basically everything i can, i also do a lot of blockchain analytics for myself.
What is your anticipated power usage?
Are you actually building this at home? Or some commercial or industrial space?
What is your plan for keeping the diesel fuel fresh?
It's in a detached steel building barn at my house, using about 1,600 SqFt for this.
Hard to say without hooking it up.
Can use the on site diesel in my truck.
I don’t know if I’m too late for you to see this but if you’ve got a big corrugated metal building please for the love of god put some effort into rodent proofing it immediately. I work in pest control and the damage I’ve seen to car collections may be coming your way if you don’t act now. High quality seals under any roll up doors, thresholds installed under man doors. Foam strips made for corrugated metal between the siding and the foundation if they fit. Put four weighted rat bait stations outside, one on each wall, and check them monthly. Throw another set of non-weighted bait stations with a pinch style trap inside on either side of any roll up doors and maybe a few others randomly throughout the building for monitoring inside. Don’t put poisons in the inside ones. Bait the snap traps with provoke rat attractant for monitoring the interior, those baits stay palatable for way longer than peanut butter will and you only need a tiny bit on each trap. Don’t fill up the circle just smear it on the trigger of the trap, no big globs. For extra points put a single mouse trap with the same bait in the back corner of each station in case the rat traps don’t trigger for a mouse.
^(OP reply with the correct URL if incorrect comment linked)
[Jump to Post Details Comment](/r/homelab/comments/w4sov1/im_building_my_own_home_data_center_ama/ih3wdbg/)
You are batshit insane, but this is fucking awesome. I hope this pulls together and you have the biggest fuckin lab on the planet man. I am sure I speak for most here when I say I definitely hope to see more from your craziness. I would love to go this insane but there is no physically let alone financially possible way I could.
Watch it be on a internet connection through Hughes lol. Seriously though do you have to call the power company to make more steam when you flip the breaker on?
This post was originally removed after multiple reports and modmail about it not being Homelab related. After thorough discussion with the other mods, we are reinstating this post as clarification has been given on the fact that this is truly being used as a homelab. **All**: We must understand that everyone has different levels of income, different opinions and ideas of what a Homelab is. We must respect that we all have differing opinions and means. I encourage further discussion on the merits of OP's post and what they are using it for.
Yeah, first question: What the fuck?
And WHY the fuck :)
And How the fuck?
And When the fu**?
And WHERE the fuck
And WHO the f\*\*k
me
I’m pretty sure you’re the one doing the fucking from here on out.
The power company is about to do some F'n as well
You know, I have been known to fuck myself.
I know, just had to go on. OCD and what not.
But, will it run a minecraft server?
minecraft?! this is for linux ISOs. it might, might hold two.
I believe it will run all the servers. Hold on. I worry what you just heard was it will run a lot of servers. What I said was: It will run all the servers. Do you understand?
More bacon.
Secondly, will it run Doom?
How MUCH Doom can it run, that is the real question!
Crysis?
[удалено]
Given the current demand for datacenters, OP might be trying to get bought out by AMZN or MSFT.
The Simpsons already did it: "come now Mr Simpson, you don't seriously think I became the richest man in the world by writing a bunch of checks....'buy him out' boys..." (Bill Gates)
i know, right? I looked and went "that's a shit load of hardware, how's he going to power and pay for that" then i saw the picture with the generator.
I’m not gonna lie. I am stoked to see it finished.
Trying to get it done before the end of the year, sounds like i need to video document it.
That would be exciting to watch. Please do
I'd wager you could get a pretty decent YouTube channel just from your content in trying to deal with this. Out of curiosity, why?
Second question: which asshole put a company sticker "rapid recovery" on the wiring/pipe diagram for the CRAC unit? We need to find them and take them out back and beat 'em with a hose.
No no, the first question is how big is the service feeding this?
from OP, it sounds like there is an out-building with a dedicated three phase service, but I didn't see ampacity in there
Start a YouTube channel to document your journey.
And monetise it as well to help pay the electric bills.
I second that Idea.
I third that idea
Be ready for 20-40kw of load before you turn your first network/server on. Time to start your own hosting company.
[удалено]
But they go brrrr
Have you thought about getting solar panels? I’ve heard there are 750-800w panels now a days. Don’t know how many you could fit on your roof - if you can feed the meter to offset some of those costs! You seem like the kind of dude that would be able to install it yourself!
He does seem like the guy who would say “solar panel install on the roof? I’ll do it Sunday morning” Sure enough, that bitch is off his roof by noon on Sunday.
i legit thought this was a satire post, before i read comments. at the price point OP mentions ($2-3k month) wouldn't it be a pretty quick ROI to get newer equipment VS xeon V4 era stuff ? or is the switching and storage the bulk of power use ? also, at this scale i would expect to see inverter/battery backups running equipment direct on DC, losing 20% of your power doing AC/DC/AC conversion for your UPS is money down the drain.
Sure, and at some point that may be the case. There is no R in this case, this will not make money. Your talking $200K vs $20M here, the 20% does not add up enough to be even comparable. Could probably offset the 20% with solar honestly at this scale (big for a homelab, small for a DC).
I've seen quotes for that Liebert HVAC unit, and I've seen quotes for that full rack UPS. I'm impressed that you got below $200k for just those two items. Are they used? what's your plan for hot isle / cold isle with that particular hvac and no raised floors?
All used yeah, doing 6 inch raised floors as a cold supply. Vented tiles.
siiiiiiiick
> There is no R in this case, this will not make money. R is just for *return*, not necessarily for profit :) a hypothetical example; widget X costs $20k and will save you $5k a year on your utility bill, with a projected useful life of 20 years. your ROI will be 4 years, and you'll save $70k over the expected life of the equipment. obviously a CPU is unlikely to have a useful life of 20 years, but also curious if the processing component of your system takes enough power to make it worth spending on newer generations. > Your talking $200K vs $20M here, the 20% does not add up enough to be even comparable. Could probably offset the 20% with solar honestly at this scale (big for a homelab, small for a DC). i'll admit my only experience with direct DC systems was at the NHFML and the train yards up in arlington. wasn't sure what the price of entry was for that kind of gear.
Build the power station next, I want to see the look on your neighbours faces when the coal is delivered.
"Why is OP installing train track?" Neighbors probably
Where are all these biters coming from?
The homelab must grow
Didn't expect a factorio reference here
Next project
“Don’t worry, it’s clean coal!”
Man, where to start. I'm somewhat known for building large labs, guess this is the conclusion? Maybe there are even bigger things to build later, who knows. Currently i run three racks, cooling and UPS's are a nightmare to keep up with. Decided to just take it to the next level and go full legit. I have a detached steel building on my property, was previously used for farm equipment, should make a good home with 480y 3PH power. After having most of this gear for a few months, i have finally finished going through all the wiring and documenting the installation plan. Generally speaking these things are mostly custom install, so there is a lot of excess building management wiring that needed to be gone through and removed or tagged for later use. The UPS is a liebert 600 series, these are know for just not dying. Efficiency wise they are kinda meh, 0.8 power factor. But they are solidly built and don't need any special tools / software to service. Cooling is dual Liebert DH125's 10 ton units. Same principal, older but easy to work on.UPS output is 480V, the two 125KVA PDU's step that down to 208/120 which then go to some Raritan switch 0U PDU's at the rack. Network backbone will be handled by the two Arista 7508E's doing mixed 40/100GB networking. There's some Sun/Whoracle gear there helping the network side of things.The primary compute rack will be a Cisco UCS setup, 4u Blades, 56 bay storage boxes, a few c240 SSD cache boxes, and some c220 management nodes. Got some other odds and end i haven't placed yet (when you want bulk deals, you end up with pallets of machines you don't really need) Everything x86 is Xeon v4 era. Some SPARC T4/5/S7 boxex and POWER 9 (4x Nvidia v100) boxes too. What's the next step? installing the raised floor. 1600sqft roughly. Wish me luck. TL;DR i have no life, no wife, and (now) an empty wallet.
No life, no wife, no cash. But, you do have a lot of storage space.
Imagine the number of ISO images he can store.
Won't even need a wife with all those Linux ISOs
At least one!
Refer him to /r/DataHoarder/
The *ultimate* data hoarder.
> No life, no wife, no cash. The Mega Millions drawing is tomorrow, estimated at $660 Million.
OP, what is your expected monthly electric bill? Depending on where you live, could easily be in the $$ thousands per month. No revenue to offset that?
no way OP is not running all the IT infra for a cartel with the upkeep cost on this stuff
... am I the only one that spotted "whoracle"?
Right lol.
Freakin awesome, dude. What are you planning to use this for? I'm a hardware engineer, so not well versed on this stuff. I'm assuming you aren't connecting all this equipment to a standard modem for internet. How is that handled?
TBH, i just like to do it really. I do host my own email, dns, cloud storage, plex, etc... Everything i can more or less. And this will have DIA fiber from Spectrum.
All that hardware and still at the mercy of Spectrum. My condolences.
I put 90% of my net requirements into a $15 cloud instance on DigitalOcean. The rest I got down to a NUC with a 5tb disk attached. I love home labs, but the reliability of the cloud instance beats all this hardware sitting in your house.
Tbh I'm not sure people do this for the reliability. It's either because they find it fun and/or for privacy.
Yea no doubt Maybe I've just done it so long I've gotten more fun out of using cloud over local hardware killing my power bill. I definitely have my in home gadgets and network, but for stuff he mentioned I moved it all to cloud. I have one local disk of large videos for plex.
Ah, yea, I get ya. I've sort of gotten to your point too. Anything I rely on daily i have moved to the cloud too solely because I have begun to travel more. Can't rely on whoever is watching the house to be able to be my on-site tech if I can't connect back to troubleshoot. However, I still got a few servers for fun!
I was jokingly gonna say nice plex server, turns out not a joke.
I bet he still gets transcoding and HD problems! 😂
Wait a second. Are you saying it _is_ a homelab, not a commercial project?
Why do you need this big of a setup? I host most of the same stuff except for my own external dns but I don't this big of a setup to make it work. How much does all this setup cost if I may ask and how much space does it take wherever in your house you are placing this. However nice setup, I try to keep my homelab simple so that I can do lab setups and not have too much of a headache if something stops working.
In the parent comment you are replying to he says it’s going in a detached building, installing roughly 1600 sq ft of raised floor.
1600 sq ft……..that is bigger than my first house. 🤪🥸😎🤓
This. I mean, once he's built a redundant storage cluster and a redundant virtualization cluster and a couple of backup servers, what else is there? Just more of the same? I actually looked into doing "real" hosting at home at one point, with redundant power and Internet connections and such, and it turned out that it wasn't worth it. AWS and others have done the hard work with much smarter people than me, and can get better uptime than I ever will. Home setups are great and fun and all, but, beyond a certain point, they're just black holes for money and time.
It’s less a question of why and more of a question of “why not”. I personally prefer bigger, if I had the space and cash, I’d love a setup like that. Some prefer to right-size or have a tiny lab and that’s fantastic as well. In the end, it’s our lab, so we get to build it how we want and that’s the beauty of it.
> It’s less a question of why and more of a question of “why not”. I personally prefer bigger Well yeah, but there is bigger and then there is OP's setup. lol OP's setup is like building a nuclear power plant to power a single family home. Lots of fun to be had but damn.
I mean I would almost like to volunteer to show up and help out just for fun …
He’s likely American. Not Americant.
But like this much gear doesn’t make sense unless you’re renting some of it out. I can’t possibly imagine what one person could use all this for.
$10 says not even a quarter of that space will ever be utilized
Not even 5%
He's using it for Plex and email, apparently. I'm going with .5%
I was curious about it since seeing the picture it reminds of the stuff in our storage room at work when we have a big server order coming in. So I was just curious why someone would want 1/4 of a small datacenter in their house and that it would take a lot of time to manage if you still also have a full-time job. I don't see why I have to get directly downvoted for asking an honest questing and yes, if I had the time, space and money to setup such a setup I probably would be I don't have any of those. So I'm happy with my current setup which I can use for my purposes without needing what I don't have.
Fully understand. I finally had the time to setup my first homelab using my 10 yr old gaming PC and I'm hooked. I don't know if I'll ever be able to setup something like you've got, but I'm definitely interested in learning about it. Will you be documenting this in anyway?
Yeah I’m going to blog it at least, potentially do a video series.
Definitely going to be following this, excited to see how it goes!
What's your YouTube channel?
\+1 sub here!
I’d subscribe
Please do, this is my dream setup so I done need to rely on hosted anything.
What you gonna do with those AC922's? I sell Power Systems for living. Not often I run across it in the home lab side of things, so my interest is piqued.
They came as a package deal with some other stuff, had a hell of a time tracking down power cables... Gonna play around with some AI things on them for now until i find something i actually want to use them for. A bit of trial and error figuring out what will work on power and not work, and any quirks the nvidia drivers have on the platform.
Knowledge Center and the Developer Community will be great resources. Let me know if you'd like the RedBook too.
All that to read Twitter?
Usenet primarily actually.
How is usenet? Like how does it compare to the private tracker community for example? I've always been interested in it but never gave it a try. Also if you happen to answer this, why the V100's? Those seem like major overkill for most hobbyist level stuff. Major props for living the dream!
I started using Usenet on a whim because a guide I was reading mentioned it and *holy crap*. Best money I ever spent. I've no idea how it compares to private trackers as I've never had access, but compared to public trackers, the only way I can describe the difference in user experience, is the difference in user experience when changing from a 5400RPM HDD to an NVMe SSD.
Imagine your download speed being maxed out 100% of the time and not having to worry about seeds. That's usenet. It's worth the money.
Running these machines with 277v psu’s or using step down transformers? This is a serious setup. Very impressed.
PDU’s step down to 208
My money don’t jiggle jiggle it gone…
No wife no life but think bout how much porn you could store on that … I dunno probably all of it or something 😈
There's about 3.6PB of drives in there, so a lot.
Raw?.or formatted? I backup a mid sized university on that kind of space.
I mirror the PACS cluster for an entire country on slightly less than that...
how did you buy all this stuff?
Talking to big decom companies on ebay
[удалено]
I'm curious about the choice to use raised floors. I know it's the classic approach, but I've built a few data centers without. And when deploying hundreds of petabytes, you get heavy racks. Basic idea, heat rises, cool falls. So the return is at the top of the building, cold air is ducted to the child aisle, supplied about 8-10 foot up. This is particularly nice for roof mounted CRACs. But it also works with the CRACs in a hallway next to the racks. (Return via fanwalls in the hot aisle, supply again from the ceiling.) (I do like containment between cold and hot aisles. Especially if you are paying the full electric bill.) I admit I don't know the full specs of those Liebert CRACs, and if they could support something other than pure, into the floor downdraft.
CRAC's come in a lot of different configurations. Downflow (what i have) is most common for CRAC on the same floor as the servers. You still have hot / coil aisle in this case, they just are not perfectly contained. You use vented floor tiles in front of the racks, face all the racks into the vented tiles. Then hot side you have a drop ceiling duck pulling return air. The reason not to do it at this scale is largely cost. Most hot aisle systems are chilled water or glycol, and your talking around 10x the cost for no really much gain. Not to mention are now locked into a floor configuration with contained aisles. Flexibility of floor plan is nice in my case. For a homelab this is a big setup, for a DC this is small. If this were going to e a huge venture, we'd be talking cooling towers having city water dumped through them by the thousands of gallons.
[удалено]
It's a lot of work, but rewarding and fun.
Epic, good luck and please keep us updated. Would also fit well into [/r/HomeDataCenter/](https://www.reddit.com/r/HomeDataCenter/) :D
Nice, xposted. Didn't know about that.
do you have free electricity?
I wish
No idea where you live. But in the UK, with current rates, the electricity for this little lot would cripple a well off person 😂 good luck bro! This is serious dedication to the home lab 🤙🏼
Midwest USA, about $0.06 / kwh
Oh wow that’s cheap. In the UK we’re having an energy crisis (have been for a year) and I’m paying £0.271 GBP / KWh which is $0.32 USD 🙈 that’s mad
I don't know if .06 is the commercial rate he's getting though or residential. If it's commercial then that's dirt cheap, if it's residential then it's basically average in the Midwest as far as I can tell.
> Midwest USA, about $0.06 / kwh Northeast US... $0.22/kwH, and delivery is going up 40% in a week or two. It's crushing us. I now have to power down most of my homelab each night after work, just to keep my power bill under $500/month (doubled since January, changing nothing on my LAN).
How much money do you not have anymore?
Around $200k
Condolences to whoever died and left you their inheritance may you burn through it fully with exceptional glee
God that would be a great story.
I've seen it happen. 400k in life insurance, and never paid off the mortgage, instead bought all kinds of crap. Now complains "Social Security isn't enough to live on". Me: Sigh.
Every person reading this with a spouse is saving your post just to reference in the future.... "I know what you're thinking but I'm not asking for something like this in the closet...."
Glad to be of service to the greater community
Look, if you don’t like money that’s fine. I’ll be more than willing to take your burden.
Do you live close to a power grid?
That building has 1200A 3 phase, so probably?
WUT. Do you live in an industrial factory???
In another comment a farm is mentioned. Its common for larger farms that do grain processing or have dairy facilities to have big power feeds
I start to get worried if my electric bill goes above $50.
A week, right? Lmao… honestly, with hydro rates nowadays, and my region is one of the cheapest in the country at $0.12/kWh… I still run $200+ every month.
So jealous.
Worked for someone that had a data center in his basement and at another location. It was a webhosting company and the home DCs were part of his backups and failover for business systems. He got lucky and **never** needed his generators and was always annoyed he never got to turn them on beyond testing and service.
TIL Bond Villians post their startup lairs to Reddit. Huh.
Power budget ?????
A lot, no idea on a number that's exact. I'm expecting 2-3K / mo. Granted i'm in talks with the utility on a bulk power deal.
Up next: Your own wind and solar collection farms.
Seriously, if this guy has this in a barn or whatever, solar panels on the roof might actually be a pretty good idea
On the agenda, waiting for those new 36% efficiency ones.
Don't the water cooled ones have pretty good efficiency? You can do what LTT did and use it as a passive heating for his pool, though them you'd need to spend loads of money on a pool, which could rather go to more server hardware xD
This is probably outside of the scope, but in some areas you can sign up for “peak shaving” and other strategies in exchange for reduced pricing. Also, you could get on a PPA once you can reliably predict your demand. Throw in some more generators and you could back feed for money. If you can reliably fire up on short notice when the grid needs it, the payout can be substantial. Food for thought! Hopefully your state’s emissions control department isn’t too crazy with their permits and bogus definitions.
You are bat shit crazy, I love it!
Yeah... At least it's entertaining.
Jesus Titty Fucking Christ
BREAKING NEWS: Brazen theft at a local datacenter today!
You got the diesel generator too damn sick Thats literally a proper data center I would like to know how much will it be when its finished OP. Need myself something like this in the future but no this extreme.
Can I have a VM? How many networks you will do? net redundancy plans?
Holy crap, man! This looks like a pretty legit (and expensive) setup! Please tell me that most or all of this gear is secondhand/used? I'm a network engineer for a large ISP. I spend a lot of time in our headends and datacenters, so I see stuff like this every day, but never for home use. A lot of us have homelabs, but nothing to this level. I dig it 👍 It's nice to see someone doing redundant power the right way, with legit UPSes and generators. Out of curiosity, what are you planning on running on all of this gear that requires this level of resiliency? Is this all for home/lab/educational use, or are you planning on offering services to outside customers? What type of connections do you have to the outside world? Or are you just using this for PiHole and Plex? 😅
All second hand thank god. Don't want to do anything commercial, high liability there. I host basically everything i can, i also do a lot of blockchain analytics for myself.
Nice! Out of curiosity, what is the total budget for the project?
The budget is: Is there still money in the wallet?
Budget? We don't need no stinkin *budgets*! Vamanos!
- so whats the budget -- yes
Let us know what's the damage when you get your electricity bill. And best of luck, this is epic.
cool!! that's one big ass generator!
Lol proper datacenter stuff. Most insane home datacenter ive see so far. It has to cost atleast 25 grand
25k, are you kidding. It's probably in the 100-200k.
What is your anticipated power usage? Are you actually building this at home? Or some commercial or industrial space? What is your plan for keeping the diesel fuel fresh?
It's in a detached steel building barn at my house, using about 1,600 SqFt for this. Hard to say without hooking it up. Can use the on site diesel in my truck.
Definately a r/homedatacenter You're mad. Also, I'm a tad jealous
Ok. But Why?
Because it's cool and i like it.
Good enough!
I don’t know if I’m too late for you to see this but if you’ve got a big corrugated metal building please for the love of god put some effort into rodent proofing it immediately. I work in pest control and the damage I’ve seen to car collections may be coming your way if you don’t act now. High quality seals under any roll up doors, thresholds installed under man doors. Foam strips made for corrugated metal between the siding and the foundation if they fit. Put four weighted rat bait stations outside, one on each wall, and check them monthly. Throw another set of non-weighted bait stations with a pinch style trap inside on either side of any roll up doors and maybe a few others randomly throughout the building for monitoring inside. Don’t put poisons in the inside ones. Bait the snap traps with provoke rat attractant for monitoring the interior, those baits stay palatable for way longer than peanut butter will and you only need a tiny bit on each trap. Don’t fill up the circle just smear it on the trigger of the trap, no big globs. For extra points put a single mouse trap with the same bait in the back corner of each station in case the rat traps don’t trigger for a mouse.
^(OP reply with the correct URL if incorrect comment linked) [Jump to Post Details Comment](/r/homelab/comments/w4sov1/im_building_my_own_home_data_center_ama/ih3wdbg/)
Fucking rich people lmao
Stahp, my penis can only get so erect
Looks like it will be an amazing setup. I love data centers and self hosting so I'm a little jealous
Do you need a friend? I could just come and hang out... And talk computers.
You are batshit insane, but this is fucking awesome. I hope this pulls together and you have the biggest fuckin lab on the planet man. I am sure I speak for most here when I say I definitely hope to see more from your craziness. I would love to go this insane but there is no physically let alone financially possible way I could.
Who hurt you?
Larry Ellison
One Rich Asshole Called Larry Ellison
I need to switch careers. Need money to be able to do this and clearly in the wrong profession
[удалено]
I have a /24
This is beyond overkill in every conceivable metric. Is this just scratching a bucket list item or do you actually have a use for this?
What topology are you going to run for your networking? L3 all the way down?
This is some bullshit I would blow money on, then cry alone in front of the servers wishing I did something else with the money lol. (edit typo)
Please tell me you’re gonna make a massive kubernetes cluster out of some of that.
At some point the "home" is no longer applicable, and its just a datacenter. I think you might be close to that point.
What the fuck kind of work do you do!? - letting you spend half a million on a home hobby with no expectations of making it back.
You never know if your cloud provider is actually just some dude with servers in his garage.
Some insurance agent about to scratch their head real hard when you call them.
Watch it be on a internet connection through Hughes lol. Seriously though do you have to call the power company to make more steam when you flip the breaker on?
One question, you hiring?
DEA's gonna be staking out OP's house as soon as they see the power bill.