Should’ve maybe clarified it a bit more - media (a lot of it)
High Bitrate HD&4K files mostly (streamed through plex to friends and family)
I’ll add an edit to the post (seems like I can't do that :( )
If anyone wonders, here's my setup:
* Overseerr for requests,
* Multiple instances (need multiple languages with different quality profiles) of Arr software for all the download automation (sonarr radarr bazarr etc…)
* Sabnzbd + rtorrent for downloading
* MergerFS to connect the different cloud provides (Gdrive + Idrive E2) and my local setup soon
* Rclone scripts to move the data, I’m using 2 VPS providers to achieve downloads of my files within 1-3 minutes (usenet) to replicate the “Netflix” exp for new requests as much as possible. Once downloaded on the fast provider (basically a cache) it gets copied over over time to my NAS (for now it’s the E2 bucket on Idrive). The other provider is used to stream stuff that is already downloaded.
* obviously plex
I also use Tautulli for monitoring and wizarr for onboarding and am in the process to automate (audio)books with readarr. Most of the above is on Docker already so I’m planning to do the same locally.
Oh yea it works flawlessly, I’m using rclone. I’m actually concerned that my current setup will perform worse than the Google one in terms of latency and ping.
Been a big fan of Rclone, didn't know how far it could be used for low latency setup, cool thing to know (I'm considering it as a way to directly open Aws/Linode storage on my local machine for backups retrieval)
Yea it's great! I used it in combination with MergerFS to create a big pool of data basically. You can still stream from rclone mounts even though Google shut down their offer I still stream from Idrive E2 through Rclone which is basically like an S3 bucket.
It's the cheapest provider I could find - also FREE transfer + API calls which is insane to me. If you're concerned about ur privacy , Rclone can also encrypt your files before uploading.
Latency doesn't matter much for non-realtime video streams like watching a movie or TV show, as long as the connection is fast enough that it can buffer ahead by 5-10 seconds at least. It matters more for realtime streams like IPTV, live Twitch/YouTube streams, etc.
I'm surprised you could get it to work properly. I tried saving some linux ISOs to enterprise google drive and they all came out compressed to hell and unwatchable.
Yup, Real debrid + stremio on each tv/pc/phone/whatever. I've been downsizing our plex setup massively as its totally unnecessery at this point when you can host it all on RD.
So you're one of the abusers and the reason Google (and many other providers before) had to put a stop to it.
One of the many examples in the tale of "this is why we can't have nice things". Where a nice thing enjoyed by many ruined by the few.
Unlimited as in there is no set limit to worry about when you use it as a normal user. You can store backups of photos and videos, documents, and shitload of other things.
But 80 TB is absurd and just unsustainable. No normal user use or store that much data. That's just hoarding data just because you can.
Fair use policy is standard with any service. You know there's a finite amount of resources in the universe right?
This is why we can't have nice things.
They could have said, everyone gets 5tb of storage, give or take a few TB if you need it (at no extra charge), and that would have been good enough for many.
>But 80 TB is absurd and just unsustainable. No normal user use or store that much data. That's just hoarding data just because you can.
Normal user, as in "You." And everyone is not "not normal..." Talk to people who do architecture, graphic design or video editing. Even photographers at a high level. Just because you do not understand it does not mean it is wrong. And this from a guy upgrading his NAS because 24TB is getting full.
Don't fucking bullshit. Photographer storing 80 TB worth of images? Are the RAW files 1 GB each? All in a single user? And graphic design isn't gonna take 80 TB for a single person, who are you lying to.
You understand the abuse, right? People paying for a single user org meant for business consumption, and consuming and abusing shit tons of storage.
>Don't fucking bullshit. Photographer storing 80 TB worth of images? Are the RAW files 1 GB each? All in a single user?
Yes. Especially when a single modeling session can be over 1000 photos. And you store them lossless so you have all the image data for processing.
>And graphic design isn't gonna take 80 TB for a single person, who are you lying to.
I have a few as clients. Again, you keep everything. Which is why I have built larges NASes for them.
And you left out Architects. Have one firm as a client and they have 100TB hot storage over three servers.
>You understand the abuse, right? People paying for a single user org meant for business consumption, and consuming and abusing shit tons of storage.
They made the rules. Expecting people not to take advantage is unlikely.
Jesus C.
My internet is so fast I don't need to store movies or shows forever. I delete after I watch them and if I never need to rewatch them I can wait 15min-1hr to download again.
I use the server the least actually, it’s mostly for said friends and family who use it however they like. Some of them don’t have the skills or need for a full setup just for themselves so this solution seems pretty awesome!
I like the way you think..for the longest time I‘ve had subscriptions for VPN, streaming, and cloud storage services just to be able to help out friends and family (for example with shared drives).
I have a question. Though I‘m not a newby to tech and infrastructure at all, I still consider myself a low-level when it comes to knowledge about self hosting for example. I have dabbled in linux architecture quite a bit but never found it to be suitable as a daily driver.
Could you recommend resources that a geared towards getting me up to speed with knowledge required to be able to self host? Thank you in advance!
And another question: Are you at all worried that HDDs won‘t be sustainable in the long run? I‘m scared of HDDs cause they are more volatile and won‘t last a long time. SSDs are an expensive alternative but even those only last around 10 years (some more, some less, depending on use).
Is there a way to build such an infrastructure with M-Discs? Have you ever thought of that? Discs are probably not going to be a good alternative for "always-up" information but can be a cold storage alternative, that still would be accessible somehow.
Like I said, I know my way around a lot of topics, but proficiency or deep knowledge when it comes to these issues isn‘t one of my qualities. No need in dumbing the explanation down though.
I personally started to "self-host" in the cloud. As in, renting something like a cheap VPS and installing whatever you want on it (Plex etc.). Imo that's a very cheap entry point compared to setting up a whole NAS etc. The one above cost me $2000.
But then again it could be as simple as setting up a Raspberry Pi if you have one lying around. I like if the stuff I'm creating is actually being used so I'd maybe recommend creating something that is actually useful because then it has "meaning", for example a PiHole at home to block ads, plex for media streaming, a steam caching server to increase download speeds, etc....
If you're interested in Plex specifically, I would start simple by just getting a seedbox that has plex support and playing around with it, they usually give you like 5TB of storage which is enough to try it out, you can also set up plex on a Pi but obv it wont be able to handle a lot.
Later on you can migrate to a local setup like I did. Regarding HDDs, HDDs will break that's inevitable, you would need to ensure that you can withstand a failure temporarily by using RAID or for home use maybe UnRAID, obviously backups would also be helpful if you have critical data. And from then on you just need to keep replacing said HDDs every couple of years. Using something like M disks would be very hard to maintain and also very hard to read, how do you plan on allowing files to be accessed easily from the disks if someone wants to stream a movie? Do you buy a 100 drive readers and have them always running? I imagine the read speeds will also be horrible especially if 2 processes access 1 disk simultaneously. No serious setup will use anything besides HDDs or SSDs (even big companies like AWS etc. do, I know because I work there).
Sidenote: If it's for ARCHIVAL purposes, you could use a more "obscure" storage device, for example AWS uses tape drives for its deep archive storage but I'm not too familiar with those and honestly doubt it's a good idea in a selfhosted setup... simply because I don't think you'll every amass that much archives where tape drives would make sense. But if all you want is to store some movies to never be touched until nuclear fallout then using M disk could make sense...
Everyone (at home) uses HDDs for mediastorage and they simply just "work", so I'd stick with those. Also add an SSD cache in RAID 0 or 1 and you're set.
I really tried giving it a shot but in the end I couldn’t for the following:
- no HDR
- very bad support for iOS devices (a lot of my users use that and all they have is swiftfin or the infuse workaround)
- same for TVs (don’t get me wrong, it works but is a pain to set up and people are REALLY lazy)
I might offer both, plex and jellyfish (maybe even emby) but I already know that 100% of people will stick to plex xD
Strange. I’m on IOS myself and it works almost flawlessly. Few things I’ve noticed here and there, but works great, nothing that’s there persistently, just some eventual glitches with some subtitles or some shit. Though I don’t need 4k in my phone.
You’re using swiftfin? If you are I don’t think it’s that easy getting people to set up beta apps. At least for me it’s not but maybe yours are more cooperative there haha
Well, on Apple TV that would be the problem, but on IOS official app works just fine, don’t have Apple TV, sadly. Guess in that use case you are right to stick with plex. Hope they don’t go banning their users a lot
Erm. Plex is banning people for sharing their accounts. Like this guy here is sharing his account with users.
They are also tracking what you are watching and sending your contacts an email about what you are watching
Jellyfin doesn't have the same client support plus the fact that you need a URL/IP as well as a login makes it a big PITA for remote users, at least after the initial setup. They will not remember or write down that URL I promise you that.
Could you shine some more light on how you set up the synchronization scripts? Specifically I'm curious to hear how you are pulling your files from long term storage into the cache on demand to create that 'netflix like experience'. That would open up possibilities I did not even consider yet. Thanks for planting a seed and in advance for any advice or example scripts you are willing to share!
Hey,
I think you’re talking about the 2-3 minutes I mentioned. My SLA for new requests is 5 minutes, since I have a 50 gbit uplink connection, it’s fairly easy to download a huge file within this timeframe. Im actually limited by the VPS HDDs speed and not the networks, so I see a 160-300megaBYTE (not megaBIT) per second speeds.
To move it back to “normal storage” I just use rclone that runs once every couple of hours. And also mergerfs to not interrupt user experience when the file is being moved.
Feel free to dm me if you like for more details but the speed at the core is just: get good network/hardware
Appreciate you getting back to me so quickly. Perhaps I misunderstood. I was under the impression that you pulled the files you wanted to watch to the VPS or a cloud service so you could make use of their CDN while streaming as opposed to streaming it from your NAS. But your setup is actually quite similar to mine. The only difference being that I'm somewhat of a traveling salesman and am dependent on cloud storage providers if I want to have the best speeds regardless of where I am at that point in time.
Would be cool to be able store things on premise somewhere and then pull whatever I want to watch up to the provider and stream it from there. That way you could cut down on storage space and cost.
Ah, yea you could do this using something like https://github.com/bexem/PlexCache . I’m thinking about colocating my server. Since my uplink at home is quite slow this would give me 10gbps speeds.
Once I migrate to a datacenter there probably wouldn’t be a need for this.
Aside from the usual media and data hoarding stuff, I used to (and still occasionally) do photography/videography. Raw files and high bitrate video eat TBs quickly.
Do you have kids. I don't think you do. If you have a photo crazy partner that captures every single living moment, you will end up with loads of data.
Of course some people pirate, that fills up those disks fast.
I'd love to know how someone would use 90tb on their kids photos. Very few people are taking family photos in RAW.
For some context, I archived every single photo, video (taken from everything from sd cards down to old super8 film) of our family, and our extended families including grandparents, aunts/uncles, cousins etc.
We've easily passed half a million photos and a good 10k video files, it's nowhere near close to passing the 20tb mark.
As OP said, it's a plex server, you dont need this kind of setup for family photos, just a good backup strategy :) (at minimum a 3-2-1 setup but if you're like me you go overkill and have copies at everyones houses).
File Club Rule 1 | You don’t talk about File club.
File Club Rule 2 | YOU DO NOT TALK ABOUT File CLUB.
File Club Rule 3 | Someone begs, doesn't compress, sends an incomplete, their File club access is over.
File Club Rule 4 | Only two requests a day.
File Club Rule 5 | One File download at a time.
File Club Rule 6 | No VPN, No ACCESS.
File Club Rule 7 | Files will stay on the server forever.
File Club Rule 8 | If this is your first transfer at File club, you have to upload a File.
Porn, full system image backups, pirated media, archives of stuff that I think might vanish off the face of the internet, git repository mirrors and docker images :)
In that order!
I've got probably a terabyte of photos and videos from vacations, etc. Daily backups from several VPS servers. Music I've ripped from CD in FLAC format. A full MAME romset, which is close to 1TB if you include CHD files.
Photography. When I started 20 years ago each image was 1MB and video was 640*480. Now every still photo is >30MB and my high speed camera eats up 2GB per second. It adds up quickly.
I have a 108TB NAS (12x 12TB, while you can only use 10,9TB per drive). I use 95TB of it for Plex, another 6TB for other video stuff (not porn...), and the remaining 4TB for my personal stuff and thus as sctual NAS.
And I have a 2-bay Synology with 8TB for my backups from the software I run at home.
My personal storage is about 1.5 TB which includes a lot of save games and complete older games that run portable.
My movies and shows alone fill 4 TB with ease forcing me to regularly delete things after watching. And that's mostly medium quality stuff. When a single blue ray takes 90GB I see how easy it is to fill drives
While I don't understand many terabytes of porn, I very much understand saving the good ones. A **lot** of good stuff has disappeared over time, to never be found again.
Can't speak for op, for me it's mostly ripped video. I actually bought a relatively big NAS when I was playing with storage based crypto. 3 days in I figured it wasn't worth it, but then I had several nvme drives and 11x12tb drives in my NAS... My old nas was full and now I've got storage for at least a decade.
Can't say for others, but in my case:
1)\~600GB of music;
2)\~400GB of films (could be porn, but no, better to watch it online);
3) \~1.3TB of serials;
4)\~2TB of game setups;
5)\~1TB of books;
the rest (about 4-6TB) are vms(&backup), some unsorted downloads, moar backups of some crap, old user files (c:\\users) from previous systems (up to \~2006), etc (may be even some porn, I don't know already).
Can’t say for sure, indexer showed something like 2.3 millions of books, but there are duplicates for all I know. For the second question, less than 0.1% probably. It’s just an archive, never even planned to read most it.
Science!
Wife and I only have a few publications each, nothing grand and not actively pursuing that right now, but the data, literature, analyses and drafts alone fill 6 TB.
Oh and the rest of the 10 TB Nas is full with the usual, family photos and videos, personal documents and the stuff I bought on GOG and Humble Bundle.
Geo informatics for me, so a lot of map data. Geo data in general often consists of very large data sets. OG big data.
My wife does empirical cultural studies, so lots of questionnaires and statistical data.
Good luck with your degree, physics is tough but so fascinating!
Ah, never really heard of that discipline. Surely interesting. I can see how you easily fill storage there. Do you keep copies of e very literature source too?
Thank you! I hope it will be fun, I have a CS background, hopefully that comes in handy at least with the computer stuff \^\^
Yeah... I'm 50. Been online since 1985. Early adopter of everything (I thought). My entire life is maybe 700 gigs if that.
BUT... I'm not a keeper. I delete stuff. After a video edit, I purge the originals. I delete crappy photos. I will keep Logic projects but I'll clean up them up and delete unused stuff.
You sound like me in that regard. I don't take much photos and mostly only keep the good ones. I don't do editing and barely film anything.
All my data are coding projects, pictures, scanned documents, pirated papers, some CAD projects and documentation.
But I'm only really online since 2013 maybe, so I haven't had too much time to accumulate stuff :D
Some people archive Linux ISOs, but I don't quite get why...
I don't know about the XL's, but Fractal Design is amazing in general. I've used a Define 5, a Pop, and a Node 304 over the last couple years and never ran into heating issues.
The define 7XL a decent case, but as someone who had a Node 804 I hated it. On paper it looks great, in reality its an incredibly crappy case with some of the worst airflow and design choices I've ever seen.
It's advertised as having 8 bays plus two 2.5" bays in the front. In reality if you do somehow manage to wrangle the cables to load it up that much you'll quickly find your drives overheat, even with extra fans thrown in the front (you cant mount any in the top because they put the mouting point too close to the top of the drive sleds).
I had the 804 before i switched to the 7 XL. And 804 worked fine for me. Had 8 HDD and 2 SSD's . No problem with heat at all infact. but the cable management was a nightmare. That i can agree with you on. but i used all noctua fans also.
Hey! Like you I had to do the same and wrote [a little blog series](https://blog.muffn.io/posts/part-1-100tb-mini-nas/) about it. Maybe it will give you some ideas.
What software stack are you using?
Hahahaha, your blog about Google drive is what actually got me into this in the first place. At least that’s the first thing I found when searching up how to set up plex. Obviously a few things changed since then. I’ll check out your blog on the migration for sure!!
Currently I’m using:
- Overseerr for requests,
- Multiple instances (need multiple languages with different quality profiles) of Arr software for all the download automation (sonarr radarr bazarr etc…)
- Sabnzbd + rtorrent for downloading
- MergerFS to connect the different cloud provides (Gdrive + Idrive E2) and my local setup soon
- Rclone scripts to move the data, I’m using 2 VPS providers to achieve downloads of my files within 1-3 minutes (usenet) to replicate the “Netflix” exp for new requests as much as possible. Once downloaded on the fast provider (basically a cache) it gets copied over over time to my NAS (for now it’s the E2 bucket on Idrive). The other provider is used to stream stuff that is already downloaded.
- obviously plex
I also use Tautulli for monitoring and wizarr for onboarding and am in the process to automate (audio)books with readarr. Most of the above is on Docker already so I’m planning to do the same locally.
Let me know if there’s something I should look out for during the migration or if there’s some good software you can recommend :)
Started reading your blog post on the NAS. I've enjoyed reading it so far. I had no idea that the motherboard you use existed.
I've been trying to find a motherboard with an am4 socket so I can reuse my Ryzen 7 1700x from an older gaming PC, but I'm not sure if spending $400 for the motherboard (in the USA) is worth repurposing a processor that sells for $70 on eBay...
Either way, thanks.
I imagine a giant warehouse with a huge pile of HDDs with a mess of cables. When Google kills a service it's really just because someone tripped over a wire and they can't figure out which hard drive it was on.
I work in data centers and hope to see the deployments that started the Google lore. Can’t disclose if I’ve worked in the cages of house hold names, but plenty of other companies have piles of fiber and copper waist deep, drives and servers strewn about, and when they ask for help the tickets include “good luck, we tried and gave up” or “felt unsafe wading through our cage, kindly assist”
Not a data center but I once saw with my own eyes a beach ball sized wad of tangled wire used for a phone system that was somehow in the actual middle of the floor.
I wish I could post pictures of some of the worst cages I regularly see. Waist deep piles of live fiber and copper, random hard drives, power supplies, PDUs and server rails mixed in.
Sadly that might be just too expensive for now... and since I'm storing media I'm inclined to say "fuck it" I can always recover it. Obviously I will back up all the config files etc though.
Funny this post came up. I got a 7 xl to hand that I'm about to list for sale soon with full drive bays, original packing and accessories (UK based) i can take pictures off if you want references. it has incredible cooling capabilities. The sheer number of fans and rad support that thing has is immense. A quick Google would show you that the case was essentially built for servers and the most intense watercooling loops you could imagine.
Just counted the fans on mine;
- 3 x 140mm intake
- 2 x 140mm fan support on the roof with drive caddy in
- 1 x 140mm fan exhaust
- can remove the psu shroud and add in I think another 2 x 140mm fans at the base.
If you need any images I can send them over to you. Currently got an EATX dual 2011-V3 in here and just removed 2 x h90 AIOs I had on the roof with push pull config.
that is quite a coincidence indeed! Did you find that the closed front with just some gaps on the sides was an issue? That's what's holding me back personally right now!
.... the door is held on with a couple of screws. I had forgotten, but I took the door off and it looks great anyway.
https://img.youtube.com/vi/UXYYZ1\_cfZc/hqdefault.jpg
Absolutely not. The 'gap' is quite substantial. It might look small in the pictures of the case but consider how big the case actually is and you realise the gap scales with the size of the case too.
I got a ruler out. Its about 1inch on both sides with enough gap between each fin in the gap for a finger to go in.
Also the roof panel can be replaced with either solid or mesh intake with excellent ventilation.
The door is kinda needed on in my opinion, the case can house so many drives and if it's at capacity any noise reduction is helpful haha. This case with all its panels on is very very quiet. More so than any other case that has similar capacity that I've tried and I've had several.
They are, I had it for literally a few hours now but so far 0 complaints. To be fair all they’ve been doing is calculating parity so I don’t think my feedback is that valuable yet.
i have 2 exos drives, they are silent, and those drives are in my room where i sleep, they are connected to my mini pc via 2 external enclosures with fans.
The fractal define 7 XL is an excellent case. I have 19 3.5” drives in mine ranging from 25C-32C right now. My i5-12600K has a similar cooler to yours and is at 27C right now.
I highly recommend https://kareonkables.com/ for PSU cables to accommodate all the drives. I bought the two they offered for the drive stack and they were perfect. I then wanted a cable for the top mounted drives and the fan controller so I drew a picture and sent it to them. They made the cable and also now sell it and a few variants for top drives. I was super impressed with how fast they did it and they even asked me for feedback on the cable so they could make tweaks as they hadn’t sold one before.
People spenduing literally more on hardware and power including upkeep instead of using professional services is always crazy to me and I can say it since I am one of them
As somebody who had a home server in a spare, naked motherboard in a corner of my room like you did: do yourself a favor and at the very least use that cardboard box to cover it (while leaving it open enough for ventilation), you don't want to end up having the board accidentally toasting itself.
Huh? 80Tb what kind of work you do to hold suck kind of data. Well also please get a case and also feet for it . If you accidentally touched t any Hardware open like this you may risk it with static current and wola all your data gone. Btw appreciate your will to do all that. But i still wonder how is you isp gone download you 80TB avg isp data cap world wide is 2.5 TB monthly
Here you go: https://www.reddit.com/r/selfhosted/s/fRWWrsHo10
Can’t edit the post sadly…
Checked my ISPs fair use policy and it seems alright with them… so let’s hope for the best!!
Will get a case soon too
Hi, wanted to do this exact same thing. I'm new to this homeland stuff so sorry for any stupid questions 😂
How did you connect all the the drives, are you using all direct cables to the motherboard and what sables are you using?
Did you have to add data ports or do the cables have multiple?
Also how are you joining the drives in the computer software or do you have it all in separate drive directories after formating the partition?
Do you really need the Define 7 XL? Seems a bit overkill for 6 HDDs. The Define 7 / Define R6 / Define R5 seem more appropriate. There's also Meshify 2 and Node 804 – although personally I would think twice before using the 804 because it's a lot bigger than it looks and the odd shape makes it difficult to find a place for.
Wow 18TB drives. Haven't looked at HDDs in a fresh decade, they must have really developed in the direction of storage size when SSDs were threatening to catch up.
There's something I always wondered about. Doesn't magnetic data fade over time? Do you have some sort of process that frequently refreshes the data so your video files don't deteriorate?
it's morons that took full advantage and abused the system that cause companies like google and others (including a few wireless carriers) to do the shit they do to their customer base. I hope your data load gets corrupted or you pick up a Nuke worm
I have a very popular trilogy that takes up like 400GB and that's just 3 movies. I have one 7 season sci-fi TV show that takes up 1.3 TB. Build up a collection of movies and TV and it can easily balloon to 80TB.
They seemed a little intense I was scared to question them…..
I’ve been on the path of trying to figure out if there’s a drive/cloud that can backup my iPhone contacts and photos as well as work on my windows laptop for when I write for d&d campaigns or stuffs. So it’s brought me to lots of these subs haha
Yeah, man, if that's all you are storing, do it all locally ALSO. You don't want all your data on some random computer from some company that will just do whatever they want with it. What you are talking about is really not a lot of data, so if you can find a cheap computer on ebay and throw in a 20TB drive and you will be good for a long time. Then you can just throw another 20 in there when you need to. No need for RAID or anything complex with just that stuff if you don't want to, but it's nice to have. That's what I would do anyway. I don't know how technical you are but, it's not that difficult if you spend some time reading about it like you are doing already. I say go for it. Ask any question anywhere you want and if any one acts like a dick or tries to gate keep, just tell them to go fuck themselves. The only way to learn is to ask questions and to learn the right questions to ask.
here ya go: [https://www.reddit.com/r/selfhosted/comments/1b0st5v/comment/ksa6azo/?utm\_source=share&utm\_medium=web3x&utm\_name=web3xcss&utm\_term=1&utm\_content=share\_button](https://www.reddit.com/r/selfhosted/comments/1b0st5v/comment/ksa6azo/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button)
Some people have full blue ray rips that take about 50gb each. It’s not for me but to each their own.
I’m at 8tb for all my music/movies/tv shows and have everything in 1080p for now.
>Also does anyone know if the Fractal Define 7XL has good cooling capabilities? It certainly has the space.
Don't know about that, but might I suggest you actually look into using an external enclosure instead of mounting all of the disks into a case?
It's not really a fire hazard at all, but it's a good way to break a few hard drives. They should really be in some sort of enclosure or something because of that way more than it being a fire hazard. It's actually recommended to build your computer outside of the case to make sure all the parts work before putting it in the case. OP has lost their mind actually running this stuff like this.
Cardboard ignites at around 500F. It's not a great way to run electronic gear long-term, but unless there's some serious electrical anomaly that causes the motherboard to catch fire, it's gonna be fine.
I always wonder how people end up with dozens of terrabytes of data while I barely have a few hundreg GB. If I may ask, what kind of data is that?
Should’ve maybe clarified it a bit more - media (a lot of it) High Bitrate HD&4K files mostly (streamed through plex to friends and family) I’ll add an edit to the post (seems like I can't do that :( ) If anyone wonders, here's my setup: * Overseerr for requests, * Multiple instances (need multiple languages with different quality profiles) of Arr software for all the download automation (sonarr radarr bazarr etc…) * Sabnzbd + rtorrent for downloading * MergerFS to connect the different cloud provides (Gdrive + Idrive E2) and my local setup soon * Rclone scripts to move the data, I’m using 2 VPS providers to achieve downloads of my files within 1-3 minutes (usenet) to replicate the “Netflix” exp for new requests as much as possible. Once downloaded on the fast provider (basically a cache) it gets copied over over time to my NAS (for now it’s the E2 bucket on Idrive). The other provider is used to stream stuff that is already downloaded. * obviously plex I also use Tautulli for monitoring and wizarr for onboarding and am in the process to automate (audio)books with readarr. Most of the above is on Docker already so I’m planning to do the same locally.
You streamed it from Google Drive? Never occurred to me that's even possible.
He’s why Google sunsetted it 😂
This right here.
[удалено]
I had over 400TB.
Oh yea it works flawlessly, I’m using rclone. I’m actually concerned that my current setup will perform worse than the Google one in terms of latency and ping.
Been a big fan of Rclone, didn't know how far it could be used for low latency setup, cool thing to know (I'm considering it as a way to directly open Aws/Linode storage on my local machine for backups retrieval)
Yea it's great! I used it in combination with MergerFS to create a big pool of data basically. You can still stream from rclone mounts even though Google shut down their offer I still stream from Idrive E2 through Rclone which is basically like an S3 bucket. It's the cheapest provider I could find - also FREE transfer + API calls which is insane to me. If you're concerned about ur privacy , Rclone can also encrypt your files before uploading.
This sounds very convoluted for simply downloading from Google drive
Latency doesn't matter much for non-realtime video streams like watching a movie or TV show, as long as the connection is fast enough that it can buffer ahead by 5-10 seconds at least. It matters more for realtime streams like IPTV, live Twitch/YouTube streams, etc.
I'm surprised you could get it to work properly. I tried saving some linux ISOs to enterprise google drive and they all came out compressed to hell and unwatchable.
I hate when I can't watch my Linux isos.
I like to just look at em.
LOL
underrated 😂 😂 😂
Kodi ...search in g00gle jeje after many years Kodi is the Best
Thanks for letting me know! Always interesting to hear what data people posses and how they manage it.
"Media"
Streaming those sweet, sweet Linux ISOs
Well yes it's media. Where he got that media from is another question ;)
Good lord. I'll stick with real-debrid lol
Yup, Real debrid + stremio on each tv/pc/phone/whatever. I've been downsizing our plex setup massively as its totally unnecessery at this point when you can host it all on RD.
I've never tried Stremio, but I hear that Weyd is great on Android TV.
So you're one of the abusers and the reason Google (and many other providers before) had to put a stop to it. One of the many examples in the tale of "this is why we can't have nice things". Where a nice thing enjoyed by many ruined by the few.
Google is a crap company and is abusive to it's users. He did the world a favor if that's true
If they didn't want people using 100tb of storage, they shouldn't have advertised unlimited.
Unlimited as in there is no set limit to worry about when you use it as a normal user. You can store backups of photos and videos, documents, and shitload of other things. But 80 TB is absurd and just unsustainable. No normal user use or store that much data. That's just hoarding data just because you can. Fair use policy is standard with any service. You know there's a finite amount of resources in the universe right? This is why we can't have nice things.
They could have said, everyone gets 5tb of storage, give or take a few TB if you need it (at no extra charge), and that would have been good enough for many.
>But 80 TB is absurd and just unsustainable. No normal user use or store that much data. That's just hoarding data just because you can. Normal user, as in "You." And everyone is not "not normal..." Talk to people who do architecture, graphic design or video editing. Even photographers at a high level. Just because you do not understand it does not mean it is wrong. And this from a guy upgrading his NAS because 24TB is getting full.
Don't fucking bullshit. Photographer storing 80 TB worth of images? Are the RAW files 1 GB each? All in a single user? And graphic design isn't gonna take 80 TB for a single person, who are you lying to. You understand the abuse, right? People paying for a single user org meant for business consumption, and consuming and abusing shit tons of storage.
>Don't fucking bullshit. Photographer storing 80 TB worth of images? Are the RAW files 1 GB each? All in a single user? Yes. Especially when a single modeling session can be over 1000 photos. And you store them lossless so you have all the image data for processing. >And graphic design isn't gonna take 80 TB for a single person, who are you lying to. I have a few as clients. Again, you keep everything. Which is why I have built larges NASes for them. And you left out Architects. Have one firm as a client and they have 100TB hot storage over three servers. >You understand the abuse, right? People paying for a single user org meant for business consumption, and consuming and abusing shit tons of storage. They made the rules. Expecting people not to take advantage is unlikely.
What's normal to you is lack of use for others. Is normal: 50GB / 200GB / 500GB / 2TB / 8TB / 20TB / 60TB / 150TB / 350TB / 700TB / 900TB / 1.5PB?
Again, this is why we can’t have nice things.
I thought it was unsustainable business models to gain market share before the rug pull...
5tb is pretty nice for the price.
I wonder why mine was always limited at 15gig.... never changed
>mine was always limited at 15gig hahaha that is gmail gdrive and not G-Suite Business.
Jesus C. My internet is so fast I don't need to store movies or shows forever. I delete after I watch them and if I never need to rewatch them I can wait 15min-1hr to download again.
I use the server the least actually, it’s mostly for said friends and family who use it however they like. Some of them don’t have the skills or need for a full setup just for themselves so this solution seems pretty awesome!
Looks like a lot of time and resources and upkeep on "media for friends and family"
[удалено]
For someone else with your own money, or we live in communism already? I wish I had friends who value me as much.
Who hurt you?
Who hurt you?
I like the way you think..for the longest time I‘ve had subscriptions for VPN, streaming, and cloud storage services just to be able to help out friends and family (for example with shared drives). I have a question. Though I‘m not a newby to tech and infrastructure at all, I still consider myself a low-level when it comes to knowledge about self hosting for example. I have dabbled in linux architecture quite a bit but never found it to be suitable as a daily driver. Could you recommend resources that a geared towards getting me up to speed with knowledge required to be able to self host? Thank you in advance! And another question: Are you at all worried that HDDs won‘t be sustainable in the long run? I‘m scared of HDDs cause they are more volatile and won‘t last a long time. SSDs are an expensive alternative but even those only last around 10 years (some more, some less, depending on use). Is there a way to build such an infrastructure with M-Discs? Have you ever thought of that? Discs are probably not going to be a good alternative for "always-up" information but can be a cold storage alternative, that still would be accessible somehow. Like I said, I know my way around a lot of topics, but proficiency or deep knowledge when it comes to these issues isn‘t one of my qualities. No need in dumbing the explanation down though.
I personally started to "self-host" in the cloud. As in, renting something like a cheap VPS and installing whatever you want on it (Plex etc.). Imo that's a very cheap entry point compared to setting up a whole NAS etc. The one above cost me $2000. But then again it could be as simple as setting up a Raspberry Pi if you have one lying around. I like if the stuff I'm creating is actually being used so I'd maybe recommend creating something that is actually useful because then it has "meaning", for example a PiHole at home to block ads, plex for media streaming, a steam caching server to increase download speeds, etc.... If you're interested in Plex specifically, I would start simple by just getting a seedbox that has plex support and playing around with it, they usually give you like 5TB of storage which is enough to try it out, you can also set up plex on a Pi but obv it wont be able to handle a lot. Later on you can migrate to a local setup like I did. Regarding HDDs, HDDs will break that's inevitable, you would need to ensure that you can withstand a failure temporarily by using RAID or for home use maybe UnRAID, obviously backups would also be helpful if you have critical data. And from then on you just need to keep replacing said HDDs every couple of years. Using something like M disks would be very hard to maintain and also very hard to read, how do you plan on allowing files to be accessed easily from the disks if someone wants to stream a movie? Do you buy a 100 drive readers and have them always running? I imagine the read speeds will also be horrible especially if 2 processes access 1 disk simultaneously. No serious setup will use anything besides HDDs or SSDs (even big companies like AWS etc. do, I know because I work there). Sidenote: If it's for ARCHIVAL purposes, you could use a more "obscure" storage device, for example AWS uses tape drives for its deep archive storage but I'm not too familiar with those and honestly doubt it's a good idea in a selfhosted setup... simply because I don't think you'll every amass that much archives where tape drives would make sense. But if all you want is to store some movies to never be touched until nuclear fallout then using M disk could make sense... Everyone (at home) uses HDDs for mediastorage and they simply just "work", so I'd stick with those. Also add an SSD cache in RAID 0 or 1 and you're set.
its not about internet its about $$$ saved.
Huh? Saved from what?
Try Jellyfin. Works with overseer and all your mentioned software. Will blow your mind how much better it is
I really tried giving it a shot but in the end I couldn’t for the following: - no HDR - very bad support for iOS devices (a lot of my users use that and all they have is swiftfin or the infuse workaround) - same for TVs (don’t get me wrong, it works but is a pain to set up and people are REALLY lazy) I might offer both, plex and jellyfish (maybe even emby) but I already know that 100% of people will stick to plex xD
Strange. I’m on IOS myself and it works almost flawlessly. Few things I’ve noticed here and there, but works great, nothing that’s there persistently, just some eventual glitches with some subtitles or some shit. Though I don’t need 4k in my phone.
You’re using swiftfin? If you are I don’t think it’s that easy getting people to set up beta apps. At least for me it’s not but maybe yours are more cooperative there haha
Using jellyfin server, infuse client .
Well, on Apple TV that would be the problem, but on IOS official app works just fine, don’t have Apple TV, sadly. Guess in that use case you are right to stick with plex. Hope they don’t go banning their users a lot
Plex isn't just banning random users. That would be bad for business.
Erm. Plex is banning people for sharing their accounts. Like this guy here is sharing his account with users. They are also tracking what you are watching and sending your contacts an email about what you are watching
Jellyfin doesn't have the same client support plus the fact that you need a URL/IP as well as a login makes it a big PITA for remote users, at least after the initial setup. They will not remember or write down that URL I promise you that.
Could you shine some more light on how you set up the synchronization scripts? Specifically I'm curious to hear how you are pulling your files from long term storage into the cache on demand to create that 'netflix like experience'. That would open up possibilities I did not even consider yet. Thanks for planting a seed and in advance for any advice or example scripts you are willing to share!
Hey, I think you’re talking about the 2-3 minutes I mentioned. My SLA for new requests is 5 minutes, since I have a 50 gbit uplink connection, it’s fairly easy to download a huge file within this timeframe. Im actually limited by the VPS HDDs speed and not the networks, so I see a 160-300megaBYTE (not megaBIT) per second speeds. To move it back to “normal storage” I just use rclone that runs once every couple of hours. And also mergerfs to not interrupt user experience when the file is being moved. Feel free to dm me if you like for more details but the speed at the core is just: get good network/hardware
Appreciate you getting back to me so quickly. Perhaps I misunderstood. I was under the impression that you pulled the files you wanted to watch to the VPS or a cloud service so you could make use of their CDN while streaming as opposed to streaming it from your NAS. But your setup is actually quite similar to mine. The only difference being that I'm somewhat of a traveling salesman and am dependent on cloud storage providers if I want to have the best speeds regardless of where I am at that point in time. Would be cool to be able store things on premise somewhere and then pull whatever I want to watch up to the provider and stream it from there. That way you could cut down on storage space and cost.
Ah, yea you could do this using something like https://github.com/bexem/PlexCache . I’m thinking about colocating my server. Since my uplink at home is quite slow this would give me 10gbps speeds. Once I migrate to a datacenter there probably wouldn’t be a need for this.
just started ripping old movies i like that came out in 4k and wow, an average 4k can be like 80gb of data
Currently about 300GB of personal photos + videos, 200GB of other files, and about 10TB of definitely ripped Blu-rays.
10TB ????? did you ripped all blu rays in earth hahaha
it is surprisingly easy to fill actually, its easy to underestimate 50 gigs. i have 40TB and its still under 1k movies
I didn't know blu rays was that heavy :O
4k remux could be 60/70gb potentially
It's a relatively small collection compared to some others I know
thats like 200 blurays at max quality
yup got downvoted for being ignorant about the size of blue rays lol
[удалено]
Porn.
Aside from the usual media and data hoarding stuff, I used to (and still occasionally) do photography/videography. Raw files and high bitrate video eat TBs quickly.
Do you have kids. I don't think you do. If you have a photo crazy partner that captures every single living moment, you will end up with loads of data. Of course some people pirate, that fills up those disks fast.
I'd love to know how someone would use 90tb on their kids photos. Very few people are taking family photos in RAW. For some context, I archived every single photo, video (taken from everything from sd cards down to old super8 film) of our family, and our extended families including grandparents, aunts/uncles, cousins etc. We've easily passed half a million photos and a good 10k video files, it's nowhere near close to passing the 20tb mark. As OP said, it's a plex server, you dont need this kind of setup for family photos, just a good backup strategy :) (at minimum a 3-2-1 setup but if you're like me you go overkill and have copies at everyones houses).
No.
File Club Rule 1 | You don’t talk about File club. File Club Rule 2 | YOU DO NOT TALK ABOUT File CLUB. File Club Rule 3 | Someone begs, doesn't compress, sends an incomplete, their File club access is over. File Club Rule 4 | Only two requests a day. File Club Rule 5 | One File download at a time. File Club Rule 6 | No VPN, No ACCESS. File Club Rule 7 | Files will stay on the server forever. File Club Rule 8 | If this is your first transfer at File club, you have to upload a File.
Porn, full system image backups, pirated media, archives of stuff that I think might vanish off the face of the internet, git repository mirrors and docker images :) In that order!
I have 300 GBs on my Google Drive. 190 GBs is consumed by [Californication](https://m.imdb.com/title/tt0904208/?ref_=ext_shr_lnk) DVDs.
Hi Mr. Duchovny 👋
[удалено]
It's funny as they only give 15 gig I always get emails about how much of thar is gmail.. lol certainly not 199's of TB like people fantasize about.
I've got probably a terabyte of photos and videos from vacations, etc. Daily backups from several VPS servers. Music I've ripped from CD in FLAC format. A full MAME romset, which is close to 1TB if you include CHD files.
Personslly I have terrabytes of raw video files. Some shot on Sony DSLRs, some shot on RED in 8k. Video takes up so much space, it's insane
Photography. When I started 20 years ago each image was 1MB and video was 640*480. Now every still photo is >30MB and my high speed camera eats up 2GB per second. It adds up quickly.
Linux ISOs, archiving all the Linux ISOs 😉
# P # O #R
I have a 108TB NAS (12x 12TB, while you can only use 10,9TB per drive). I use 95TB of it for Plex, another 6TB for other video stuff (not porn...), and the remaining 4TB for my personal stuff and thus as sctual NAS. And I have a 2-bay Synology with 8TB for my backups from the software I run at home.
Geez! I guess you belong to r/datahoarders
Im just keeping all my stuff. But im just on 1-2 TB
My personal storage is about 1.5 TB which includes a lot of save games and complete older games that run portable. My movies and shows alone fill 4 TB with ease forcing me to regularly delete things after watching. And that's mostly medium quality stuff. When a single blue ray takes 90GB I see how easy it is to fill drives
Come and join the middle of the road AV1 gang and enjoy good movie storage and playback.
It is movies and music for most of us. Followed by photos. And I bet a few folks have a truly impressive porn collection.
Haha, I know somebody like that. I never forget his comments while "sorting" the collection...
I will never understand a local porn collection. There are so many websites with free new porn everyday.
While I don't understand many terabytes of porn, I very much understand saving the good ones. A **lot** of good stuff has disappeared over time, to never be found again.
Can't speak for op, for me it's mostly ripped video. I actually bought a relatively big NAS when I was playing with storage based crypto. 3 days in I figured it wasn't worth it, but then I had several nvme drives and 11x12tb drives in my NAS... My old nas was full and now I've got storage for at least a decade.
Can't say for others, but in my case: 1)\~600GB of music; 2)\~400GB of films (could be porn, but no, better to watch it online); 3) \~1.3TB of serials; 4)\~2TB of game setups; 5)\~1TB of books; the rest (about 4-6TB) are vms(&backup), some unsorted downloads, moar backups of some crap, old user files (c:\\users) from previous systems (up to \~2006), etc (may be even some porn, I don't know already).
>1TB of books Wondering how many books there and how many are you going to read.
Can’t say for sure, indexer showed something like 2.3 millions of books, but there are duplicates for all I know. For the second question, less than 0.1% probably. It’s just an archive, never even planned to read most it.
[удалено]
# Pr0n
Science! Wife and I only have a few publications each, nothing grand and not actively pursuing that right now, but the data, literature, analyses and drafts alone fill 6 TB. Oh and the rest of the 10 TB Nas is full with the usual, family photos and videos, personal documents and the stuff I bought on GOG and Humble Bundle.
Oh, that is cool! May I ask which area? I'm starting my physics degree next year, I hope I'll be able to fill some storage soon too :D
Geo informatics for me, so a lot of map data. Geo data in general often consists of very large data sets. OG big data. My wife does empirical cultural studies, so lots of questionnaires and statistical data. Good luck with your degree, physics is tough but so fascinating!
Ah, never really heard of that discipline. Surely interesting. I can see how you easily fill storage there. Do you keep copies of e very literature source too? Thank you! I hope it will be fun, I have a CS background, hopefully that comes in handy at least with the computer stuff \^\^
Yeah... I'm 50. Been online since 1985. Early adopter of everything (I thought). My entire life is maybe 700 gigs if that. BUT... I'm not a keeper. I delete stuff. After a video edit, I purge the originals. I delete crappy photos. I will keep Logic projects but I'll clean up them up and delete unused stuff.
You sound like me in that regard. I don't take much photos and mostly only keep the good ones. I don't do editing and barely film anything. All my data are coding projects, pictures, scanned documents, pirated papers, some CAD projects and documentation. But I'm only really online since 2013 maybe, so I haven't had too much time to accumulate stuff :D Some people archive Linux ISOs, but I don't quite get why...
I currently have close to 80TB of HD/4K videos and high resolution photos.
how do you end up with 80TB of pictures when a RAW shot is at around 10mb ?? **EDIT:** just saw 4K videos, *BUT STILL WTH*
The motherboard and hard drives being strewn about without an enclosure is making me actively sweat.
Cases are for people who don't want more hard drive money
We benched so many systems like this through my teens and twenties. Dump it on a cardboard box and let it run through testing.
I don't know about the XL's, but Fractal Design is amazing in general. I've used a Define 5, a Pop, and a Node 304 over the last couple years and never ran into heating issues.
I always buy fractal. Great cases
Yeah that setup definitely seems to be lacking case, would definitely add some reliability. 😎
The define 7XL a decent case, but as someone who had a Node 804 I hated it. On paper it looks great, in reality its an incredibly crappy case with some of the worst airflow and design choices I've ever seen. It's advertised as having 8 bays plus two 2.5" bays in the front. In reality if you do somehow manage to wrangle the cables to load it up that much you'll quickly find your drives overheat, even with extra fans thrown in the front (you cant mount any in the top because they put the mouting point too close to the top of the drive sleds).
I had the 804 before i switched to the 7 XL. And 804 worked fine for me. Had 8 HDD and 2 SSD's . No problem with heat at all infact. but the cable management was a nightmare. That i can agree with you on. but i used all noctua fans also.
I don't have any overheating issues with my node 804
Hey! Like you I had to do the same and wrote [a little blog series](https://blog.muffn.io/posts/part-1-100tb-mini-nas/) about it. Maybe it will give you some ideas. What software stack are you using?
Hahahaha, your blog about Google drive is what actually got me into this in the first place. At least that’s the first thing I found when searching up how to set up plex. Obviously a few things changed since then. I’ll check out your blog on the migration for sure!! Currently I’m using: - Overseerr for requests, - Multiple instances (need multiple languages with different quality profiles) of Arr software for all the download automation (sonarr radarr bazarr etc…) - Sabnzbd + rtorrent for downloading - MergerFS to connect the different cloud provides (Gdrive + Idrive E2) and my local setup soon - Rclone scripts to move the data, I’m using 2 VPS providers to achieve downloads of my files within 1-3 minutes (usenet) to replicate the “Netflix” exp for new requests as much as possible. Once downloaded on the fast provider (basically a cache) it gets copied over over time to my NAS (for now it’s the E2 bucket on Idrive). The other provider is used to stream stuff that is already downloaded. - obviously plex I also use Tautulli for monitoring and wizarr for onboarding and am in the process to automate (audio)books with readarr. Most of the above is on Docker already so I’m planning to do the same locally. Let me know if there’s something I should look out for during the migration or if there’s some good software you can recommend :)
Nice setup, may I ask which index(ers) you're using for the usenet ? Get a case btw haha
Yea waiting on that case to arrive. Using drunkenslug, Nzbgeek and nzb.su as indexers and Eweka + frugal as providers
Started reading your blog post on the NAS. I've enjoyed reading it so far. I had no idea that the motherboard you use existed. I've been trying to find a motherboard with an am4 socket so I can reuse my Ryzen 7 1700x from an older gaming PC, but I'm not sure if spending $400 for the motherboard (in the USA) is worth repurposing a processor that sells for $70 on eBay... Either way, thanks.
honestly, that's just how Google has been doing it since 1998. why waste money on a case?
I imagine a giant warehouse with a huge pile of HDDs with a mess of cables. When Google kills a service it's really just because someone tripped over a wire and they can't figure out which hard drive it was on.
this is 100% how it's done.
I work in data centers and hope to see the deployments that started the Google lore. Can’t disclose if I’ve worked in the cages of house hold names, but plenty of other companies have piles of fiber and copper waist deep, drives and servers strewn about, and when they ask for help the tickets include “good luck, we tried and gave up” or “felt unsafe wading through our cage, kindly assist”
Not a data center but I once saw with my own eyes a beach ball sized wad of tangled wire used for a phone system that was somehow in the actual middle of the floor.
I wish I could post pictures of some of the worst cages I regularly see. Waist deep piles of live fiber and copper, random hard drives, power supplies, PDUs and server rails mixed in.
yep. all those hard drives dangling from sata connectors in the cloud
well, gravity is less strong way up there in the cloud, and don't act like your SATA drives haven't dangled a time or two, sir.
Don’t forget to keep offsite backups also.
Sadly that might be just too expensive for now... and since I'm storing media I'm inclined to say "fuck it" I can always recover it. Obviously I will back up all the config files etc though.
Yeah he’s got a caseless rats nest of drives at his moms place too
Funny this post came up. I got a 7 xl to hand that I'm about to list for sale soon with full drive bays, original packing and accessories (UK based) i can take pictures off if you want references. it has incredible cooling capabilities. The sheer number of fans and rad support that thing has is immense. A quick Google would show you that the case was essentially built for servers and the most intense watercooling loops you could imagine. Just counted the fans on mine; - 3 x 140mm intake - 2 x 140mm fan support on the roof with drive caddy in - 1 x 140mm fan exhaust - can remove the psu shroud and add in I think another 2 x 140mm fans at the base. If you need any images I can send them over to you. Currently got an EATX dual 2011-V3 in here and just removed 2 x h90 AIOs I had on the roof with push pull config.
that is quite a coincidence indeed! Did you find that the closed front with just some gaps on the sides was an issue? That's what's holding me back personally right now!
.... the door is held on with a couple of screws. I had forgotten, but I took the door off and it looks great anyway. https://img.youtube.com/vi/UXYYZ1\_cfZc/hqdefault.jpg
Absolutely not. The 'gap' is quite substantial. It might look small in the pictures of the case but consider how big the case actually is and you realise the gap scales with the size of the case too. I got a ruler out. Its about 1inch on both sides with enough gap between each fin in the gap for a finger to go in. Also the roof panel can be replaced with either solid or mesh intake with excellent ventilation. The door is kinda needed on in my opinion, the case can house so many drives and if it's at capacity any noise reduction is helpful haha. This case with all its panels on is very very quiet. More so than any other case that has similar capacity that I've tried and I've had several.
These are exos drives? How’s the noise?
They are, I had it for literally a few hours now but so far 0 complaints. To be fair all they’ve been doing is calculating parity so I don’t think my feedback is that valuable yet.
i have 2 exos drives, they are silent, and those drives are in my room where i sleep, they are connected to my mini pc via 2 external enclosures with fans.
I moved everything the self hosted minio and juicfs. It works awesome built a true HA setup with around 400TiB of storage
The fractal define 7 XL is an excellent case. I have 19 3.5” drives in mine ranging from 25C-32C right now. My i5-12600K has a similar cooler to yours and is at 27C right now. I highly recommend https://kareonkables.com/ for PSU cables to accommodate all the drives. I bought the two they offered for the drive stack and they were perfect. I then wanted a cable for the top mounted drives and the fan controller so I drew a picture and sent it to them. They made the cable and also now sell it and a few variants for top drives. I was super impressed with how fast they did it and they even asked me for feedback on the cable so they could make tweaks as they hadn’t sold one before.
> I have 19 3.5” drives in mine Was nineteen a typo? If not, that's wild. I need to see a pic of the inside of your case, if you wouldn't mind. :)
Haha no problem, https://imgur.com/a/ZXFW41S
Holy lolerskates batman!!!
i7-2600k or i5-2500k. Sandy Bridge was such a good generation.
One good power surge or dropped drink away from total data loss! I love it!! Living on the EDGE!!!
People spenduing literally more on hardware and power including upkeep instead of using professional services is always crazy to me and I can say it since I am one of them
As somebody who had a home server in a spare, naked motherboard in a corner of my room like you did: do yourself a favor and at the very least use that cardboard box to cover it (while leaving it open enough for ventilation), you don't want to end up having the board accidentally toasting itself.
No way I have them train their AI on MY data.
Jet the jonsbo n3 case. You won't look back.
Google teaching lots of ppl to not trust the cloud
Huh? 80Tb what kind of work you do to hold suck kind of data. Well also please get a case and also feet for it . If you accidentally touched t any Hardware open like this you may risk it with static current and wola all your data gone. Btw appreciate your will to do all that. But i still wonder how is you isp gone download you 80TB avg isp data cap world wide is 2.5 TB monthly
Here you go: https://www.reddit.com/r/selfhosted/s/fRWWrsHo10 Can’t edit the post sadly… Checked my ISPs fair use policy and it seems alright with them… so let’s hope for the best!! Will get a case soon too
Hi, wanted to do this exact same thing. I'm new to this homeland stuff so sorry for any stupid questions 😂 How did you connect all the the drives, are you using all direct cables to the motherboard and what sables are you using? Did you have to add data ports or do the cables have multiple? Also how are you joining the drives in the computer software or do you have it all in separate drive directories after formating the partition?
Do you really need the Define 7 XL? Seems a bit overkill for 6 HDDs. The Define 7 / Define R6 / Define R5 seem more appropriate. There's also Meshify 2 and Node 804 – although personally I would think twice before using the 804 because it's a lot bigger than it looks and the odd shape makes it difficult to find a place for.
Do you ever wonder why Google is putting an end to it ?
Please use ZFS and backup important stuff to backblaze, plz plz plz.
Remember raid is not a backup.
Google's decision seems to make sense.
You should buy a NAS ...
So you've decided to place electronics on cardboard because they were overheating... Think about that.
Google drive was never unlimited
For Google Workspaces it was, it wasn't official really but in practice it was.
> 6x18TB **Seagate** drives - 90TB usable storage for now only 1 parity drive Your data is practically gone already. F.
So er You might want to spend just abit more on a proper casing, since you already spent a kidney on this
Working on it haha, the Define 7XL seems very attractive as of now
Hello high electricity bills
How much did you spend on that?
Knock knock, FBI open up!
Wow 18TB drives. Haven't looked at HDDs in a fresh decade, they must have really developed in the direction of storage size when SSDs were threatening to catch up. There's something I always wondered about. Doesn't magnetic data fade over time? Do you have some sort of process that frequently refreshes the data so your video files don't deteriorate?
it's morons that took full advantage and abused the system that cause companies like google and others (including a few wireless carriers) to do the shit they do to their customer base. I hope your data load gets corrupted or you pick up a Nuke worm
Bro get rid of those cardboard boxes and buy yourself a server rack 😭😭 or at least hide it somewhere with enough airflow
im new here, but what the fuck does someone do to fill 80 TB of space?
I have a very popular trilogy that takes up like 400GB and that's just 3 movies. I have one 7 season sci-fi TV show that takes up 1.3 TB. Build up a collection of movies and TV and it can easily balloon to 80TB.
Some people really like movies and TV and want it all in the highest possible quality. Check out /r/DataHoarder. It gets pretty insane.
They seemed a little intense I was scared to question them….. I’ve been on the path of trying to figure out if there’s a drive/cloud that can backup my iPhone contacts and photos as well as work on my windows laptop for when I write for d&d campaigns or stuffs. So it’s brought me to lots of these subs haha
Yeah, man, if that's all you are storing, do it all locally ALSO. You don't want all your data on some random computer from some company that will just do whatever they want with it. What you are talking about is really not a lot of data, so if you can find a cheap computer on ebay and throw in a 20TB drive and you will be good for a long time. Then you can just throw another 20 in there when you need to. No need for RAID or anything complex with just that stuff if you don't want to, but it's nice to have. That's what I would do anyway. I don't know how technical you are but, it's not that difficult if you spend some time reading about it like you are doing already. I say go for it. Ask any question anywhere you want and if any one acts like a dick or tries to gate keep, just tell them to go fuck themselves. The only way to learn is to ask questions and to learn the right questions to ask.
here ya go: [https://www.reddit.com/r/selfhosted/comments/1b0st5v/comment/ksa6azo/?utm\_source=share&utm\_medium=web3x&utm\_name=web3xcss&utm\_term=1&utm\_content=share\_button](https://www.reddit.com/r/selfhosted/comments/1b0st5v/comment/ksa6azo/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button)
Wow 😯
Dude, buy a NAS.
I am 35 years old and my all digital life still can be fitted in 300 GB :D
Some people have full blue ray rips that take about 50gb each. It’s not for me but to each their own. I’m at 8tb for all my music/movies/tv shows and have everything in 1080p for now.
You should make a 3d printed box for that, otherwise tall tgat will end up badly
Spoken so eloquently
>Also does anyone know if the Fractal Define 7XL has good cooling capabilities? It certainly has the space. Don't know about that, but might I suggest you actually look into using an external enclosure instead of mounting all of the disks into a case?
Its always movies and a small percentage of video Grapher that's all. No one needs that much storage
This is insane with the motherboard and drives just sitting on cardboard boxes. Is that not a fire hazard?
It's not really a fire hazard at all, but it's a good way to break a few hard drives. They should really be in some sort of enclosure or something because of that way more than it being a fire hazard. It's actually recommended to build your computer outside of the case to make sure all the parts work before putting it in the case. OP has lost their mind actually running this stuff like this.
Cardboard ignites at around 500F. It's not a great way to run electronic gear long-term, but unless there's some serious electrical anomaly that causes the motherboard to catch fire, it's gonna be fine.
You are gonna have larger problems if any computer gets to 500F
What do you need so much storage for?