Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
astral
Apr 26, 2004

Matt Zerella posted:

Friend. Don't do it. If you're using external drives just use them as JBOD.

My experience with trying to use a USB hard drive as 'intended' was that the Easystore enclosure tried to murder the hard drive inside with an overly aggressive attempt at power saving, completely ignoring the external drive hibernation settings of the Synology device it was connected to. If I recall correctly, the drive was starting and stopping a bit over twice per hour, which adds up rather quickly.

It's possible this was just a bug with the Synology, but it was enough to discourage me from trying that again for a long time.

Adbot
ADBOT LOVES YOU

Zorak of Michigan
Jun 10, 2006


Do any of these unRaid / SnapRAID / etc solutions let you import existing drives that have data on them? Taking a quick stroll through them, it looks like they're all designed so that you configure a new storage set and create new file systems, though you can then add new blank drives to grow capacity. That's part of what I meant when I told WWWWWWWWWWWWWWWWWW that he was going to have to put a lot of work into this- you can't just buy your sixteenth drive and make it hold parity data for the existing fifteen, you need to create a new file system using some newer technology and then start playing shell games with your data and drives. None of the steps involved are especially hard, but if you go into it with no plan, you can end up with a lot of confusion and frustration.

bobfather
Sep 20, 2001

I will analyze your nervous system for beer money

Zorak of Michigan posted:

Do any of these unRaid / SnapRAID / etc solutions let you import existing drives that have data on them? Taking a quick stroll through them, it looks like they're all designed so that you configure a new storage set and create new file systems, though you can then add new blank drives to grow capacity. That's part of what I meant when I told WWWWWWWWWWWWWWWWWW that he was going to have to put a lot of work into this- you can't just buy your sixteenth drive and make it hold parity data for the existing fifteen, you need to create a new file system using some newer technology and then start playing shell games with your data and drives. None of the steps involved are especially hard, but if you go into it with no plan, you can end up with a lot of confusion and frustration.

Stablebit Drivepool will let you import data. In fact, there’s an advanced method to create your pool and manually move the files from each drive into the hidden folder that Drivepool uses, so duplication can begin immediately.

Drivepool is amazing.

Matt Zerella
Oct 7, 2002

Norris'es are back baby. It's good again. Awoouu (fox Howl)
I am 99% sure you can add a second parity drive to an existing array in the webui without needing to do anything.

BlankSystemDaemon
Mar 13, 2009



eames posted:

then please be so kind and fix the wikipedia article :)
Yeah, no, I just wrote a dumb thing that's untrue.
UnRAID is beyond quantification. :colbert:

Smashing Link
Jul 8, 2003

I'll keep chucking bombs at you til you fall off that ledge!
Grimey Drawer
Improve my backup strategy:

(home) Synology 1515+ / 5x 6TB WD Red Pro / SHR2 / ~ 5TB used
- backs up to (work) Synology 1515+ / 5x 3TB WD Red Pro / SHR2 / via HyperBackup using rsync, connected by OpenVPN served by the home NAS
- backs up to 2x 8TB WD EasyStore (non-shucked) by HyperBackup that are alternately kept at home and office
(work) Synology backs up to G-suite Drive via HyperBackup (encrypted)

Family photos and critical documents are also backed up on a couple of other random HDDs and USB disks on an occasional basis.

BlankSystemDaemon
Mar 13, 2009



Smashing Link posted:

Improve my backup strategy:

(home) Synology 1515+ / 5x 6TB WD Red Pro / SHR2 / ~ 5TB used
- backs up to (work) Synology 1515+ / 5x 3TB WD Red Pro / SHR2 / via HyperBackup using rsync, connected by OpenVPN served by the home NAS
- backs up to 2x 8TB WD EasyStore (non-shucked) by HyperBackup that are alternately kept at home and office
(work) Synology backs up to G-suite Drive via HyperBackup (encrypted)

Family photos and critical documents are also backed up on a couple of other random HDDs and USB disks on an occasional basis.
Shot of using something that does checksumming, that's basically as good as is possible. The biggest thing I can think of is adding UPS' so that you can't get RAID write holes.
So, ZFS it is! :haw:

eames
May 9, 2009

Smashing Link posted:

Improve my backup strategy:

(home) Synology 1515+ / 5x 6TB WD Red Pro / SHR2 / ~ 5TB used
- backs up to (work) Synology 1515+ / 5x 3TB WD Red Pro / SHR2 / via HyperBackup using rsync, connected by OpenVPN served by the home NAS
- backs up to 2x 8TB WD EasyStore (non-shucked) by HyperBackup that are alternately kept at home and office
(work) Synology backs up to G-suite Drive via HyperBackup (encrypted)

Family photos and critical documents are also backed up on a couple of other random HDDs and USB disks on an occasional basis.

seems very hard to improve upon! 1515+ are prone to sudden death (due to the atom bug, usually starts with uncontrolled reboots) and I'd maybe do one of the backups with something other than HyperBackup so you don't lose your stuff in case that particular program decides to write garbage or deletes all previous backups because of a bug.
BTRFS does checksumming and seems to work fine with SHR2 (unlike the BTRFS RAID5/6 implementations).

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

necrobobsledder posted:

Services like Backblaze B2 or perhaps Tarsnap are much cheaper than the usual cloud service providers like Google, Amazon, Microsoft, or even Apple because when you retrieve data back outside their cloud you get charged a metric ton.

Atomizer
Jun 24, 2007



D. Ebdrup posted:

You said this before but last time you also said it's for uptime so I didn't crawl down your throat about it.
RAID is only about redundancy when you're using nested levels of mirrors or striping with distributed parity. If there's any device striping without distributed parity, you're doing it for availability and speed.
Put another way: RAID 11, 15, 16, 17, 51, 55, 61, 65, 66, 71, 75, 76, or 77 provides you with redundancy, everything else is just for availability and/or speed.

I agree, aside from "redundant" being literally the first word in the initialism of RAID, I mischaracterized the "redundancy" component because there's no direct duplication (or more) of data, but technically any extra drives (i.e. ones that can fail without immediately compromising the array) are literally redundant, but we shouldn't argue semantics here. I think it's pretty clear I was emphasizing real backup solutions above all else and discouraging RAID (which I don't even use myself, at all) but trying to explain these topics to individuals without a proper foundation is not that simple.

Moey
Oct 22, 2010

I LIKE TO MOVE IT
Pull up, thread.

Schadenboner
Aug 15, 2011

by Shine

Moey posted:

Pull up, thread.

Nah, gently caress it, we'll just pull backups and restore.

Wait, the "B" in RAID stands for "Backup", right?

Smashing Link
Jul 8, 2003

I'll keep chucking bombs at you til you fall off that ledge!
Grimey Drawer

D. Ebdrup posted:

Shot of using something that does checksumming, that's basically as good as is possible. The biggest thing I can think of is adding UPS' so that you can't get RAID write holes.
So, ZFS it is! :haw:


eames posted:

seems very hard to improve upon! 1515+ are prone to sudden death (due to the atom bug, usually starts with uncontrolled reboots) and I'd maybe do one of the backups with something other than HyperBackup so you don't lose your stuff in case that particular program decides to write garbage or deletes all previous backups because of a bug.
BTRFS does checksumming and seems to work fine with SHR2 (unlike the BTRFS RAID5/6 implementations).

Thanks for the feedback -- I learned a lot from this thread!

Forgot to add that I do have a UPS. And although I am regretting that I bought two 1515+ I don't need to be tempted to buy anymore hardware at the moment. If one dies I will happily pull the trigger on an 1819+ or perhaps even roll my own.

Rexxed
May 1, 2010

Dis is amazing!
I gotta try dis!

Hey guys my raid array of independent disks is pretty redundant.

Rooted Vegetable
Jun 1, 2002

Schadenboner posted:

Nah, gently caress it, we'll just pull backups and restore.

The backup drive is part of our RAID array. Why would we even worry about including a tape drive in a striped array alongside our SSDs?!

sharkytm
Oct 9, 2003

Ba

By

Sharkytm doot doo do doot do doo


Fallen Rib

Heners_UK posted:

The backup drive is part of our RAID array. Why would we even worry about including a tape drive in a striped array alongside our SSDs?!

Where can I find a floppy interface expander? I've got an idea...

BlankSystemDaemon
Mar 13, 2009



Just stripe ZFS on top of 45 1.44MB floppies for the minimum 64MB required for a zpool.

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast

Atomizer posted:

I agree, aside from "redundant" being literally the first word in the initialism of RAID, I mischaracterized the "redundancy" component because there's no direct duplication (or more) of data

Except in RAID1, which is literally that

BlankSystemDaemon
Mar 13, 2009



The R in RAID is short for redundant because when RAID was made at Berkeley in California, it used simple mirroring for RAID1 or striping with parity for RAID2, with either Hamming or Reed-Solomon code which was meant to be done in hardware.

Schadenboner
Aug 15, 2011

by Shine

D. Ebdrup posted:

The R in RAID is short for redundant because when RAID was made at Berkeley in California, it used simple mirroring for RAID1 or striping with parity for RAID2, with either Hamming or Reed-Solomon code which was meant to be done in hardware.

P. sure it stands for "Backup" (because there is no R in backup and if you use RAID you will not have to worry about backups anymore because they are they best backup solution available)?

BlankSystemDaemon
Mar 13, 2009



Schadenboner posted:

P. sure it stands for "Backup" (because there is no R in backup and if you use RAID you will not have to worry about backups anymore because they are they best backup solution available)?
Eh, close enough definition for government work.

greazeball
Feb 4, 2003



I want to upgrade my Plex server but I don't know what to look for or how much it should cost. Right now I've got a WD MyCloudMirror with 3.5/4TB on it. Video files are getting bigger so I'm going to need to do a big clean out soon (nah) or buy something new. I want something bigger and faster that won't throw up so many the server is not powerful enough errors. It would be great if it could also run Subsonic (or another app that's similar for playing a big mp3 library). I'd like to spend about $1000.

I'd then send the MyCloud to my mother-in-law's house and just use that for backing up photos and important docs.

Matt Zerella
Oct 7, 2002

Norris'es are back baby. It's good again. Awoouu (fox Howl)
You honestly don't need to spend that much. Have an old PC laying around?

greazeball
Feb 4, 2003



I have a Dell something or other that's about 3 years old and I've been looking to upgrade that. The processor is not very powerful though.

mystes
May 31, 2006

greazeball posted:

I have a Dell something or other that's about 3 years old and I've been looking to upgrade that. The processor is not very powerful though.
Most of the NAS boxes have really slow arm processors. Your dell is going to be way faster than that.

Rooted Vegetable
Jun 1, 2002

greazeball posted:

I have a Dell something or other that's about 3 years old and I've been looking to upgrade that. The processor is not very powerful though.

Almost anything 3 years old will be fine.

Moey
Oct 22, 2010

I LIKE TO MOVE IT
Give us more details on your Plex usage.

Do you just stream inside your home or do you stream remotely as well?
What clients are going to be used to consume media?
Do you share with friends/family remotely?
How many concurrent users?
What resolution media do you hoard?

Beaucoup Haram
Jun 18, 2005

I've got a dual 2680v2 with 128gb of ECC ram running my plex server but please don't do this to yourself.

H110Hawk
Dec 28, 2006

Beaucoup Haram posted:

I've got a dual 2680v2 with 128gb of ECC ram running my plex server but please don't do this to yourself.

Your local electric utility thanks you.

Atomizer
Jun 24, 2007



HalloKitty posted:

Except in RAID1, which is literally that

D. Ebdrup posted:

The R in RAID is short for redundant because when RAID was made at Berkeley in California, it used simple mirroring for RAID1 or striping with parity for RAID2, with either Hamming or Reed-Solomon code which was meant to be done in hardware.

FYI to everyone who responded: the point is that "RAID" is complicated (and then you throw RAID0 in there) and isn't appropriate for the average user. Remember that this all came about in response to a guy who: dropped and broke an HDD without having any backups, begged for help in recovering data, then asked if RAID was backup for his media collection; he ended up opting not to implement any proper backup solution after all this. (And I'm not trying to criticize him by recapping this, just pointing out why this all came up.) The "proper" answer to all this is not to get into what RAID is and describe all the configurations: it's to insist that a) everyone needs to backup their important data, and b) you do not need RAID for your media library.

greazeball posted:

I want to upgrade my Plex server but I don't know what to look for or how much it should cost. Right now I've got a WD MyCloudMirror with 3.5/4TB on it. Video files are getting bigger so I'm going to need to do a big clean out soon (nah) or buy something new. I want something bigger and faster that won't throw up so many the server is not powerful enough errors. It would be great if it could also run Subsonic (or another app that's similar for playing a big mp3 library). I'd like to spend about $1000.

I'd then send the MyCloud to my mother-in-law's house and just use that for backing up photos and important docs.

greazeball posted:

I have a Dell something or other that's about 3 years old and I've been looking to upgrade that. The processor is not very powerful though.

You should post in the Plex thread. We're still going to ask what the hardware is on your Dell, though. And for reference, both of the PCs I've used for PMS are over 3 years old and work perfectly; you can almost certainly use that for PMS, Subsonic, etc. (I use my current PMS system for a Steam caching server and other stuff.)

Moey
Oct 22, 2010

I LIKE TO MOVE IT

Beaucoup Haram posted:

I've got a dual 2680v2 with 128gb of ECC ram running my plex server but please don't do this to yourself.

Hopefully it's doing something else besides Plex, because that's just wasted RAM.

IOwnCalculus
Apr 2, 2003





H110Hawk posted:

Your local electric utility thanks you.

I am so very glad that I don't have to pay for the energy mine uses.

greazeball
Feb 4, 2003



Moey posted:

Give us more details on your Plex usage.

Do you just stream inside your home or do you stream remotely as well?
What clients are going to be used to consume media?
Do you share with friends/family remotely?
How many concurrent users?
What resolution media do you hoard?


Atomizer posted:

You should post in the Plex thread. We're still going to ask what the hardware is on your Dell, though. And for reference, both of the PCs I've used for PMS are over 3 years old and work perfectly; you can almost certainly use that for PMS, Subsonic, etc. (I use my current PMS system for a Steam caching server and other stuff.)


Thanks, I posted my replies in the Plex thread

Beaucoup Haram
Jun 18, 2005

Moey posted:

Hopefully it's doing something else besides Plex, because that's just wasted RAM.

Yeah it's a virt lab also and does a bunch of stuff but it's still ridiculous overkill.

Raldikuk
Apr 7, 2006

I'm bad with money and I want that meatball!

Zorak of Michigan posted:

Do any of these unRaid / SnapRAID / etc solutions let you import existing drives that have data on them? Taking a quick stroll through them, it looks like they're all designed so that you configure a new storage set and create new file systems, though you can then add new blank drives to grow capacity. That's part of what I meant when I told WWWWWWWWWWWWWWWWWW that he was going to have to put a lot of work into this- you can't just buy your sixteenth drive and make it hold parity data for the existing fifteen, you need to create a new file system using some newer technology and then start playing shell games with your data and drives. None of the steps involved are especially hard, but if you go into it with no plan, you can end up with a lot of confusion and frustration.

Snapraid lets you use existing drives since it doesn't touch the file system config itself. You can also remove drives at will. So you just need to add the parity drives and have it compute that. You can expand it at will too...just need to recalculate for your given parity level. It allows you to go up or down in parity level too without losing any data (other than the parity calc...which itself isn't lost until a successful rebuild).

Moey
Oct 22, 2010

I LIKE TO MOVE IT
RAM for my frankenstein Gen8 Microserver came in. So far so good.

I'll post some pictures when I finish up with it, but the plan is 4x3.5" and up to 4x2.5" internally.

Moey fucked around with this message at 18:55 on Mar 19, 2019

Atomizer
Jun 24, 2007



Not related to anything else, I broke down several old (~15 years), obsolete laptops (mainly removing & separating the batteries for disposal and the HDDs for wiping user data) earlier today. Despite their age, I was pleasantly surprised that 3 of the 8 HDDs were SATA (I thought they'd all be PATA/IDE.) Going back and dealing with PATA is a royal pain in the rear end, but it makes me appreciate SATA that much more (from the ease-of-use alone.) The ~15-year-old 60 GB 2.5" HDD I pulled was identical to work with to the 2 TB FireCuda that I also got delivered today. No different connectors, no jumpers, no USB adapters not passing SMART diagnostics to the host, none of that bullshit to deal with. :allears:

BlankSystemDaemon
Mar 13, 2009



Atomizer posted:

Not related to anything else, I broke down several old (~15 years), obsolete laptops (mainly removing & separating the batteries for disposal and the HDDs for wiping user data) earlier today. Despite their age, I was pleasantly surprised that 3 of the 8 HDDs were SATA (I thought they'd all be PATA/IDE.) Going back and dealing with PATA is a royal pain in the rear end, but it makes me appreciate SATA that much more (from the ease-of-use alone.) The ~15-year-old 60 GB 2.5" HDD I pulled was identical to work with to the 2 TB FireCuda that I also got delivered today. No different connectors, no jumpers, no USB adapters not passing SMART diagnostics to the host, none of that bullshit to deal with. :allears:
USB PATA/SATA adapters masking S.M.A.R.T data is one of the weirdest things I've had to deal with with computers, because sometimes a device that doesn't work on Windows works on FreeBSD, and other times it's the other way around.

Atomizer
Jun 24, 2007



D. Ebdrup posted:

USB PATA/SATA adapters masking S.M.A.R.T data is one of the weirdest things I've had to deal with with computers, because sometimes a device that doesn't work on Windows works on FreeBSD, and other times it's the other way around.

Oh, I didn't even know it had anything to do with the OS; I just figured that the janky, kinda-works PATA-USB3 cable I got had such a cheap controller that it didn't even pass SMART data. (I've a couple of USB2 enclosures that work just fine, but bottleneck anything >100 GB in capacity.) And that's pretty significant: of the 3 SATA drives I pulled, 2 were healthy but one had a ton of reallocated sectors and other warning flags thrown. I have to wait to peek into the PATA drives because I don't have a 40-44-pin cable and the janky USB3 thing I mentioned above only intermittently works.

I wrote about this a few months back, I think in another thread, that as an alternative to PATA-USB3 (to achieve better performance than USB2) I tried to use eSATA, but was never able to get it to work at all; I couldn't get a drive recognized, period, regardless of combination of: [apparently-properly-configured] host PC, internal drive, enclosure/adapter. I even tried a USB3-eSATA cable, and a SATA-eSATA enclosure (for the hell of it) with no luck. Anyone have tips for getting eSATA to work?

Adbot
ADBOT LOVES YOU

BlankSystemDaemon
Mar 13, 2009



Atomizer posted:

Oh, I didn't even know it had anything to do with the OS; I just figured that the janky, kinda-works PATA-USB3 cable I got had such a cheap controller that it didn't even pass SMART data. (I've a couple of USB2 enclosures that work just fine, but bottleneck anything >100 GB in capacity.) And that's pretty significant: of the 3 SATA drives I pulled, 2 were healthy but one had a ton of reallocated sectors and other warning flags thrown. I have to wait to peek into the PATA drives because I don't have a 40-44-pin cable and the janky USB3 thing I mentioned above only intermittently works.

I wrote about this a few months back, I think in another thread, that as an alternative to PATA-USB3 (to achieve better performance than USB2) I tried to use eSATA, but was never able to get it to work at all; I couldn't get a drive recognized, period, regardless of combination of: [apparently-properly-configured] host PC, internal drive, enclosure/adapter. I even tried a USB3-eSATA cable, and a SATA-eSATA enclosure (for the hell of it) with no luck. Anyone have tips for getting eSATA to work?
I suspect the core of it to be controller issues caused by the OEMs asking the ODMs to build down to a price rather than conforming to spec, because I have one USB3 to PATA/SATA adapter which works with S.M.A.R.T consistently across all OS' I've tried it on; it uses a NEC chip (before RENESAS took over their production line).
Unfortunately it's also a completely no-name/un-branded device which I have no clue where I bought, so I can't buy another one or recommend them.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply