Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
pzy
Feb 20, 2004

Da Boom!
:siren: as if Best Buy was listening :siren:

8 TB for $129: https://www.bestbuy.com/site/wd-easystore-8tb-external-usb-3-0-hard-drive-black/5792401.p?skuId=5792401
10 TB for $159: https://www.bestbuy.com/site/wd-easystore-10tb-external-usb-3-0-hard-drive-black/6278208.p?skuId=6278208

Adbot
ADBOT LOVES YOU

Roundboy
Oct 21, 2008
Sniping in with a stupid question

How do these compare to WD RED drives I usually buy? These would be roughly $100 less then the same thing, and I need to update my storage solution

dexefiend
Apr 25, 2003

THE GOGGLES DO NOTHING!
Where do you even find 230TB of Hentai?

Edit: with that much you probably want dual parity, sooo 220TB of Hentai?

THF13
Sep 26, 2007

Keep an adversary in the dark about what you're capable of, and he has to assume the worst.

Roundboy posted:

Sniping in with a stupid question

How do these compare to WD RED drives I usually buy? These would be roughly $100 less then the same thing, and I need to update my storage solution

The 8 and 10TB are going to be either actual WD Red or whites. The Whites are functionally rebranded Reds but you may need to tape over/pull the wire/use molex to deal with the 3.3v issue.

Shaocaholica
Oct 29, 2002

Fig. 5E
White drives don't use standard SATA power in?

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Roundboy posted:

Sniping in with a stupid question

How do these compare to WD RED drives I usually buy? These would be roughly $100 less then the same thing, and I need to update my storage solution

As said, they're literally the same drive. The only catch is that if you need to warranty return one, you may need to also send back the shell, meaning you gotta keep a shell or two hanging out in a closet somewhere. But that's it, otherwise they're just a poo poo ton cheaper for reasons no one has yet to be able to figure out.

Actuarial Fables
Jul 29, 2014

Taco Defender

Shaocaholica posted:

White drives don't use standard SATA power in?

They do use the standard SATA power - the caveat is that in SATA rev. 3.3, which the aforementioned drives use, the 3.3v line is repurposed to trigger the Power Disable feature. The 5v and 12v lines are the same as they ever were.

Very few drives actually ever used the 3.3v line and a lot of power supplies don't even have a 3.3v cable for the SATA power connectors.

Actuarial Fables fucked around with this message at 18:28 on Oct 25, 2019

Duck and Cover
Apr 6, 2007

How paranoid is using 2 bays for backup in a 5 bay system? While the files aren't irreplaceable I am lazy. I also don't like deleting things (that I'll never open again) and that's an extra 8tb.


Thanks. Why yes I do have two unused bays.

Duck and Cover fucked around with this message at 18:23 on Oct 25, 2019

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Duck and Cover posted:

How paranoid is using 2 bays for backup in a 5 bay system? While the files aren't irreplaceable I am lazy. I also don't like deleting things (that I'll never open again) and that's an extra 8tb.
Thanks. Why yes I do have two unused bays.

9/10 paranoia, 2-drive redundancy is usually a 8+ drive thing.

How about 1-drive redundancy and you open a Backblaze account for the 1tb of stuff that really needs to be backed up? 2-drive redundancy on a 5 drive array doesn't really protect against the problems that really matter.

Paul MaudDib fucked around with this message at 18:50 on Oct 25, 2019

Duck and Cover
Apr 6, 2007

Paul MaudDib posted:

9/10 paranoia, 2-drive redundancy is usually a 8+ drive thing.

How about 1-drive redundancy and you open a Backblaze account for the 1tb of stuff that really needs to be backed up?

I guess I undersold how unimportant/replaceable the files are. It's anime, terabytes of anime (well actually no not even a terabyte), well and other TV/ Movies.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Duck and Cover posted:

I guess I undersold how unimportant/replaceable the files are. It's anime, terabytes of anime (well actually no not even a terabyte), well and other TV/ Movies.

like I said, how about you run 1 drive redundancy and back up the 1tb of poo poo that is actually irreplaceable

(not all torrents are easily replaceable, there is stuff I've had to strugglebus for months to pull down and that stuff gets backed up at a higher redundancy)

Chris Knight
Jun 5, 2002

me @ ur posts


Fun Shoe

ILikeVoltron posted:

X10DRD-iNTP vs X10DRD-iNT
Doesn't matter because Meyers-Briggs is bullshit anyway.

Atomizer
Jun 24, 2007



Roundboy posted:

Sniping in with a stupid question

How do these compare to WD RED drives I usually buy? These would be roughly $100 less then the same thing, and I need to update my storage solution

"White-label" drives are basically relabeled retail drives with less warranty. In this case (with the external drives) they're often Red-equivalent, but can be of a variety of individual drives (e.g. with the 8 TB drives there are at least 4 different possible drives, with varying buffer capacity.) The thing to keep in mind is that the drive inside can vary; you're not guaranteed of getting anything that isn't explicitly advertised.

Duck and Cover posted:

How paranoid is using 2 bays for backup in a 5 bay system? While the files aren't irreplaceable I am lazy. I also don't like deleting things (that I'll never open again) and that's an extra 8tb.

Duck and Cover posted:

I guess I undersold how unimportant/replaceable the files are. It's anime, terabytes of anime (well actually no not even a terabyte), well and other TV/ Movies.

You don't need online backups; you can put your 1 TB of anime on a 1 TB HDD that you keep disconnected, in a safe location. Your "2 of 5 drives for backup" is either referring to RAID, which is not backup, or at best unnecessary active drives with duplicate copies of your data.

Duck and Cover
Apr 6, 2007

See this is why when asking questions I should keep it concise because people start talking about stuff that isn't just answering the question. Then I look ungrateful and rude when I'm like YES YES I KNOW I CAN BACKUP poo poo IN OTHER WAYS. I suppose a better way to phrase the question would be "how likely am I to have multiple hardrives fail before before I'm able to purchase and repopulate a replacement to one in a five harddrive setup".

Whatever I had a feeling I was being unnecessarily paranoid.

Atomizer posted:

"Your "2 of 5 drives for backup" is either referring to RAID, which is not backup, or at best unnecessary active drives with duplicate copies of your data.

Well actually it's Frankenstein's monster. 1. It doesn't matter. 2. I don't care. 3. I know. 4. Maybe learn how language works and try understanding the context instead of smugly trying to inform some one of what is technically correct. Language isn't rigid.

Duck and Cover fucked around with this message at 03:25 on Oct 26, 2019

Axe-man
Apr 16, 2005

The product of hundreds of hours of scientific investigation and research.

The perfect meatball.
Clapping Larry
Or. You can take that paranoia and max it to 12/10.

You can setup another NAS offsite that replicates the current storage, then use a NAS on site with a full backup copy connected to your current NAS, and then, to top Everything off, use another nas to back up your offsite NAS, located on your offsite, all running RAID 6.

This is my setup for my 4 terabytes total of computer backups and movies and random stuff. :c00lbert:

BobHoward
Feb 13, 2012

The only thing white people deserve is a bullet to their empty skull

Duck and Cover posted:

Well actually it's Frankenstein's monster. 1. It doesn't matter. 2. I don't care. 3. I know. 4. Maybe learn how language works and try understanding the context instead of smugly trying to inform some one of what is technically correct. Language isn't rigid.

My dude you are massively overreacting, I don't think you read Atomizer's tone correctly at all

Crunchy Black
Oct 24, 2017

by Athanatos
While we *do* tend to trend towards being data nerds in this thread, as it seems most of us have some amount of industry experience, and its fun to get into the weeds, Atomizer actually answered your question, realistically, in the most direct and succinct way possible lol

BlankSystemDaemon
Mar 13, 2009



Another way to look at it is that the number of drives used for distributed parity in RAID5/6/7 is a function of how big the array is for a given probability of encountering an URE.
With harddisk sizes growing as much as they have, and bandwidth having stayed more or less the same, it's getting to the point where traditional RAID levels cannot reliably resilver the array before encountering an URE during the resilver, which will mean dataloss and more importantly loss of availability (which is what RAID is REALLY used for, in RAS systems like mainframes where it comes from).

Terabytes worth of your Japanese cartoons is right on the edge of the territory where if it was business data I'd recommend a company to switch some higher level of redundancy, but since it's probably not business critical, you might be better off taking the advice of Atomizer and keeping a backup at your parents' place or somewhere else trustworthy since you might not need to access it immediately after your array has gone down.

Axe-man
Apr 16, 2005

The product of hundreds of hours of scientific investigation and research.

The perfect meatball.
Clapping Larry
If you want to check for yourself that probability you can use a calculator and just take the information off your manufacture spec list how much error percent you have for that disk, and pop it in there

http://www.raid-failure.com/raid5-failure.aspx

Though as it points out in a link the assumptions using that formula are rather naive, (like that one bit error will kill the whole array for example)

But yeah, at work, I deal with businesses that lost everything sometimes doctor offices and lawyers cause they figured RAID was a backup. It is not. It is the ability to keep your data accessible with a set number of drives having failed.

The only true way to do it is to use backups, and have a copy of the data somewhere else that is up to date and had its integrity checked, or at least tested.

I'm not in data recovery, but from what I have heard, you get to pay them thousands of dollars for them to say there is nothing to little they can do for you.

:shrug:

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
Even if the data is not the most business critical or anything having to rebuild your data kind of sucks and I'm paying for the conveniences of not having to do that again with offsite backup systems and even going down a gigabit pipe it sucks to have none of your data for days. Cloud storage as your primary makes sense for the majority of people and businesses IMO but you know when you know what you need.

Roundboy
Oct 21, 2008
picking up a couple WD external 10tb drives today, i am so very low on server space.

Next up is seeing if i have enough sata cables and figuring out how to shuffle my data from 2 separate 2tb drives

Actually, question on that. My os drive is ssd, and my storage is primarily 2x 2tb dives. Each one is mounted to a different mount point w/ physicalpartitions , TV and Movies

I feel I should install the 2x 10TB drives as a single 20tb combined logical partition, with a new mount point and created directories. I can then move data over, and then delete the existing partitions, and add the 2x 2tb drives to the new logical partition giving me one big 24TB drive.

Sure I am not RAID'ed but its stuff i download anyway, so it only costs me time to rebuild. I guess the hard part would be figuring out which drive(s) are bad. Any real downside to storing data this way for a plex server ?


edit: confused logical and physical partitions

Roundboy fucked around with this message at 16:29 on Oct 26, 2019

Smashing Link
Jul 8, 2003

I'll keep chucking bombs at you til you fall off that ledge!
Grimey Drawer

pzy posted:

Limit 2 per customer is kind of a bummer. I just built an unraid server with 24x 10TB Easystores at $180 each, I wish they'd hit $160 again, maybe with a free dumb thumbdrive! Though Best Buy just threw them back to $249 each, with the 12TB ones at $279.

That's...a lot of storage. May I ask what your use case is? I'm building a 24TB Unraid and thinking it's overkill for my 8TB of media, though I want room to grow.

Smashing Link
Jul 8, 2003

I'll keep chucking bombs at you til you fall off that ledge!
Grimey Drawer

THF13 posted:

The 8 and 10TB are going to be either actual WD Red or whites. The Whites are functionally rebranded Reds but you may need to tape over/pull the wire/use molex to deal with the 3.3v issue.

I'm thinking I read somewhere that newer PSUs don't require this fix, or do I have it backward?

THF13
Sep 26, 2007

Keep an adversary in the dark about what you're capable of, and he has to assume the worst.

Smashing Link posted:

I'm thinking I read somewhere that newer PSUs don't require this fix, or do I have it backward?

Yes but not universally, so it's very much a your mileage may vary situation.

One discord I'm in recommends getting a SATA power cable splitter like these and just pulling out the 3.3v wire altogether (It's the one closest to the 'L'). Easy to do and only tool you need is a knife/credit card/guitar pick to wedge out the back of each sata connector. More reliable than tape, don't have to mess with your drive or power supply cables. Normal drives don't need this wire either so you can hook up whatever you want to the modified splitter.

If you want to go a step further you can also adjust the length of the modded cable to match your hard drives perfectly.

Axe-man
Apr 16, 2005

The product of hundreds of hours of scientific investigation and research.

The perfect meatball.
Clapping Larry

necrobobsledder posted:

Even if the data is not the most business critical or anything having to rebuild your data kind of sucks and I'm paying for the conveniences of not having to do that again with offsite backup systems and even going down a gigabit pipe it sucks to have none of your data for days. Cloud storage as your primary makes sense for the majority of people and businesses IMO but you know when you know what you need.

For a business, i think that most cloud sites are great as a last resort. Having an onsite backup, or even an high availability switch over system is more in line with what to do when you have that urgent need to keep things going.

I would say having at the very least onsite, replication, and up to minute backup for a business would be reasonable, while having offsite, data recovery compressed backups, would make more sense.

For home use, I honestly recommend that people have at least 2 NAS, and a cloud storage. 1 for them, that backup computers, and important stuff. 1 for a family member with it set on their computers as well. Especially if that person is bad with computers.

My parents were hit by a crypto scam a few years ago and had to pay a few hundred dollars to recover the data (apparently it was on a unencrypted usb drive, but they didnt know how to recover from it.)

Now, they have a NAS to backup their stuff, and have the computers do a daily update, and I have a backup destination target to do my versioned compressed backup.

As we get hackers and scammers attempting to push what have been mostly business related issues more and more on the general populous, like for example the most recent crypto attacks attempting to blackmail bitcoiners for their porn habits, having some sort of solution will be necessary. I think as time goes on it will only increase. And having to pull everything directly off a cloud... well, I've seen a few companies take weeks to do that. They have enterprise grade connections too, just that much data.

https://blog.malwarebytes.com/cybercrime/2019/02/sextortion-bitcoin-scam-makes-unwelcome-return/

Seeing that most bitcoiners tend to defend illegal porn, why on earth would that work? :thunk:

Smashing Link
Jul 8, 2003

I'll keep chucking bombs at you til you fall off that ledge!
Grimey Drawer

THF13 posted:

Yes but not universally, so it's very much a your mileage may vary situation.

One discord I'm in recommends getting a SATA power cable splitter like these and just pulling out the 3.3v wire altogether (It's the one closest to the 'L'). Easy to do and only tool you need is a knife/credit card/guitar pick to wedge out the back of each sata connector. More reliable than tape, don't have to mess with your drive or power supply cables. Normal drives don't need this wire either so you can hook up whatever you want to the modified splitter.

If you want to go a step further you can also adjust the length of the modded cable to match your hard drives perfectly.


That is useful information, thank you.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Duck and Cover posted:

Well actually it's Frankenstein's monster. 1. It doesn't matter. 2. I don't care. 3. I know. 4. Maybe learn how language works and try understanding the context instead of smugly trying to inform some one of what is technically correct. Language isn't rigid.

wow the irony

Duck and Cover
Apr 6, 2007

BobHoward posted:

My dude you are massively overreacting, I don't think you read Atomizer's tone correctly at all

This isn't really an example or one person doing something worse then others have done, more me getting sick of reading it yet again. I certainly could have ignored it I choose not to, in an attempt to maybe change future behavior of not just Atomizer, but others.

Crunchy Black posted:

While we *do* tend to trend towards being data nerds in this thread, as it seems most of us have some amount of industry experience, and its fun to get into the weeds, Atomizer actually answered your question, realistically, in the most direct and succinct way possible lol

No he didn't. The only one who did was PaulmcDib oh and Axe-Man calculator post. People assumed what I was asking and tried to answer that, which happens partly because I give it the chance to. I know my options I was just looking to weigh them properly not have options listed for me/people giving their opinion on what's best.

Whatever I've decided that the risk is worth the extra space. Well until I change my mind, the harddrive is still getting it's sector check. 8tb is probably enough extra though... hmmmm.

Roundboy
Oct 21, 2008
bought 2 ezstores, first one is a White label WD100EMAZ, not sure if i have rto deal with 3.3v pin yet.

debating opening the other one up or just returning and waiting a month for a possible black friday deal on these or actual red drives. or just pick up a couple more and kit out all my storage needs

Hughlander
May 11, 2005

Went to upgrade my NAS from proxmox 5.4 yesterday to 6.0 and something went wrong where zfs was segfaulting on startup. So just wiped the root drive and reinstalled. I've been pretty diligent that the only things installed on the host are things that deal with zfs or backups (sanoid, duplicacy, rclone) so I think recovering will be pretty easliy. Though restoring /etc/pve didn't let the lxc containers start. I'm pulling /var/lib from a backup to see if it's /var/lib/lxc missing that's the problem.

I already have the 3 zpools mounted and validated that it's all fine and i've rebooted a few times with 6 w/o the segfault issue. No idea what was stopping my boot originally :iiam:

Rexxed
May 1, 2010

Dis is amazing!
I gotta try dis!

Roundboy posted:

bought 2 ezstores, first one is a White label WD100EMAZ, not sure if i have rto deal with 3.3v pin yet.

debating opening the other one up or just returning and waiting a month for a possible black friday deal on these or actual red drives. or just pick up a couple more and kit out all my storage needs

I'd just use them; I could be wrong but I doubt the prices will get so much lower as to be worth the hassle of packing them back up and returning them. The 3.3v pin thing is a super minor issue, it adds one small step to installing the disks but it took me about a minute to cut a little piece of kapton tape and cover the pins. If I was removing the drives and reinserting them frequently I'd be a little concerned about the longevity but mine are working great as-is. I've got three 8TB easystores and a couple of white label WD 8TB I got from ebay sellers.

Roundboy
Oct 21, 2008
was going to return, decided to keep them. I only have 4 sata ports on this mobo and i am topped out. <ight be worth grabbing a new micro atx board for more sata 3 connections.

The rack i have it all in supports 6 drives anyway, and i just added the power cables and tape to my amazon cart, I guss i can upgrade all at once.

Roundboy fucked around with this message at 00:59 on Oct 27, 2019

Gyrotica
Nov 26, 2012

Grafted to machines your builders did not understand.
Is there an easy way to set up regular data integrity verification and repair in Windows 10 while staying on NTFS? I’ve got 8tb of media that isn’t going to do anything but grow. It seems like Multipar is best for single files, not whole disk collections.

BlankSystemDaemon
Mar 13, 2009



Proper filesystem integrity requires checksumming of both the metadata (not uncommon, but not a feature in NTFS as far as I recall), as well as the data.
It's an even better to do it in some tree-like structure that lets you verify everything back to a single root block (which preferably is written many places to the disk).

Gyrotica
Nov 26, 2012

Grafted to machines your builders did not understand.

D. Ebdrup posted:

Proper filesystem integrity requires checksumming of both the metadata (not uncommon, but not a feature in NTFS as far as I recall), as well as the data.
It's an even better to do it in some tree-like structure that lets you verify everything back to a single root block (which preferably is written many places to the disk).

Right, and I’m looking for an easy way to do that (and repair, if possible)- an application I can set to run every day or the like. Willing to buy something toward that end - just haven’t found anything yet.

H110Hawk
Dec 28, 2006

Gyrotica posted:

Right, and I’m looking for an easy way to do that (and repair, if possible)- an application I can set to run every day or the like. Willing to buy something toward that end - just haven’t found anything yet.

Can you script? You could bodge together a script which runs a sha2 sum of all the files and barfs out a flat file with the information, comparing it with the last n copies of the file to ensure 0 changes. I would do a file per run and keep the last 30 of them. The most expensive part will be reading the files in.

BlankSystemDaemon
Mar 13, 2009



Gyrotica posted:

Right, and I’m looking for an easy way to do that (and repair, if possible)- an application I can set to run every day or the like. Willing to buy something toward that end - just haven’t found anything yet.
FreeBSD (or FreeNAS if you're more into appliances) to export a zvol via CAM Target Layer to Windows iSCSI initiator, then format that iSCSI disk as NTFS.

EDIT: Have a video by one of the FreeBSD developers:
https://www.youtube.com/watch?v=lBE4BfxVDQc

BlankSystemDaemon fucked around with this message at 23:01 on Oct 27, 2019

Gyrotica
Nov 26, 2012

Grafted to machines your builders did not understand.

H110Hawk posted:

Can you script? You could bodge together a script which runs a sha2 sum of all the files and barfs out a flat file with the information, comparing it with the last n copies of the file to ensure 0 changes. I would do a file per run and keep the last 30 of them. The most expensive part will be reading the files in.

I can probably learn if there isn’t another pushbutton solution floating around out there.

Methylethylaldehyde
Oct 23, 2004

BAKA BAKA

D. Ebdrup posted:

FreeBSD (or FreeNAS if you're more into appliances) to export a zvol via CAM Target Layer to Windows iSCSI initiator, then format that iSCSI disk as NTFS.

EDIT: Have a video by one of the FreeBSD developers:
https://www.youtube.com/watch?v=lBE4BfxVDQc

Haha. I do this. RaidZ2 pools published as iSCSI drives fed into a windows VM that bitlockers them and serves files to the rest of the systems.

Adbot
ADBOT LOVES YOU

BlankSystemDaemon
Mar 13, 2009



Methylethylaldehyde posted:

Haha. I do this. RaidZ2 pools published as iSCSI drives fed into a windows VM that bitlockers them and serves files to the rest of the systems.
How does that work out against just sharing the files, on the machine that hosts the iSCSI initiator target, with samba?
Why pools, though? The whole point of ZFS is pooled storage.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply