Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Rexxed
May 1, 2010

Dis is amazing!
I gotta try dis!

With Logitech, they do seem to have gotten worse about RMAs since they started using lower voltage in some mice which is cited as a reason for the switches to start failing earlier. I don't know why running a switch on 3V instead of 5 would cause the debouncing to fail prematurely but it's a thing and has been a big problem for them in the last few years. I think a lot of the company being obnoxious about RMAs is just that they've shipping a ton of mice that end up with prematurely failing switches. I have six logitechs on my desk right now with no failures in many years, but I'm also glad they're finally moving to optical switches for the main buttons on their gaming mice, starting with the G502 X series which released a month or so back. They weren't the only brand with switch issues but they took longer than many to switch to optical switches for the main buttons.

Western Digital took much longer than I'm used to for an RMA from earlier this year on a DOA 16TB disk. I suspect that's either personnel or supply chain shortages over a company move to just randomly take 3+ weeks to turn around a single RMA disk, though.

Adbot
ADBOT LOVES YOU

Ihmemies
Oct 6, 2012

So I got 4 disks first and then read about zfs. Now I know I can't add disks to the "vdev", I have to decide beforehand how many disks it will ever have. So it's kind of a waste to use only 4 for z2. One or two more then? :shrug: Some say that only numbers like 4,8 are allowed and nothing between. But I definitely do not need 8, even 5 or 6 is really unnecessary. But it will suck if I can't extend it later so...

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
OpenZFS 3.0 is going to deliver RAID-Z expansion finally. Also, ZFS’ stripe width is pretty dynamic and doesn’t nor won’t align to vdev geometry.

Ihmemies
Oct 6, 2012

So in theory I could make a 4 disk z2 vdev and when openzfs 3.0 comes I could expand that vdev with a disk or two? I don't know when I will fill that 36TB, current need is only like 10. But not being able to expand the vdev sucks...

Ihmemies fucked around with this message at 12:03 on Oct 13, 2022

Motronic
Nov 6, 2009

Rexxed posted:

Western Digital took much longer than I'm used to for an RMA from earlier this year on a DOA 16TB disk. I suspect that's either personnel or supply chain shortages over a company move to just randomly take 3+ weeks to turn around a single RMA disk, though.

Pretty sure I posted about it in this thread, but I got several DOA drives from newegg some of which I determined failed immediately and got replaced by newegg, some which unfortunately I didn't install until after the return period only to find out they were dead (one of the replacements in fact). The WD RMA process took entirely too long with little communication and no explanations.

Generic Monk
Oct 31, 2011

Ihmemies posted:

So I got 4 disks first and then read about zfs. Now I know I can't add disks to the "vdev", I have to decide beforehand how many disks it will ever have. So it's kind of a waste to use only 4 for z2. One or two more then? :shrug: Some say that only numbers like 4,8 are allowed and nothing between. But I definitely do not need 8, even 5 or 6 is really unnecessary. But it will suck if I can't extend it later so...

the 4,8 disk stuff is bullshit based on a misunderstanding of how the technology works. use however many disks you want, though obviously not a ridiculous amount (idk what the recommended upper limit is with raidz2 - 10? 12?)

i guess you could do a pool made up of 2 vdevs of 2 mirrored disks each? then if you wanted to add space and maintain consistent performance you could just add another 2-disk mirror? that gives you the same space as raidz2 with 4 disks

https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/

or a raidz1 with the 4 disks, probably fine for home use and gives you a bit of extra space. idk it’s all a game of tradeoffs, gotta figure out what you want

Motronic posted:

Pretty sure I posted about it in this thread, but I got several DOA drives from newegg some of which I determined failed immediately and got replaced by newegg, some which unfortunately I didn't install until after the return period only to find out they were dead (one of the replacements in fact). The WD RMA process took entirely too long with little communication and no explanations.

took over a month for me iirc and they sent the wrong type of drive. sucks

Generic Monk fucked around with this message at 14:36 on Oct 13, 2022

Keito
Jul 21, 2005

WHAT DO I CHOOSE ?

Generic Monk posted:

the 4,8 disk stuff is bullshit based on a misunderstanding of how the technology works. use however many disks you want, though obviously not a ridiculous amount (idk what the recommended upper limit is with raidz2 - 10? 12?)

i guess you could do a pool made up of 2 vdevs of 2 mirrored disks each? then if you wanted to add space and maintain consistent performance you could just add another 2-disk mirror? that gives you the same space as raidz2 with 4 disks

https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/

or a raidz1 with the 4 disks, probably fine for home use and gives you a bit of extra space. idk it’s all a game of tradeoffs, gotta figure out what you want

took over a month for me iirc and they sent the wrong type of drive. sucks

RAIDZ1 is generally not recommended for pools made up of large disks due to risk of secondary failure during long resilver operations. Might be better off going with mirrored vdevs instead then.

I built my pool with RAIDZ2 as being able to deal with any 2 disks dying is much more important for my use than performance. If perf was important I wouldn't be using HDDs, the way I see it.

Ihmemies
Oct 6, 2012

I don't care too much about performance. More abut messing with backups if over 1 drive dies. So at least 2 parity drives.

I'll do a raid z2 with 4 drives and figure out later what to do. That gives 36TB which should last a while in home use.

A little space is wasted in openzfs 3.0 when adding disks to vdev. I guess that is a reasonable tradeoff for flexibility. I didn't want to buy 5 disks since even 18TB is too much space for now. 54GB would be way over the top.

I'd be basically running an empty 18TB drive probably for a year or two if not more, only to get an insignificant amount of space savings later on.

Ihmemies fucked around with this message at 15:22 on Oct 13, 2022

IOwnCalculus
Apr 2, 2003





Keito posted:

RAIDZ1 is generally not recommended for pools made up of large disks due to risk of secondary failure during long resilver operations.

I mean, it's not a zero risk, but it's low. raidz doesn't poo poo the whole array for a single error on another disk during recovery, and that's the scenario that every "RAID5 is dead" article is handwringing about.

Zorak of Michigan
Jun 10, 2006



He seems to care a lot more about degraded performance and resilver speed than I do, for home use.

Ihmemies
Oct 6, 2012

I mean data written to 4 disk z2 vdev is 50% efficient. If I add +1 the data written after that is 60% efficient, but the already written data stays at 50% efficiency.

So not going directly to 5 disk z2 means I lose some space to parity with existing data, depending on how much data I save to the vdev before getting a 5th disk.

Wibla
Feb 16, 2011

I would just do a four-drive raid6 with mdadm or raidz2 with ZFS and call it good. mdadm can be expanded anytime, ZFS will have that feature soon (tm).

OpenZFS 3.0 was tentatively slated for a 2022 release, I don't think they'll make it, but work is being done on growing raidz vdevs.

withoutclass
Nov 6, 2007

Resist the siren call of rhinocerosness

College Slice
Striped mirrored vdevs, gotta go fast!

Ihmemies
Oct 6, 2012

withoutclass posted:

Striped mirrored vdevs, gotta go fast!

My plan is to keep my photos in my SSD for a year and then move last year's folder to the HDD stack for archiving. Read speeds should be enough to stream some movies, music or casually browse photos from HDD's.

Generic Monk
Oct 31, 2011

Zorak of Michigan posted:

He seems to care a lot more about degraded performance and resilver speed than I do, for home use.

by the same token all this poo poo is pretty overkill for home use tbf. i use a 3 disk raidz1 despite the protestations of others in this thread and have never had any problems with it since like 2015 when i first set it up, knock on wood. granted i only used 3 3tb, now 6tb drives so it’s not like the resilvers take ages

when raidz expansion hits i’ll probably fill up the empty bay with another drive and really tempt fate. all the non-ephemeral data is backed up anyway

Zorak of Michigan
Jun 10, 2006


As a random data point, I'm a 5 and 1/4 years into my basement FreeNAS server's RAID-Z2 array of 8 x 8TB Red drives. Almost exactly at 5 years, I had one drive throw a SMART alert about pending sectors, and then another one throw the same error a couple months later. No problems ever surfaced at the zpool level. I replaced both drives with new 16TB Red Pros, and the resilvering for my ~25TB of data has taken a couple days, during which I did not notice any performance issues at all. I plan to replace all the 8TB drives with the same 16TB model over the next six months, and that should set me up nicely until 2028 or so.

Ihmemies
Oct 6, 2012

How would one copy files from a HDD and verify that they were copied successfully?

I found a stack of 17 HDD's totalling about ~16TB. I'd like to find out if they work, what's in them, copy all files and then zero out the drives. Any pro tips? Preferrably in Linux enviroment. I especially don't care about metadata like ntfs owner thingys, preferrablly they'd get lost in transfer, I just want the files.

I ordered an USB 3 dock which should hopefully have enough speed. Just insert a drive to the slot, and it should work without any screws.

Keito
Jul 21, 2005

WHAT DO I CHOOSE ?

Ihmemies posted:

How would one copy files from a HDD and verify that they were copied successfully?

I found a stack of 17 HDD's totalling about ~16TB. I'd like to find out if they work, what's in them, copy all files and then zero out the drives. Any pro tips? Preferrably in Linux enviroment. I especially don't care about metadata like ntfs owner thingys, preferrablly they'd get lost in transfer, I just want the files.

I ordered an USB 3 dock which should hopefully have enough speed. Just insert a drive to the slot, and it should work without any screws.



rsync will verify that checksums match post transfer, so you can be certain that the source data has been copied as-is to the destination.

Verifying the integrity of the source data is another matter entirely. It can be automated to some degree but is challenging as the checks would have to be implemented specific to each file format, and even then 100% assurance can't be guaranteed for many formats.

Ihmemies
Oct 6, 2012

I mean it is enough that the file copier copies the files how they were on the hdd, and verifies that it copied them successfully, as they were. I guess I have to learn to use rsync.

Tesseraction
Apr 5, 2009

Ihmemies posted:

I mean it is enough that the file copier copies the files how they were on the hdd, and verifies that it copied them successfully, as they were. I guess I have to learn to use rsync.

Pretty easy:

rsync -Pav /mnt/yourdisk/* /wherever/you/want/it/to/go/

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



Tesseraction posted:

Pretty easy:

rsync -Pav /mnt/yourdisk/* /wherever/you/want/it/to/go/

A ZFS pool as the destination will also ensure that going forward your files maintain their checksums and will autocorrect random bit rot (assuming you setup a pool with RaidZ or higher fault tolerance)

Flipperwaldt
Nov 11, 2011

Won't somebody think of the starving hamsters in China?



Probably several frontend gui things for rsync exist if you're more the clicky clicky type.

Ihmemies
Oct 6, 2012

Flipperwaldt posted:

Probably several frontend gui things for rsync exist if you're more the clicky clicky type.

I'd have to install a gui. I guess the command works which Tesseract gave. Thanks! Yes I will make a z2 pool and I have ecc memory.

Klyith
Aug 3, 2007

GBS Pledge Week
If you are posting ITT it is probably worth your time to learn the basics of rsync. It is the tool for moving data around in a reliable way.

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



Klyith posted:

If you are posting ITT it is probably worth your time to learn the basics of rsync. It is the tool for moving data around in a reliable way.

https://linux.die.net/man/1/rsync

Or just man rsync
or
rsync --help
in your terminal

IOwnCalculus
Apr 2, 2003





Klyith posted:

If you are posting ITT it is probably worth your time to learn the basics of rsync. It is the tool for moving data around in a reliable way.

This, but the command that Tesseraction gave you really is 99% of what you need to know.

When you get to the point of moving large chunks of data between ZFS filesystems, it pays to learn zfs send | zfs receive because that's much faster than rsync if the dataset contains a lot of small files.

BlankSystemDaemon
Mar 13, 2009



IOwnCalculus posted:

This, but the command that Tesseraction gave you really is 99% of what you need to know.

When you get to the point of moving large chunks of data between ZFS filesystems, it pays to learn zfs send | zfs receive because that's much faster than rsync if the dataset contains a lot of small files.
You're absolutely right, but the real advantage of zfs send | receive is the speed of the future delta transfers.

Ihmemies
Oct 6, 2012

This thing even appears to work. I shared the pci-e 6-port SATA controller completely to TrueNAS VM.



Impressive. I wonder when it will break... I must noob out something sooner or later.

Wibla
Feb 16, 2011


Temp setup (i5-2500k) with truenas scale on bare metal, pushing data from my old fileserver.

IOwnCalculus
Apr 2, 2003





BlankSystemDaemon posted:

You're absolutely right, but the real advantage of zfs send | receive is the speed of the future delta transfers.

I've only used it for one-time transfers so far but yes, holy poo poo the second "okay is this whole data set really up to date" pass is massively faster than doing a second rsync for the same purpose.

Ihmemies
Oct 6, 2012

So I managed to setup a Data vdev in truenas, and Video, Music and Photos datasets.

I even managed to make a user and share the Video folder with SMB so my Windows PC's can see it.

I assume there is no downsides by using SMB datasets, hopefully, even altough linux pc's/devices will use them too...

Anyways, I managed to make a docker container for sabnzbd and qbittorrent. How do you set up the downloads and file sharing? I mean do you have a separate Downloads dataset or something?

How do you share Truenas SMB datasets to the sabnzbd containers etc so they can see them.. I guess I need to put the share to the linux machine I'm running docker on, and somehow share the share with sabnzbd... hnngh.

BlankSystemDaemon
Mar 13, 2009



Apropos of nothing, since not everyone has IPMI but being able to control the power of a machine used as a server is very useful, it's cool that remote power buttons is apparently super easy.

Ihmemies posted:

So I managed to setup a Data vdev in truenas, and Video, Music and Photos datasets.

I even managed to make a user and share the Video folder with SMB so my Windows PC's can see it.

I assume there is no downsides by using SMB datasets, hopefully, even altough linux pc's/devices will use them too...

Anyways, I managed to make a docker container for sabnzbd and qbittorrent. How do you set up the downloads and file sharing? I mean do you have a separate Downloads dataset or something?

How do you share Truenas SMB datasets to the sabnzbd containers etc so they can see them.. I guess I need to put the share to the linux machine I'm running docker on, and somehow share the share with sabnzbd... hnngh.
Windows datasets are called that because Samba needs Windows access control lists, which are largely incompatible with the POSIX 1e ACLs that you typically find on lots of Unix-like boxen (especially Linux, for reasons that I'll touch on later).

TrueNAS is FreeBSD which means it has a better NFS implementation, but it also means that it supports NFS 4(.1), which implements a new type of NFSv4 ACLs.
Those were explicitly made to be compatible with Windows ACLs, because Microsoft joined the other companies in re-tooling the specification.
NFSv4 touches more than ACLs though, for example locking is now done in-line instead of via out-of-band RPC to a separate lockd, everything uses one TCP port, and caching has been reworked , there's a parallel mode for increasing IOPS if you've got more money than sense, and there's server side copies, application hole punching, and many other nice things - and the future will likely bring NFS-over-TLS (which already works on FreeBSDs development branch).

What this means is that as long as you're sticking to Samba shares, it's fine - but if you wanna do Samba+NFS shares, you need to stick with FreeBSD or migrate to something Illumos-based until Linux gets support for NFSv4 ACLs.
That isn't going to happen any time soon, as there's apparently nobody interested in using NFS on Linux - which is also evident when you see how both docker and git (both very commonly used on Linux) not working well over NFS.

BlankSystemDaemon fucked around with this message at 12:55 on Oct 15, 2022

Ihmemies
Oct 6, 2012

Truenas scale is based on Debian linux. So I'll just do SMB datasets, fine. Now if I knew how to figure out if truenas made ashift 9, 12 or 13 pool. 9 sucks on a 18TB disk..

Wibla
Feb 16, 2011

Ihmemies posted:

Truenas scale is based on Debian linux. So I'll just do SMB datasets, fine. Now if I knew how to figure out if truenas made ashift 9, 12 or 13 pool. 9 sucks on a 18TB disk..

From my Truenas scale install:
code:
# zpool get all|grep ashift
boot-pool  ashift                         12                             local
data       ashift                         12                             local
scratch    ashift                         12                             local

Ihmemies
Oct 6, 2012

Wibla posted:

From my Truenas scale install:
code:
# zpool get all|grep ashift
boot-pool  ashift                         12                             local
data       ashift                         12                             local
scratch    ashift                         12                             local

I figured out that get all displays the properties. But there was ashift mentioned 0 times? :confuoot:

I understan that 512B sectors are very demanding on 4K sector disks. SSD's use 8K sectors?

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
Yea, TrueNAS Scale chose ashift 12 for my HD pool, and 13 for the SSD pool.

smax
Nov 9, 2009

Sanity check on Synology models: Give the choice between a DS218 (not +) and a DS220j (assuming they're very close in price), I should go with the DS218 right? They look like they have almost identical specs, aside from the DS218 having 2 GB RAM vs the DS220j's 512 MB and the front USB port on the DS218.

frogbs
May 5, 2004
Well well well
I'm just looking for a 4tb or 5tb hard drive to use with my Macbook for Timemachine. Ideally I want something that uses USB-C. Is this my best bet/option? https://www.amazon.com/WD-4TB-My-Passport-Ultra-Silver-Portable-External/dp/B07GKBT14V/

Some reviews say the cable it comes with sucks, and/or it's even slower when using the included usb-c -> usb-a cable. Should I be worried or just go for it?

Rexxed
May 1, 2010

Dis is amazing!
I gotta try dis!

frogbs posted:

I'm just looking for a 4tb or 5tb hard drive to use with my Macbook for Timemachine. Ideally I want something that uses USB-C. Is this my best bet/option? https://www.amazon.com/WD-4TB-My-Passport-Ultra-Silver-Portable-External/dp/B07GKBT14V/

Some reviews say the cable it comes with sucks, and/or it's even slower when using the included usb-c -> usb-a cable. Should I be worried or just go for it?

The upside of those kind of disks is that they're small, but the downside is that internally it's a thick laptop style 2.5" drive. They make them like that because they can be entirely powered by the one plug instead of an external power brick like a 3.5" disk and they're fairly cheap. They're also kind of slow just as drives.

As long as you remember to never have only one copy of anything it'll probably be okay. Also it will just inherently not be super fast, cable or interface won't matter (unless you plug into a USB 2.0 port or use a USB-C to C 2.0 cable). It may die just because they're not fantastic disks or anything but one of the comments says it has a 3 year warranty which is pretty good for externals, I thought they were all 1-2 years at this point.

Adbot
ADBOT LOVES YOU

Flipperwaldt
Nov 11, 2011

Won't somebody think of the starving hamsters in China?



smax posted:

Sanity check on Synology models: Give the choice between a DS218 (not +) and a DS220j (assuming they're very close in price), I should go with the DS218 right? They look like they have almost identical specs, aside from the DS218 having 2 GB RAM vs the DS220j's 512 MB and the front USB port on the DS218.
I would think so yeah. You can't put extra ram in either, so 2gb is a way better starting point. There will be a point when one can get the newest software and the other won't, but currently it's only the 2012 and earlier models that can't update to dsm 7, so that's still a good while away for the 218 (longer than the lifespan of the disks you'd put in now). I can't really think of a positive reason to go for the 220j if it isn't maybe the price being half that of the other. If the 220+ were in your price range, that would be a worthwhile upgrade cpu wise, but I'm assuming you've looked at that.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply