NAS/Storage Megathread: What is this "File Deletion" You Speak of

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›847 »

eightysixed: Sep 23, 2004; I always tell the truth. Even when I lie.

What does the thread think about teh 4TB Toshiba NAS drives? That's sort of cheap, and I could use a few :ohdear:

# ? Nov 21, 2017 18:14

Adbot: ADBOT LOVES YOU

# ? May 26, 2024 01:24

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

X300s are made on the same line as the enterprise drives, and the NAS drives probably come off the same line. They're fine.

The 4 TB and 5 TB models are kinda questionable value IMO vs the 6 GB X300s or shucking some 8 TB WD Reds for $180 or some 8 GB white-label HGSTs for $130.

# ? Nov 21, 2017 18:52

eclectic taste: Jun 5, 2004; Future Schmidt

Newegg has a deal right now on 4x4TB HGST NAS drives for $400: https://www.newegg.com/Product/ComboDealDetails.aspx?ItemList=Combo.3683807

# ? Nov 21, 2017 19:55

DrDork: Dec 29, 2003; commanding officer of the Army of Dorkness

Paul MaudDib posted:

some 8 TB white-label HGSTs for $130.

If these ever come back, they're hands down the best option considering the price. Sadly, I doubt we'll see them again anytime soon--500 units from a retailer as large as BestBuy just screams loss-leader or ditching stock.

# ? Nov 21, 2017 20:03

Thanks Ants: May 21, 2004; #essereFerrari

Are WD Re disks any good? I have been gifted a pair of 6TB ones.

# ? Nov 21, 2017 20:10

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

DrDork posted:

If these ever come back, they're hands down the best option considering the price. Sadly, I doubt we'll see them again anytime soon--500 units from a retailer as large as BestBuy just screams loss-leader or ditching stock.

Those are the ones in the cheaper WD Easystore, right?

I thought they were going to be available in store at Best Buy on Friday...

# ? Nov 21, 2017 20:11

BlankSystemDaemon: Mar 13, 2009

Thanks Ants posted:

Are WD Re disks any good? I have been gifted a pair of 6TB ones.

They're enterprise drives, so far as I know. Whoever gifted you those is very kind.

EDIT: WD helpfully relabeled them as WD Gold disks, not too long ago.

BlankSystemDaemon fucked around with this message at 20:53 on Nov 21, 2017

# ? Nov 21, 2017 20:49

eightysixed: Sep 23, 2004; I always tell the truth. Even when I lie.

eclectic taste posted:

Newegg has a deal right now on 4x4TB HGST NAS drives for $400: https://www.newegg.com/Product/ComboDealDetails.aspx?ItemList=Combo.3683807

I guess I'll go ahead and buy this. I wish I could get it from Amazon though, as I have amassed like $550 in gift cards :saddowns:

# ? Nov 21, 2017 21:32

astral: Apr 26, 2004

eightysixed posted:

I guess I'll go ahead and buy this. I wish I could get it from Amazon though, as I have amassed like $550 in gift cards

https://www.groupon.com/deals/newegg-4-national could help a little

# ? Nov 21, 2017 23:42

Thanks Ants: May 21, 2004; #essereFerrari

D. Ebdrup posted:

They're enterprise drives, so far as I know. Whoever gifted you those is very kind.

EDIT: WD helpfully relabeled them as WD Gold disks, not too long ago.

That explains why I couldn't find them - WD's website only lists 250GB and 500GB models and that was confusing me. I wish they wouldn't rebrands things on a monthly basis.

# ? Nov 22, 2017 00:01

Chilled Milk: Jun 22, 2003; No one here is alone,
satellites in every home

So, I finally migrated off my rapidly decaying Corral install over the weekend. I was trying to hold off until 11.1 for the Docker support. But predictably, 11.1 is late, and probably won't even have it anyway if I'm reading the tea leaves right. Between plugins/a couple hand rolled jails I have my essential services running again.

Two questions:

Is there extra upkeep I need to do for each jail, or does it get updated with the rest of the system? I only used plugins back on 9.x so I never really learned much about actual jails, also aside from Emby none of it was exposed publicly.

Aside from media stuff, what else do y'all run on your servers? Probably gonna up a Certbot jail so I can HTTPS up my Emby (and NextCloud whenever I get around to setting that up), but I'm wondering if there's any other little services that would make life easier that I'm not thinking of.

# ? Nov 22, 2017 01:00

EVIL Gibson: Mar 23, 2001; Internet of Things is just someone else's computer that people can't help attaching cameras and door locks to!; Switchblade Switcharoo

The Milkman posted:

So, I finally migrated off my rapidly decaying Corral install over the weekend. I was trying to hold off until 11.1 for the Docker support. But predictably, 11.1 is late, and probably won't even have it anyway if I'm reading the tea leaves right. Between plugins/a couple hand rolled jails I have my essential services running again.

Two questions:

Is there extra upkeep I need to do for each jail, or does it get updated with the rest of the system? I only used plugins back on 9.x so I never really learned much about actual jails, also aside from Emby none of it was exposed publicly.

Aside from media stuff, what else do y'all run on your servers? Probably gonna up a Certbot jail so I can HTTPS up my Emby (and NextCloud whenever I get around to setting that up), but I'm wondering if there's any other little services that would make life easier that I'm not thinking of.

If you haven't yet, scheduling your zfs scrub to perform weekly.

Installed a service to connect to my Google drive through command line so I can upload/process stuff to my server remotely via the drive .

Script to download direct 1080p from services I subscribe to via youtube-dl (not just YouTube, check out the filter list).

Let me know how that emby thing works out for you. I wanted to host it on a reverse ssl proxy connection where it only accepts connections that have the matching certs on the port I've created.

EVIL Gibson fucked around with this message at 01:24 on Nov 22, 2017

# ? Nov 22, 2017 01:21

G-Prime: Apr 30, 2003; Baby, when it's love,
if it's not rough it isn't fun.

EVIL Gibson posted:

If you haven't yet, scheduling your zfs scrub to perform weekly.

This just reminded me to check. Apparently, despite being scheduled in Corral, my system hasn't done a scrub since 9/2. I am super annoyed and scared now.

eightysixed posted:

What does the thread think about teh 4TB Toshiba NAS drives? That's sort of cheap, and I could use a few

For what it's worth, I just retired 8 of these, only because they ran out of space and I upgraded to 8TB drives. They were rock solid, and quiet, over the two years they ran.

Edit: When suddenly, Corral starts making GBS threads all over itself. Thinks the Docker management service doesn't exist, insists a VM is running, but then gets mad and says it doesn't exist when I want to restart it, gave me a grub error on boot... Jesus.

G-Prime fucked around with this message at 03:41 on Nov 22, 2017

# ? Nov 22, 2017 02:22

DrDork: Dec 29, 2003; commanding officer of the Army of Dorkness

Thermopyle posted:

Those are the ones in the cheaper WD Easystore, right?

I thought they were going to be available in store at Best Buy on Friday...

Yup, those are the ones.

:iiam:

They were available as a pre-Black Friday sale both through eBay and their website, and sold out almost immediately. They've made no indication either way on if they'll restock for Friday.

# ? Nov 22, 2017 05:30

Chilled Milk: Jun 22, 2003; No one here is alone,
satellites in every home

Good call on checking my scheduled tasks, I forgot to do that. Does this look okay?

Scrubs

SMART tests

# ? Nov 22, 2017 06:27

BeastOfExmoor: Aug 19, 2003; I will be gone, but not forever.

DrDork posted:

Yup, those are the ones.

They were available as a pre-Black Friday sale both through eBay and their website, and sold out almost immediately. They've made no indication either way on if they'll restock for Friday.

They're in the Best Buy Black Friday (which starts Thursday night) Ad. It's unclear if they'll be sold online or only in stores.

# ? Nov 22, 2017 09:01

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

Trip report: the Home Networking thread is awful. Never post there, you may answer as many as 8 impossible questions before breakfast

:fireman:

Paul MaudDib fucked around with this message at 12:36 on Nov 22, 2017

# ? Nov 22, 2017 11:49

eclectic taste: Jun 5, 2004; Future Schmidt

4TB Reds for $119 at newegg (limit 2): https://www.newegg.com/Product/Prod...S0A&ignorebbr=1

# ? Nov 22, 2017 16:50

Proteus Jones: Feb 28, 2013

Paul MaudDib posted:

Trip report: the Home Networking thread is awful. Never post there, you may answer as many as 8 impossible questions before breakfast

This is truth. There's a lot of Cargo Cult Networking going on in there. The last time I descended into that cess pit there was a lot of "This is how it works because these 3 million config changes I made indiscriminately fixed my problem!" and they got very angry when you pointed out that it made no sense.

# ? Nov 22, 2017 17:08

Eletriarnation: Apr 6, 2005; People don't appreciate the substance of things...
objects in space.; Oven Wrangler

You gotta be the change you want to see, not snipe from another thread. poo poo posts come from poo poo posters and not the thread aether. If you get sick of posting back at them you can always vote 1 and move on. :shrug:

# ? Nov 22, 2017 17:26

DrDork: Dec 29, 2003; commanding officer of the Army of Dorkness

BeastOfExmoor posted:

They're in the Best Buy Black Friday (which starts Thursday night) Ad. It's unclear if they'll be sold online or only in stores.

I hope you're right, because they're a hell of a deal. That they've already been on sale once at that price and sold out almost immediately makes me less than hopeful for available stock, though, especially when I'd need 4 of them to do anything with

# ? Nov 22, 2017 20:29

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

DrDork posted:

I hope you're right, because they're a hell of a deal. That they've already been on sale once at that price and sold out almost immediately makes me less than hopeful for available stock, though, especially when I'd need 4 of them to do anything with

Order one to your house and one to your office. If you had ordered them the first time, this would be an opportunity to snag 4 total drives.

# ? Nov 22, 2017 20:47

BlankSystemDaemon: Mar 13, 2009

Might be worth mentioning that Intel C3000/Denverton is affected by the Intel ME security issue, too (In addition to every other post-Skylake CPU; Basically if you buy anything Intel, you need to patch as soon as you have an OS installed).

# ? Nov 22, 2017 21:16

ChiralCondensate: Nov 13, 2007; what is that man doing to his colour palette?; Grimey Drawer

BeastOfExmoor posted:

They're in the Best Buy Black Friday (which starts Thursday night) Ad. It's unclear if they'll be sold online or only in stores.

NEBB for $130 sold out online but I went ahead and got a NESN for $160.

# ? Nov 23, 2017 12:30

Incessant Excess: Aug 15, 2005; Cause of glitch:
Pretentiousness

Is anyone familiar with the QNAP TS-453A? I'm looking for a NAS that allows me to run Usenet software to automatically download stuff (Sabnzbd, Sonarr, Radarr) and maybe also do some automatic backups. From what I've seen, Docker support is recommended for that sort of stuff as those versions are apparently best maintained. I don't know a ton about networking stuff, so I would like something that is relatively easy to set up and use. The 453 has 4GB of RAM and an Intel cpu which is recommended for Docker from my understanding.

# ? Nov 24, 2017 10:54

SlowBloke: Aug 14, 2017

Incessant Excess posted:

Is anyone familiar with the QNAP TS-453A? I'm looking for a NAS that allows me to run Usenet software to automatically download stuff (Sabnzbd, Sonarr, Radarr) and maybe also do some automatic backups. From what I've seen, Docker support is recommended for that sort of stuff as those versions are apparently best maintained. I don't know a ton about networking stuff, so I would like something that is relatively easy to set up and use. The 453 has 4GB of RAM and an Intel cpu which is recommended for Docker from my understanding.

Buy the 453B model if you can, more expansions options and a more powerful cpu for container-station(the name qnap uses for docker on their literature/documentation).

SlowBloke fucked around with this message at 12:58 on Nov 24, 2017

# ? Nov 24, 2017 12:47

Main Paineframe: Oct 27, 2010

eclectic taste posted:

4TB Reds for $119 at newegg (limit 2): https://www.newegg.com/Product/Prod...S0A&ignorebbr=1

Nice, time to finally fill up the empty bays in my 4-bay QNAP box. Of course, to upgrade it from RAID-1 to RAID-5, I need to take all the data off it, so maybe I should grab an external drive too...

# ? Nov 24, 2017 19:19

SlowBloke: Aug 14, 2017

Main Paineframe posted:

Nice, time to finally fill up the empty bays in my 4-bay QNAP box. Of course, to upgrade it from RAID-1 to RAID-5, I need to take all the data off it, so maybe I should grab an external drive too...

Unless your QNAP is very old you can upgrade inplace without any volume wipe/format https://www.qnap.com/en/how-to/tutorial/article/online-raid-level-migration , having a backup always help tho.

# ? Nov 24, 2017 19:26

BlankSystemDaemon: Mar 13, 2009

Wasn't there some mentions of Infiniband on FreeBSD at some point? Well, a big load of changes just landed in HEAD with Chelsio and Mellanox drivers.

BlankSystemDaemon fucked around with this message at 21:06 on Nov 24, 2017

# ? Nov 24, 2017 21:01

Azhais: Feb 5, 2007; Switchblade Switcharoo

eclectic taste posted:

4TB Reds for $119 at newegg (limit 2): https://www.newegg.com/Product/Prod...S0A&ignorebbr=1

Limit 5 atm

# ? Nov 24, 2017 22:02

Chuu: Sep 11, 2004; Grimey Drawer

I'm experimenting with FreeNAS on a Hyper-V VM. I have an 8 disk RaidZ3 array. I'm trying to fill it up the dataset, so I just do "cat /dev/urandom > random_data". I see the dataset filling up as expected, except when it finally gets full . . . nothing happens. No system error, no broken pipe, nothing that indicates a write error has occurred. 'cat' is still piping data to . . . somwhere.

This is a bit disconcerting. What is happening under the covers? Why isn't an error raised by bash when the write fails due to the dataset being full like what would happen in Linux?

# ? Nov 25, 2017 07:14

Chuu: Sep 11, 2004; Grimey Drawer

One more ZFS question. So I brought FreeNAS down, and used a Hex Editor to edit three of the eight disks in my Z3 pool to simulate data rot. Bring FreeNAS back up . . . and for some reason the web client wont start. I scrub the pool to see what ZFS makes of the errors I've introduced. I get the following result:

code:

root@freenas:/ # zpool status -v
  pool: PrimaryVolume
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: [url]http://illumos.org/msg/ZFS-8000-9P[/url]
  scan: scrub repaired 72K in 0h0m with 0 errors on Fri Nov 24 22:23:20 2017
config:

        NAME                                            STATE     READ WRITE CKSUM
        PrimaryVolume                                   ONLINE       0     0     0
          raidz3-0                                      ONLINE       0     0     0
            gptid/be639b4e-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/bf25c58b-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/bff059ab-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/c0cbad00-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/c1a2e809-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/c269673e-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     1
            gptid/c2eaeb56-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     1
            gptid/c36c0c72-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     1

errors: No known data errors

I thought Z3 should be able to recover from three disks failing? Am I misinterpreting the message, or did flipping a few bytes on three different drives really lead to an unrecoverable error? Also, what's with that 0h0m time? Every time I've run scrub before with no errors I would get a realistic scrub time.

Chuu fucked around with this message at 07:34 on Nov 25, 2017

# ? Nov 25, 2017 07:31

IOwnCalculus: Apr 2, 2003

Reread that status. It hasn't lost data, but it's also marked all three of those drives as defective and doesn't believe it has anything new to rebuild with.

# ? Nov 25, 2017 07:53

redeyes: Sep 14, 2002; by Fluffdaddy

I have an 8 drive JBOD array which is in my main workstation box. I really want to get the disks out of the box and into a separate rackmount. I would like to not build a whole computer around the JBOD array but maybe I have to. I was checking out Esata with port multipliers. Seems sketchy at best. Any recommendations?

I am trying to save the power of running a server 24/7, I really just need the array online.

Newegg has HGST 4TB with 20 bux off coupon = $99 https://www.newegg.com/Product/Product.aspx?item=N82E16822146116

redeyes fucked around with this message at 16:24 on Nov 25, 2017

# ? Nov 25, 2017 16:14

SamDabbers: May 26, 2003

redeyes posted:

I have an 8 drive JBOD array which is in my main workstation box. I really want to get the disks out of the box and into a separate rackmount. I would like to not build a whole computer around the JBOD array but maybe I have to. I was checking out Esata with port multipliers. Seems sketchy at best. Any recommendations?

Don't use SATA port multipliers. They're definitely sketchy. I'd grab an LSI 9200-8e for ~$50 on eBay, and either a Lenovo SA120 chassis or something like the Rosewill RSV-L4412 with an SFF-8088 PCI bracket.

# ? Nov 25, 2017 16:26

Chuu: Sep 11, 2004; Grimey Drawer

IOwnCalculus posted:

Reread that status. It hasn't lost data, but it's also marked all three of those drives as defective and doesn't believe it has anything new to rebuild with.

Thanks for the response. I did notice after reboot, the status was reset. Specifically after a reboot this is what the status looks like:

code:

root@freenas:/ # zpool status -v
  pool: PrimaryVolume
 state: ONLINE
  scan: scrub repaired 72K in 0h0m with 0 errors on Fri Nov 24 22:23:20 2017
config:

        NAME                                            STATE     READ WRITE CKSUM
        PrimaryVolume                                   ONLINE       0     0     0
          raidz3-0                                      ONLINE       0     0     0
            gptid/be639b4e-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/bf25c58b-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/bff059ab-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/c0cbad00-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/c1a2e809-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/c269673e-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/c2eaeb56-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0
            gptid/c36c0c72-d1a4-11e7-8e3c-00155d017f10  ONLINE       0     0     0

errors: No known data errors

I was hoping the specific error was preserved until you did an explicit `zpool clear`, but it looks like that's not the case. More worrying is I can't find any way to track down any error occurred in the GUI -- even that the scrub had to repair some bytes. Is this to be expected, i.e. a reboot does an implicit `zpool clear`? Is there somewhere in the FreeNAS GUI it would report that a scrub had to repair blocks?

# ? Nov 25, 2017 16:46

Kashekya: Dec 11, 2003; Pillbug

Chuu posted:

I was hoping the specific error was preserved until you did an explicit `zpool clear`, but it looks like that's not the case. More worrying is I can't find any way to track down any error occurred in the GUI -- even that the scrub had to repair some bytes. Is this to be expected, i.e. a reboot does an implicit `zpool clear`? Is there somewhere in the FreeNAS GUI it would report that a scrub had to repair blocks?

In the FreeNAS gui: Storage -> select the pool -> Click "Volume Status" at the bottom (button looks like a blank piece of paper).

# ? Nov 25, 2017 16:55

Chuu: Sep 11, 2004; Grimey Drawer

Kashekya posted:

In the FreeNAS gui: Storage -> select the pool -> Click "Volume Status" at the bottom (button looks like a blank piece of paper).

Thanks, I see it now. Is there somewhere on the system it preserves which disks had the checksum error? Also, in ZFS is repaired data after a scrub a common enough occurrence it doesn't change the big green light from "OK" or something else?

I've also read there is a threshold after which "bit rot" is considered a hard error. Is this limit per-scrub, or is it cumulative? If cumulative where can I see the cumulative statistics for the drives in the pool?

(Also, is there a good in-depth technical doc that could answer all these questions? Most of the zfs/freenas guides I've seen focus on setup/configuration, not how ZFS/FreeNAS tracks/handles faults)

Chuu fucked around with this message at 17:01 on Nov 25, 2017

# ? Nov 25, 2017 16:59

redeyes: Sep 14, 2002; by Fluffdaddy

SamDabbers posted:

Don't use SATA port multipliers. They're definitely sketchy. I'd grab an LSI 9200-8e for ~$50 on eBay, and either a Lenovo SA120 chassis or something like the Rosewill RSV-L4412 with an SFF-8088 PCI bracket.

OK that's good advice. One thing, I already have an iStar 4u rack with 2 iStar SATA hotswap bays (4x). Any way to accomplish the same kind of thing without chassis? Can I just use that SFF-8088 PCI bracket in 'whatever'? I assume one end has SAS to SATA cables coming out which then plugs directly into the drives?

redeyes fucked around with this message at 17:16 on Nov 25, 2017

# ? Nov 25, 2017 17:06

Adbot: ADBOT LOVES YOU

# ? May 26, 2024 01:24

SamDabbers: May 26, 2003

redeyes posted:

OK that's good advice. One thing, I already have an iStar 4u rack with 2 iStar SATA hotswap bays (4x). Any way to accomplish the same kind of thing without chassis?

SFF-8088 PCI bracket, SFF-8088 cables for external connection to the LSI card, and SFF-8087 [forward] breakout cables inside the iStar case to the hot swap bays.

# ? Nov 25, 2017 17:13

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›847 »