NAS/Storage Megathread: What is this "File Deletion" You Speak of

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›848 »

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

error1 posted:

The problem is made worse with consumer drives like the WD Green and others, where the drive might lock up for a very long period trying to read a block, causing the raid controller to think the drive is completely dead when there's really just a tiny bit of inaccessible data on it.

At least with the WD drives, you can (or used to be able to) enable TLER. I've done that on my WD Greens (EADS) just fine.

(Of course, one of two WD Greens was a dud, too, constantly being stuck in self-test while idle. Theoretically not a problem, if it wasn't that these weren't supported in 24/7 either. So you wouldn't want additional wear on them. And it was annoying because of the constant seeking sounds.)

# ? Sep 9, 2012 23:56

Adbot: ADBOT LOVES YOU

# ? May 31, 2024 16:56

Longinus00: Dec 29, 2005; Ur-Quan

spaceship posted:

I could do a Synology ds512+ with 5 3tb caviar greens for $1594. I have heard that the Caviar greens aren't great for Nas's, but will it really matter in my case? I will rarely ever have more than 1 user connected at a time.

You sure like your entry level buisness solutions. At that price point you might as well just build a computer to serve music/movies around the house, it would certainly be cheaper up front.

# ? Sep 10, 2012 00:47

necrobobsledder: Mar 21, 2005; Lay down your soul to the gods rock 'n roll; Nap Ghost

The Microserver is highly recommended for lots of reasons including that you can probably sell it to someone else in a jiffy if you really had to, not quite so sure with that Western Digital NAS and all those expensive drives. I just find it really peculiar how someone could need enterprise drives for home environments unless they're doing something professional as well.

The primary advantage an enterprise drive would have for home use (aside from the TLER / 512 byte sector issues) would be the 5 year warranty on them (but those were reduced last I remember so....). I'd say it's substantially cheaper and less risky to just buy one more drive (say, a Black variant, not Raid Edition) and up the RAID level to compensate. Unless you need thousands of IOPS at home... (holy hell you have one amazing porn or anime habit if you do).

Longinus00 posted:

Are you basing your experience of atom+network cards on NAS systems or consumer atom boards?

Both! I had an old mini ITX board I had bought for the purpose of being a file server and it was a dual core Atom. While the chipset and NIC were factors in the slow overall file performance, with ZFS' checksum upon read, it does affect the throughput for even basic file serving. If you read around for those that ran ZFS with Atom CPUs (a guy that used that one Chenbro case that was the rage a couple years ago comes to mind) their ZFS read performance on 512 byte sector drives were eyebrow-raising slow - in the 40MBps or so range. Wife's netbook runs an N280, and it can't even keep up with Youtube videos properly (kinda scary considering that I thought they were supposed to be about as fast as a 550 MHz P3 or so). Even the Ion2 based HTPC I had was sluggish doing anything but playing back a movie in XBMC, and even then it was dog slow compared to the mediocre AMD CPU in the HP Microserver running, say, the Aeon Nox skin.

While we may say that only one user might be using a file server at any time, this breaks down pretty quickly when people start running things like sabnzbdplus and Couchpotato / Sickbeard, an iTunes server, etc. I found that getting a file server up has resulted in more and more reliance upon the sucker to do more than one thing at a time, plain and simple. Parkinson's Law like a mofo

Longinus00 posted:

Wait, this thing runs windows server? Yea this is aimed at small businesses and not home users who don't care about AD features.

That's pretty much the other part of the equation - software trumping the capex of hardware. If you're trying to integrate with AD faster and so forth, the enterprise drive cost likely means nothing to you anyway compared to the opex costs of having your sysadmin spend another day setting up ZFS or even Linux and XFS + LVM with mdraid. For home situations... well it's a question of pain threshold for mucking about with computers when you could theoretically be doing something you value more like getting drunk and macking on girls or something.

# ? Sep 10, 2012 02:57

Telex: Feb 11, 2003

I know it's not necessarily storage related, but anyone know a good UPS that will work with FreeNAS and shut things down if there's a power event? I see that there's a thing to interface with one, so I'd really like to get one that's going to be compatible.

Had a few power issues lately, so I think it's a good idea (not that it's ever been a BAD idea) to put the RAID machine on a UPS.

# ? Sep 10, 2012 05:07

arbybaconator: Dec 18, 2007; All hat and no cattle

Okay, back to the drawing board. I'll look into these HP miniservers.

# ? Sep 10, 2012 06:17

Jolan: Feb 5, 2007

error1 posted:

The thing that makes raid5 dangerous for large drives though, is that once a drive breaks and you replace it, recreating the data on it means reading all blocks from every other drive in the raid to calculate the missing data.

Do you thin QNAP raid controllers and WD RE4's with TLER would pose problems in Raid5 with read errors?

# ? Sep 10, 2012 11:35

evil_bunnY: Apr 2, 2003

Jolan posted:

Do you thin QNAP raid controllers and WD RE4's with TLER would pose problems in Raid5 with read errors?

Depends on the underlying tech. I don't think these NAS devices checksum at all, except for those running ZFS.

evil_bunnY fucked around with this message at 11:44 on Sep 10, 2012

# ? Sep 10, 2012 11:38

Jolan: Feb 5, 2007

evil_bunnY posted:

Depends on the underlying tech

Then what information do you need to be able to discern the underlying tech? If it's the NAS model, I'm talking about a TS-412.

# ? Sep 10, 2012 11:44

evil_bunnY: Apr 2, 2003

Jolan posted:

Then what information do you need to be able to discern the underlying tech? If it's the NAS model, I'm talking about a TS-412.

Their own datasheet says EXT3/4

# ? Sep 10, 2012 11:53

DigitalMocking: Jun 8, 2010; Wine is constant proof that God loves us and loves to see us happy.
Benjamin Franklin

Storage Bros, I need some help. I would appear that my Intel� RT3WB080 has died after 4 months of use (system will no longer complete POST with the card installed, remove card, everything is happy). There are 8 drives connected to the controller in raid 6 with 5 virtual drives configured on top of that (2tb, 2tb, 2tb, 2tb, 2tb, 960mb).

There aren't a lot of options for an 8 port card, and while I'll RMA the Intel card, I need to get this array back up. Is it possible to buy a pair of 4 port cards but still recreate the entire 8 drive array over multiple cards?

If the answer is yes (I hope it is). What 4 port cards would you bros recommend without breaking the bank?

Motherboard is a Supermicro X9SCM-F, Xeon proc, 8gb of RAM.
OS is installed on a pair of mirrored SSD drives attached to the motherboard.

# ? Sep 10, 2012 13:08

r u ready to WALK: Sep 29, 2001

Jolan posted:

Do you thin QNAP raid controllers and WD RE4's with TLER would pose problems in Raid5 with read errors?

From what little googling i've done, it sounds like qnap uses linux and software mdraid, so timeouts should be configurable and i saw someone ssh in to troubleshoot a broken array with mdadm commands. Not to mention that RE4 drives were made for 24/7 raid use, your data should be pretty safe.

I wouldn't worry that much about the qnap ext4 filesystem not being checksummed, if you're using it for media storage it basically means you risk a glitched frame in a movie or half a second of garbled audio in an mp3 file each time you make a new copy of your entire collection. The risk is very low to the point where most people never even notice a single bit error.

ZFS (and btrfs) is a great solution to bit rot, but not an absolute must-have for regular users. It becomes hugely important if you're storing research or financial data though. Wouldn't it be fun if your harddrive changed the account number for a customer transaction, or added extra numbers in front of someones account balance?

# ? Sep 10, 2012 14:41

Longinus00: Dec 29, 2005; Ur-Quan

necrobobsledder posted:

The Microserver is highly recommended for lots of reasons including that you can probably sell it to someone else in a jiffy if you really had to, not quite so sure with that Western Digital NAS and all those expensive drives. I just find it really peculiar how someone could need enterprise drives for home environments unless they're doing something professional as well.

The primary advantage an enterprise drive would have for home use (aside from the TLER / 512 byte sector issues) would be the 5 year warranty on them (but those were reduced last I remember so....). I'd say it's substantially cheaper and less risky to just buy one more drive (say, a Black variant, not Raid Edition) and up the RAID level to compensate. Unless you need thousands of IOPS at home... (holy hell you have one amazing porn or anime habit if you do).
Both! I had an old mini ITX board I had bought for the purpose of being a file server and it was a dual core Atom. While the chipset and NIC were factors in the slow overall file performance, with ZFS' checksum upon read, it does affect the throughput for even basic file serving. If you read around for those that ran ZFS with Atom CPUs (a guy that used that one Chenbro case that was the rage a couple years ago comes to mind) their ZFS read performance on 512 byte sector drives were eyebrow-raising slow - in the 40MBps or so range. Wife's netbook runs an N280, and it can't even keep up with Youtube videos properly (kinda scary considering that I thought they were supposed to be about as fast as a 550 MHz P3 or so). Even the Ion2 based HTPC I had was sluggish doing anything but playing back a movie in XBMC, and even then it was dog slow compared to the mediocre AMD CPU in the HP Microserver running, say, the Aeon Nox skin.

Yea I was talking about turnkey atom NAS solutions not roll-your-own freenas ones.

DigitalMocking posted:

Storage Bros, I need some help. I would appear that my Intel� RT3WB080 has died after 4 months of use (system will no longer complete POST with the card installed, remove card, everything is happy). There are 8 drives connected to the controller in raid 6 with 5 virtual drives configured on top of that (2tb, 2tb, 2tb, 2tb, 2tb, 960mb).

There aren't a lot of options for an 8 port card, and while I'll RMA the Intel card, I need to get this array back up. Is it possible to buy a pair of 4 port cards but still recreate the entire 8 drive array over multiple cards?

If the answer is yes (I hope it is). What 4 port cards would you bros recommend without breaking the bank?

Motherboard is a Supermicro X9SCM-F, Xeon proc, 8gb of RAM.
OS is installed on a pair of mirrored SSD drives attached to the motherboard.

Were you using the RAID functionality of your Intel card or doing softraid? If you were using the on card RAID then you will probably need another RT3WB080 (maybe if you're lucky another card with the same chipset might work). If you were running any form of softraid then it doesn't matter how you connect the disks so long as the cards you connect them with don't try to "initialize" them.

error1 posted:

From what little googling i've done, it sounds like qnap uses linux and software mdraid, so timeouts should be configurable and i saw someone ssh in to troubleshoot a broken array with mdadm commands. Not to mention that RE4 drives were made for 24/7 raid use, your data should be pretty safe.

I wouldn't worry that much about the qnap ext4 filesystem not being checksummed, if you're using it for media storage it basically means you risk a glitched frame in a movie or half a second of garbled audio in an mp3 file each time you make a new copy of your entire collection. The risk is very low to the point where most people never even notice a single bit error.

ZFS (and btrfs) is a great solution to bit rot, but not an absolute must-have for regular users. It becomes hugely important if you're storing research or financial data though. Wouldn't it be fun if your harddrive changed the account number for a customer transaction, or added extra numbers in front of someones account balance?

Even businesses that do need data integrity tend to just do it by adding an extra layer on top(/below) of everything else. This is because you need to have checksumming from the time the data is read/written by a client all the way to when it hits a disk on a server.

Longinus00 fucked around with this message at 16:22 on Sep 10, 2012

# ? Sep 10, 2012 16:13

movax: Aug 30, 2008

It might be due to my laziness in not wanting to gently caress with my existing setup, but I feel like bit of a dinosaur still running a pure Solaris ZFS setup in the Norco RPC-4020. The thing hasn't choked once in almost...three years now I think, and it's ridiculous the amount of "professional/enterprise"-like features I have for basically free.

I mean, seriously, the OS is free, my UPS software is free (apcupsd), I can hotswap drives for free (lsiutil), serve up files in various ways (FTP/SCP/SMB/NFS/etc) all for only the cost of hardware and powering the drat thing.

I'm looking at turning it into an ESXi box in the winter, and taping SSDs to various places inside the case to give me the extra drives I need. 18 drive bays are used for my zpool, the last 2 are just a mirror of two drives I keep to gently caress around with, so I could drop a mirror of some of the spare 2TB drives I have laying around to serve as a backup destination for the SSDs that will (hopefully) hold some of the VMs.

Only real complaint is that the 1068Es are PCIe 1.0, and I'm eating PCIe lanes that I could use to throw in a multi-port PCIe NIC or something in, but buying new controllers would be pricey. Imagine all the drives you could run off one physical system with PCIe 3.0 + liberal use of PCIe switches, and no expanders. They wouldn't fit in the chassis :fap:

e: serious, regarding UPSes, I feel it should be a mandatory addition to the cost of your NAS setup. If you can't afford a NAS + UPS, you can't afford the setup. CyberPower's are obnoxiously cheap from Amazon, and they have a decent Linux daemon now. "Dead" APC units on Craigslist 95% of the time simply need new batteries, which you can purchase from refurbUPS.com or similar. I bought a "dead" pure sinewave APC SmartUPS 1500 with rack hardware for $50, threw new batteries in it (~$100) and it's a loving beast.

movax fucked around with this message at 19:15 on Sep 10, 2012

# ? Sep 10, 2012 19:10

UndyingShadow: May 15, 2006; You're looking ESPECIALLY shadowy this evening, Sir

movax posted:

e: serious, regarding UPSes, I feel it should be a mandatory addition to the cost of your NAS setup. If you can't afford a NAS + UPS, you can't afford the setup. CyberPower's are obnoxiously cheap from Amazon, and they have a decent Linux daemon now. "Dead" APC units on Craigslist 95% of the time simply need new batteries, which you can purchase from refurbUPS.com or similar. I bought a "dead" pure sinewave APC SmartUPS 1500 with rack hardware for $50, threw new batteries in it (~$100) and it's a loving beast.

Ohh I bought one, but the cyberpower UPS absolutely refused to be seen by FreeNAS, so it's there, but I have no management. I guess I'll have to spring for a real APC UPS one of these days, but it's not always so easy as getting a UPS

movax posted:

It might be due to my laziness in not wanting to gently caress with my existing setup, but I feel like bit of a dinosaur still running a pure Solaris ZFS setup in the Norco RPC-4020. The thing hasn't choked once in almost...three years now I think, and it's ridiculous the amount of "professional/enterprise"-like features I have for basically free.

When it comes to storage, I've learned to never mess with a setup that is working and isn't in impending danger of data loss.

# ? Sep 10, 2012 19:24

movax: Aug 30, 2008

UndyingShadow posted:

Ohh I bought one, but the cyberpower UPS absolutely refused to be seen by FreeNAS, so it's there, but I have no management. I guess I'll have to spring for a real APC UPS one of these days, but it's not always so easy as getting a UPS

Ah, that sucks. I wasn't sure if CP put out a BSD-version/had any BSD support for their software. Seems to work OK on *nix and of course Windows though.

There's always the comedy option of running some kind of VM just to run UPS management software though :v:

# ? Sep 10, 2012 19:35

Delta-Wye: Sep 29, 2005

I am ever-so-slowly moving to getting my NAS upgrade finished. At this point, it is clear I will be rebuilding the machine from scratch. New case and new power supply so far, along with 5x 2TB drives. My old motherboard only has four SATA ports, so I either need a new motherboard or one of the SAS cards discussed a few pages ago.

I looked into the cards, and they are pretty much all pci-express. My current motherboard has a PCI slot, so I am stuck replacing the motherboard too. I figure i might as well try and get something with more SATA ports and save myself the trouble and cost of the additional SAS card. My current system disk is a PATA disk, so that will need to replaced too!

This whole project has gotten much more expensive than just "replace case and harddisks" but I think it will be for the best. I would like to turn it into a hybrid HTPC/NAS and found a mini-itx motherboard with 5x SATA ports and HDMI out. I want to slap FreeBSD on it to use ZFS and probably some sort of media management software (I'll get that figured out down the road). The board in question is the Asus E45M1-I Deluxe which has every feature I need plus some, but software support may be spotty.

The video card, at the very least, will probably only work in VESA mode. The SATA ports are 6Gb/s, so they should work nicely. I like the passive cooling; the current motherboard's fans are so. damned. annoying. The case has some larger dimension fans that I'm hoping will push enough air to keep the board reasonably cool at a much lower volume. It's also low power, which is good in the long term. In theory, it includes the processor, enough SATA ports, usable GigE network port, video card, etc, so this and 8GB of RAM may be my last purchase for this project. One last expensive purchase, but it will be over with.

A few questions:

Does anyone have any personal experience with this board? I imagine the processor will be enough for ZFS, but doing ZFS/background tasks like sabnzbd/playing media all at the same time might be pushing it. Any thoughts?

There is no PATA connector and I'm planning on using all 5 SATA ports for the storage disks, meaning I need a solution for my boot medium. Am I stupid for considering something like a thumb drive and the USB 3.0 port for my boot partition? FreeBSD seems to support it, and I figure I can probably move most of the active stuff (logging, etc) to a partition on the storage drives if I'm worried about write cycles on a thumb drive. This used to be a concern, but I'm not sure how big of a deal it is anymore. Hell, with the cost of thumb drives I can easily keep a snapshot of the boot image around and just replace it every once in a while.

# ? Sep 10, 2012 19:50

DigitalMocking: Jun 8, 2010; Wine is constant proof that God loves us and loves to see us happy.
Benjamin Franklin

Longinus00 posted:

Were you using the RAID functionality of your Intel card or doing softraid? If you were using the on card RAID then you will probably need another RT3WB080 (maybe if you're lucky another card with the same chipset might work). If you were running any form of softraid then it doesn't matter how you connect the disks so long as the cards you connect them with don't try to "initialize" them.

Well poo poo.

# ? Sep 10, 2012 20:00

evil_bunnY: Apr 2, 2003

movax posted:

It might be due to my laziness in not wanting to gently caress with my existing setup, but I feel like bit of a dinosaur still running a pure Solaris ZFS setup in the Norco RPC-4020. The thing hasn't choked once in almost...three years now I think, and it's ridiculous the amount of "professional/enterprise"-like features I have for basically free.

If you can deal with a bit of downtime they can't be beat. I'd probably go with FreeBSD these days, but ZFS is pure awesomeness.

# ? Sep 10, 2012 20:32

movax: Aug 30, 2008

Delta-Wye posted:

I am ever-so-slowly moving to getting my NAS upgrade finished. At this point, it is clear I will be rebuilding the machine from scratch. New case and new power supply so far, along with 5x 2TB drives. My old motherboard only has four SATA ports, so I either need a new motherboard or one of the SAS cards discussed a few pages ago.

I looked into the cards, and they are pretty much all pci-express. My current motherboard has a PCI slot, so I am stuck replacing the motherboard too. I figure i might as well try and get something with more SATA ports and save myself the trouble and cost of the additional SAS card. My current system disk is a PATA disk, so that will need to replaced too!

This whole project has gotten much more expensive than just "replace case and harddisks" but I think it will be for the best. I would like to turn it into a hybrid HTPC/NAS and found a mini-itx motherboard with 5x SATA ports and HDMI out. I want to slap FreeBSD on it to use ZFS and probably some sort of media management software (I'll get that figured out down the road). The board in question is the Asus E45M1-I Deluxe which has every feature I need plus some, but software support may be spotty.

The video card, at the very least, will probably only work in VESA mode. The SATA ports are 6Gb/s, so they should work nicely. I like the passive cooling; the current motherboard's fans are so. damned. annoying. The case has some larger dimension fans that I'm hoping will push enough air to keep the board reasonably cool at a much lower volume. It's also low power, which is good in the long term. In theory, it includes the processor, enough SATA ports, usable GigE network port, video card, etc, so this and 8GB of RAM may be my last purchase for this project. One last expensive purchase, but it will be over with.

For my most recent upgrade to my box, I bought a used X58 motherboard, used i7-920 and 24GB of RAM all for a little less than $400, if I recall correctly. The RAM was the only part I got new. I'd say taking advantage of frequent upgraders and buying their used gear is the way to go.

# ? Sep 10, 2012 21:00

Longinus00: Dec 29, 2005; Ur-Quan

movax posted:

It might be due to my laziness in not wanting to gently caress with my existing setup, but I feel like bit of a dinosaur still running a pure Solaris ZFS setup in the Norco RPC-4020. The thing hasn't choked once in almost...three years now I think, and it's ridiculous the amount of "professional/enterprise"-like features I have for basically free.

Lots of those features are the exact same ones you'd get on professional/enterprise solutions. Yay for open source. Well not ZFS anymore, jokes on you. Although honestly who know what the hell Oracle is going to do about it, I have heard that all the important ZFS people bailed pretty quick. Hell, even the btrfs lead jumped ship to work at Fusion-io along with a few other prominent linux fs hackers.

Delta-Wye posted:

I am ever-so-slowly moving to getting my NAS upgrade finished. At this point, it is clear I will be rebuilding the machine from scratch. New case and new power supply so far, along with 5x 2TB drives. My old motherboard only has four SATA ports, so I either need a new motherboard or one of the SAS cards discussed a few pages ago.

I looked into the cards, and they are pretty much all pci-express. My current motherboard has a PCI slot, so I am stuck replacing the motherboard too. I figure i might as well try and get something with more SATA ports and save myself the trouble and cost of the additional SAS card. My current system disk is a PATA disk, so that will need to replaced too!

PCI has a max bandwidth (assuming 32bit @ 33Mhz) of around 133MB/s so don't stick more than 1 disks on their if you want max throughput. Obviously if your bottleneck is your processor then you can probably get away with more.

Delta-Wye posted:

This whole project has gotten much more expensive than just "replace case and harddisks" but I think it will be for the best. I would like to turn it into a hybrid HTPC/NAS and found a mini-itx motherboard with 5x SATA ports and HDMI out. I want to slap FreeBSD on it to use ZFS and probably some sort of media management software (I'll get that figured out down the road). The board in question is the Asus E45M1-I Deluxe which has every feature I need plus some, but software support may be spotty.

For a HTPC you *really* want hardware assisted decoding. Have you researched how well this is going to work in FreeBSD?

Delta-Wye posted:

The video card, at the very least, will probably only work in VESA mode. The SATA ports are 6Gb/s, so they should work nicely. I like the passive cooling; the current motherboard's fans are so. damned. annoying. The case has some larger dimension fans that I'm hoping will push enough air to keep the board reasonably cool at a much lower volume. It's also low power, which is good in the long term. In theory, it includes the processor, enough SATA ports, usable GigE network port, video card, etc, so this and 8GB of RAM may be my last purchase for this project. One last expensive purchase, but it will be over with.

A few questions:

Does anyone have any personal experience with this board? I imagine the processor will be enough for ZFS, but doing ZFS/background tasks like sabnzbd/playing media all at the same time might be pushing it. Any thoughts?

There is no PATA connector and I'm planning on using all 5 SATA ports for the storage disks, meaning I need a solution for my boot medium. Am I stupid for considering something like a thumb drive and the USB 3.0 port for my boot partition? FreeBSD seems to support it, and I figure I can probably move most of the active stuff (logging, etc) to a partition on the storage drives if I'm worried about write cycles on a thumb drive. This used to be a concern, but I'm not sure how big of a deal it is anymore. Hell, with the cost of thumb drives I can easily keep a snapshot of the boot image around and just replace it every once in a while.

I think some cheap thumbdrives might not have any wear leveling at all or other funny stuff. Definitely do some research on the drive you pick.

I think in general you might need to do more research into what you're planning to do. Particularly regarding how you're actually going to go about doing the HTPC side of things.

Longinus00 fucked around with this message at 21:33 on Sep 10, 2012

# ? Sep 10, 2012 21:28

Delta-Wye: Sep 29, 2005

movax posted:

For my most recent upgrade to my box, I bought a used X58 motherboard, used i7-920 and 24GB of RAM all for a little less than $400, if I recall correctly. The RAM was the only part I got new. I'd say taking advantage of frequent upgraders and buying their used gear is the way to go.

Not a bad idea, but my needs are so odd that it's hard. For instance, I need a shitload of SATA ports on a mini-itx board. It's not easy being green.

Longinus00 posted:

PCI has a max bandwidth (assuming 32bit @ 33Mhz) of around 133MB/s so don't stick more than 1 disks on their if you want max throughput. Obviously if your bottleneck is your processor then you can probably get away with more.

Er, poorly stated. I figured the SAS card would be the solution so I looked into it, then realized it wasn't compatible with my motherboard, not that I went hunting for a PCI SAS card in the first place. Mostly a problem because my dumb self didn't memorize the features of my NAS box, it's been really good to me in regards to just working since i put it together.

Longinus00 posted:

For a HTPC you *really* want hardware assisted decoding. Have you researched how well this is going to work in FreeBSD?

Probably not at all! The HTPC aspect is very much a "well, that would be nice," mostly because even if I get everything else working, FreeBSD won't support Silverlight so I can't run Netflix. I don't care, but the SO would be disappointed and it would be mostly useless for her.

Longinus00 posted:

I think some cheap thumbdrives might not have any wear leveling at all so you'll want to use a filesystem made for flash in those cases (e.g. linux's logfs).

I think in general you might need to do more research into what you're planning to do. Particularly regarding how you're actually going to go about doing the HTPC side of things.

I think I've got the NAS end of it pretty situated. I'm really just stuck on the usb3 boot device; good idea, bad idea, or merely doable?

# ? Sep 10, 2012 21:37

Jolan: Feb 5, 2007

error1 posted:

From what little googling i've done, it sounds like qnap uses linux and software mdraid, so timeouts should be configurable and i saw someone ssh in to troubleshoot a broken array with mdadm commands. Not to mention that RE4 drives were made for 24/7 raid use, your data should be pretty safe.

I wouldn't worry that much about the qnap ext4 filesystem not being checksummed, if you're using it for media storage it basically means you risk a glitched frame in a movie or half a second of garbled audio in an mp3 file each time you make a new copy of your entire collection. The risk is very low to the point where most people never even notice a single bit error.

ZFS (and btrfs) is a great solution to bit rot, but not an absolute must-have for regular users. It becomes hugely important if you're storing research or financial data though. Wouldn't it be fun if your harddrive changed the account number for a customer transaction, or added extra numbers in front of someones account balance?

Great, thanks for your answer. A Raid5 with 4x2TB WD RE4 drives in a QNAP TS412 it is, then.

Now I just have to get myself to pay the �1050+ price.

# ? Sep 10, 2012 21:54

Longinus00: Dec 29, 2005; Ur-Quan

Delta-Wye posted:

Not a bad idea, but my needs are so odd that it's hard. For instance, I need a shitload of SATA ports on a mini-itx board. It's not easy being green.

Er, poorly stated. I figured the SAS card would be the solution so I looked into it, then realized it wasn't compatible with my motherboard, not that I went hunting for a PCI SAS card in the first place. Mostly a problem because my dumb self didn't memorize the features of my NAS box, it's been really good to me in regards to just working since i put it together.
Probably not at all! The HTPC aspect is very much a "well, that would be nice," mostly because even if I get everything else working, FreeBSD won't support Silverlight so I can't run Netflix. I don't care, but the SO would be disappointed and it would be mostly useless for her.

I think I've got the NAS end of it pretty situated. I'm really just stuck on the usb3 boot device; good idea, bad idea, or merely doable?

The usb3 boot device shouldn't really be an issue so long as you make sure /var is somewhere else I guess. FreeNAS (or nas4free?) should support installing to usb sticks directly if you want to go that route. Since the board comes with builtin video, why not just buy a cheap 2/4 SATA pci-e card to increase your sata ports?

# ? Sep 10, 2012 22:15

FISHMANPET: Mar 3, 2007; Sweet 'N Sour
Can't
Melt
Steel Beams

Trying to build a NAS plus HTPC is a fools errand. A $100 Roku box can deal with basically every online streaming service ever. A $200 HTPC (E-350/E-450 based) will play all the content off your NAS, and be as quiet as a whisper.

# ? Sep 11, 2012 02:30

Moey: Oct 22, 2010; I LIKE TO MOVE IT

FISHMANPET posted:

A $100 Roku box can deal with basically every online streaming service ever. A $200 HTPC (E-350/E-450 based) will play all the content off your NAS, and be as quiet as a whisper.

My Raspberry Pi just came in the mail today. I am pretty excited to start testing these as xbmc front end units.

# ? Sep 11, 2012 03:48

Telex: Feb 11, 2003

FISHMANPET posted:

Trying to build a NAS plus HTPC is a fools errand. A $100 Roku box can deal with basically every online streaming service ever. A $200 HTPC (E-350/E-450 based) will play all the content off your NAS, and be as quiet as a whisper.

But a Roku can't play anything from your NAS, so that's a lovely suggestion.

Get the WDTV Live, or a Boxee. Skip the Roku because it's honestly a pile of poo poo compared to the WD, especially for the price.

# ? Sep 11, 2012 04:04

FISHMANPET: Mar 3, 2007; Sweet 'N Sour
Can't
Melt
Steel Beams

Telex posted:

But a Roku can't play anything from your NAS, so that's a lovely suggestion.

Get the WDTV Live, or a Boxee. Skip the Roku because it's honestly a pile of poo poo compared to the WD, especially for the price.

Hence also getting a seperate HTPC.

This really isn't the right place for it, but since you brought it up, what's wrong with the Roku? I discovered my fiancee has been paying for a Netflix subscription for months now so I might buy something for Netflix on the TV.

# ? Sep 11, 2012 04:11

Delta-Wye: Sep 29, 2005

FISHMANPET posted:

Trying to build a NAS plus HTPC is a fools errand. A $100 Roku box can deal with basically every online streaming service ever. A $200 HTPC (E-350/E-450 based) will play all the content off your NAS, and be as quiet as a whisper.

That may end up being what I do (if I do anything on the HTPC front) so it's not a big deal if I can't coerce the NAS into doing it the way I wish; a barebones HTPC would be pretty cheap, especially if I go all out with removing unnecessary features like harddisks, etc. It's just the NAS is already on all the time, and at the moment will probably be parked next to the TV anyways so double-duty is ideal. However, it is pretty clear that ZFS and netflix are not going to run on the same machine without something relatively insane like a VM solution.

# ? Sep 11, 2012 04:41

Telex: Feb 11, 2003

FISHMANPET posted:

Hence also getting a seperate HTPC.

This really isn't the right place for it, but since you brought it up, what's wrong with the Roku? I discovered my fiancee has been paying for a Netflix subscription for months now so I might buy something for Netflix on the TV.

The WD TV does both in one for $100. Streaming from internet stuff (Netflix, Spofity, the major streaming sites and all that) as well as play things from your NAS. The sorting is a little crappy in that the only options are alphabetical and not by date, but that ain't too bad.

The Roku doesn't talk to your NAS and for the most part all the things Roku does that the WD doesn't, don't overcome the lack of NAS playback and not having Spotify for me.

For someone without their own media storage the Roku is probably fine, although I let someone have mine and they don't even use it and they're playing from their DVD player that has netflix built-in instead. The Amazon prime streaming sorta sucks on it and looks way worse than Netflix. It's just not an awesome device, although it presents itself in a very slick manner.

# ? Sep 11, 2012 05:53

necrobobsledder: Mar 21, 2005; Lay down your soul to the gods rock 'n roll; Nap Ghost

I might be the only weirdo here with a NAS and HTPC in one. I added a GT520 video card to my Microserver N36L and it's working super duper fine for me running sabnzbdplus, Couchpotato Server, and Sickbeard. Downloads are decompressed on the SSD and it runs fine while I'm watching a 1080P movie. I can run a massive backup job between other machines and not worry about interrupting my movie either. All my media that I suspect I'd have great difficulty re-ripping are backed up to a ZFS-based NAS with more storage and I can just fire that up once in a while and scrub.

I think there's some crazy people out there running XBMC in a VM on top of ESXi with direct access to the GPU and a Solaris VM managing the storage for the media, but this is kinda backwards to me because Solaris is pretty readily setup for an iSCSI target and NFS setup so it's like the egg evolved a knife-edge and castrated the chicken. A box that can do all that though is probably not going to be very small, likely runs hot, nor concealable in a media center is my concern here.

# ? Sep 11, 2012 06:38

Telex: Feb 11, 2003

necrobobsledder posted:

I think there's some crazy people out there running XBMC in a VM on top of ESXi with direct access to the GPU and a Solaris VM managing the storage for the media, but this is kinda backwards to me because Solaris is pretty readily setup for an iSCSI target and NFS setup so it's like the egg evolved a knife-edge and castrated the chicken. A box that can do all that though is probably not going to be very small, likely runs hot, nor concealable in a media center is my concern here.

I didn't do XBMC, but I tried doing that sort of setup with a Plex server and FreeNAS running off an ESXi to stream to an Apple TV.

Kinda sucked when two people were doing it at the same time and a download was un-rar'ing with SABnzbd or SickBeard was doing its searches, I think mostly because I only had 8GB of ram in the machine.

It's tempting to try and re-do it with like a quad-core setup and a lot of ram but honestly the effort needed to make that all work seems to be stupid compared to just getting one of the set-top boxes to handle it all for cheap.

# ? Sep 11, 2012 18:54

Matt Zerella: Oct 7, 2002; Norris'es are back baby. It's good again. Awoouu (fox Howl)

Telex posted:

I didn't do XBMC, but I tried doing that sort of setup with a Plex server and FreeNAS running off an ESXi to stream to an Apple TV.

Kinda sucked when two people were doing it at the same time and a download was un-rar'ing with SABnzbd or SickBeard was doing its searches, I think mostly because I only had 8GB of ram in the machine.

It's tempting to try and re-do it with like a quad-core setup and a lot of ram but honestly the effort needed to make that all work seems to be stupid compared to just getting one of the set-top boxes to handle it all for cheap.

Yeah, my jailbroken aTV handles everything I throw at it like a champ with XBMC on it.

# ? Sep 11, 2012 19:00

Longinus00: Dec 29, 2005; Ur-Quan

Telex posted:

I didn't do XBMC, but I tried doing that sort of setup with a Plex server and FreeNAS running off an ESXi to stream to an Apple TV.

Kinda sucked when two people were doing it at the same time and a download was un-rar'ing with SABnzbd or SickBeard was doing its searches, I think mostly because I only had 8GB of ram in the machine.

It's tempting to try and re-do it with like a quad-core setup and a lot of ram but honestly the effort needed to make that all work seems to be stupid compared to just getting one of the set-top boxes to handle it all for cheap.

If you were using a microserver then it was almost certainly not a ram issue. In the future you should consider ionice'ing the SABnzbd and sickbeard (does bsd even support this?)

# ? Sep 11, 2012 20:45

caberham: Mar 18, 2009; by Smythe; Grimey Drawer

I'm a bit confused in how much work my NAS can do before I have to delegate tasks to another machine/server? I have a synology 2212+ and am planning on getting sickbeard and usenet for it to run in the background. Can I handle setting up a Open VPN server at the same time? I do go to China from time to time, should I just buy a dedicated VPN service for facebook/youtube stuff or is VPN tunnel to my home computer suffice?

I have looked at the website Amahi and it sounds very interesting. The iCal/outlook/calendar features sounds very tempting but I have trouble working with google contacts and apple iCal working 100% seamlessly on my own. There's always some minor set back with the custom naming fields and how each architecture handles non English characters. Hopefully Amahi just works.

Sorry if this is a bit of a tangent, is there a media program which allows me to not just stream but sync different iphone/ipad/android devices with media instead of just streaming? I can mount my itunes media folder into my NAS but itunes is more of a single user/device program. For instance, I might have songs/movies in my laptop and phone for different users but play counts and play lists get totaled up and aggregated into the server. Kind of like how XBMC has a MySQL database for different users accessing the central media hub. But with sync functionality for pictures/movies/songs/etc. Without manually clicking and dragging!

I just want something platform neutral that works well with all my devices because Apple home sharing is really really limited

I was cross posing int the iTunes thread and found this guide http://www.macworld.com/article/1163242/organize_and_play_your_media_from_a_nas.html but it seems more of a mish mash band aid function :smith:

Once I have the software side of things figured out, I can't wait to blow money on a 5Bay JOBD NAS full of anime :downsrim:

Actually, I really don't care whether I use RAIDZ/ZFS or whatever file system. I have a feeling the server will regardless crash and burn. At least there's flickr for my photos when things go wrong.

caberham fucked around with this message at 21:33 on Sep 11, 2012

# ? Sep 11, 2012 21:24

necrobobsledder: Mar 21, 2005; Lay down your soul to the gods rock 'n roll; Nap Ghost

Telex posted:

Kinda sucked when two people were doing it at the same time and a download was un-rar'ing with SABnzbd or SickBeard was doing its searches, I think mostly because I only had 8GB of ram in the machine.

Did you look at the ESXi host's performance graphs to determine if you were having memory problems? I'm running a Microserver with less RAM and can serve out and watch & share media plenty while downloading and decompressing stuff.

caberham posted:

Sorry if this is a bit of a tangent, is there a media program which allows me to not just stream but sync different iphone/ipad/android devices with media instead of just streaming?

That's something similar to what Media Rover (specific for iTunes) sorta tries to do, but syncing and registering media across multiple devices is a local media problem in need of a Holy Grail that most companies just aren't doing, but I suppose the one company that could pull it off would be Dropbox. It's a pretty complex problem as you've experienced, and the fact of the matter is that it's monumentally easier to transcode and stream instead of manage separate libraries and profiles across devices.

# ? Sep 11, 2012 22:08

caberham: Mar 18, 2009; by Smythe; Grimey Drawer

I'm not in US or Europe so I don't have cool internet services like Pandora, last fm, hulu, spotify, or google music. Itunes match just came out a few months ago and last month people can finally buy legal music online. Even though we have ultra fast and cheap fibre optic internet connections like 1000MB line for USD $36/month :qq:

necrobobsledder posted:

Media Rover, Dropbox, or just stream

Thanks for the media rover link. I think the last update to its blog was 1 year ago but hopefully support is still going strong and will eventually work with android. I guess drop box works in a brute force way but I'm kind of looking for a more elegant solution of having 2 lists, one for personal/device playback count and another central master repository.

Streaming does sound good but it does have a few quirks. It uses a lot more battery life than regular playback, eats up internet bandwidth (roaming is expensive) and doesn't work when you are really mobile or a "on the road" situation. Everyday battery life doesn't matter so much because there's always some car charger or computer/outlet at work to plug into. The problem is that being online can be spotty at times. Streaming with the ability to retain and sync devices on the go like AirVideo would be awesome. Or have some sort of master database to keep track of activities after streaming.

Anyways, back to more NAS talk. Maybe in a few years this point of contention would be moot.

# ? Sep 11, 2012 23:30

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

caberham posted:

Streaming does sound good but it does have a few quirks. It uses a lot more battery life than regular playback, eats up internet bandwidth (roaming is expensive) and doesn't work when you are really mobile or a "on the road" situation. Everyday battery life doesn't matter so much because there's always some car charger or computer/outlet at work to plug into. The problem is that being online can be spotty at times. Streaming with the ability to retain and sync devices on the go like AirVideo would be awesome. Or have some sort of master database to keep track of activities after streaming.

I keep all my music on a server that runs subsonic and listen to it on my phone using the iSub application. After you play a song it caches it locally on the phone, so you don't even need to have an internet connection the next time you want to play it.

# ? Sep 11, 2012 23:40

FCKGW: May 21, 2006

For anyone still on the fence about Crashplan:

Crashplan is running a special for one free year of their online Crashplan Central backup service which usually runs $5/mo. This deal is good for one computer only, but if you have multiple PCs in the house backup to that one computer, you can have everything online. I have this installed on a Windows Home Server box that backs up my important docs and pictures.

I've used Crashplan for a little over a year now and it's great. You can setup PC-to-PC backups on your LAN and also setup to external or online back. Online backups are unlimited in storage and upload/download speed. If you have a large amount of files you can even mail them a harddrive to pre-seed your backups without waiting weeks to do it. It's also cross-platform and they have clients for Windows, Mac and Linux.

This deal is supposed to be for Carbonite users, but it works on all new or currently free plan customers. Crashplan has the right to revoke the deal at any time blah blah blah.

https://www.crashplan.com/carboniteswitcher/

Must be a Carbonite customer
This offer is available in the continental US only.
This offer is not available to current or previous CrashPlan customers (users who have paid for licenses.)
CrashPlan Free or users still in their 30-day trial period are eligible.
Only one Free 1-year 1-computer plan or $50 Family Plan per customer. May not be combined with any other offers.

quote:

We're so sure you'll like CrashPlan better than Carbonite, we're giving switchers a free year!
Yep. One computer, backing up an unlimited amount of personal files with no restrictions, costs nothing for one full year. Backup doesn't get any better than that! And if you've got multiple computers, our Family Plan is only $50 for the first year!

CrashPlan backup is better because you get up-to-the-minute protection every time you change a file. You can restore deleted files even after 30 days and restore files that you backed up to friends and family in addition to what you backed up online. Unlike Carbonite, CrashPlan backup doesn't get slower as your backups get bigger.

Also, the Crashplan Family plan (2-10 machines) is only $50/yr with that link instead of the normal $120/yr.

FCKGW fucked around with this message at 03:51 on Sep 12, 2012

# ? Sep 12, 2012 03:49

Astro7x: Aug 4, 2004; Thinks It's All Real

I started this discussion in the Video Editing thread since I work for a small post house, but I guess it is more appropriate for this thread.

What would be a good way to back up a 16TB server offsite for a small business? My boss is starting to get concerned about liability of losing data in case of a catastrophic event, and even though we are backed up internally, if the building were to burn down we'd be kind of screwed. This isn't stuff that is archived, I'm talking about 16TB of data we need access on almost a daily basis.

I'm assuming sites like Crashplan and Carbonite are too expensive for that size, or too slow to transfer so much data over the net. Of the few I knew of and looked into, I was surprised to see that Dropbox Teams was the most affordable. But yeah, was hoping SHSC had something proved you knew of to recommend.

# ? Sep 12, 2012 06:10

Adbot: ADBOT LOVES YOU

# ? May 31, 2024 16:56

DrDork: Dec 29, 2003; commanding officer of the Army of Dorkness

Is this data that is continuously updated and time critical? Eg, banking data or something where you literally need backups running continuously. Or is this more like live/production data that you want backed up regularly, but you could stand to wait to batch process it at the end of the day/week, and if the place burned down it wouldn't hurt too much to lose a day or two of data?

Also, why would you need to access it on a daily basis if you're already backing it up internally--you've already got your working set, your backup set, and now you're looking at a third copy; I can't for the life of me think of why you'd actually need to read from said third copy unless both your first and second set died.

Additionally, how much new/changed data are you producing daily? I understand that 16TB is a lot of data to push over the net, but there's a world of difference between "16TB total collected over years with only a few gigs new/changed a day" and "16TB almost all of which rotates constantly."

# ? Sep 12, 2012 07:55

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›848 »