Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Gism0
Mar 20, 2003

huuuh?

Civil posted:

Some AMD southbridge thing. Dunno. I just ran across it while reading. If it's bullshit, I'd be happy, but I'm not going to be the one to find out.

http://forums.overclockers.com.au/showthread.php?t=958208&page=316

Others speculate that if you just use a more modern SATA PCI-E card, you'll be able to use the entire thing. 2TB drives still seem to be the price/capacity king, so that's what I'm sticking with for now.

Seems 3TB drives are getting cheaper though http://edwardbetts.com/price_per_tb/internal_hdd/

Adbot
ADBOT LOVES YOU

Bonobos
Jan 26, 2004

bloodynose posted:

Another N40L question here:

I've got mine coming in on Monday, with 4x 2TB Drives (Samsung F4s) and a flash drive for the OS. My intent is to run SABnzbd, SickBeard, Couchpotato, and rTorrent on top of your typical NAS fileserving. I'm familiar with unix-based stuff, and the general consensus seems to be that ZFS/RAID-Z is super legit.

So the question: Is my best route, then, from a usability, speed, and safety standpoint (assume I'm also taking snapshots here), to go with FreeNAS setup, and setup RAID-Z1 on one big pool with the 4 drives? Anyone want to throw a better/different option in the hat before I roll this all out on Monday?

FreeNAS 8 won't let you do torrent / sick beard/ sabnzbd stuff, and FreeNAS 7 won't support ZFS/ is outdated.

Fangs404
Dec 20, 2004

I time bomb.

Bonobos posted:

FreeNAS 8 won't let you do torrent / sick beard/ sabnzbd stuff, and FreeNAS 7 won't support ZFS/ is outdated.

FreeNAS 8 doesn't...yet. FreeNAS 8.2 allows plugins. It's in beta status right now, so you can try it out. There are already several plugins available, one of which being Transmission (torrents).

On another note, FreeNAS 8.0.4-p1 was released 2 days ago. It addresses a critical Samba vulnerability.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Bonobos posted:

FreeNAS 8 won't let you do torrent / sick beard/ sabnzbd stuff, and FreeNAS 7 won't support ZFS/ is outdated.
It will certainly let you do torrent stuff. 8.x (including the stable 8.0.x) can use Transmission via a simple pkg_add command and some setup, and 8.2-Multimedia comes with it built-in. Not the most advanced torrent application, but it works.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb
How difficult is it to set up some of the basic FreeNAS functionality with just plain FreeBSD?

Gism0
Mar 20, 2003

huuuh?
I had no problems installing SABnzbd, sickbeard, couchpotato, etc on my freeNAS 8 box. I haven't tried a torrent client yet, however.

Bonobos
Jan 26, 2004

DrDork posted:

It will certainly let you do torrent stuff. 8.x (including the stable 8.0.x) can use Transmission via a simple pkg_add command and some setup, and 8.2-Multimedia comes with it built-in. Not the most advanced torrent application, but it works.

I stand corrected. I am assuming these can be installed easily without the CLI?

Guess I will check them out to find out.

Does anyone know how does the torrenting aspect affect ZFS? Will the tiny pieces of files being written all over the place affect its performance/function ? I would be concerned especially since ZFS has no defragging tools.

adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer

Bonobos posted:

I would be concerned especially since ZFS has no defragging tools.
its a copy on write file system, there is no point to defragging.

Longinus00
Dec 29, 2005
Ur-Quan

adorai posted:

its a copy on write file system, there is no point to defragging.

Defragging is important even in a COW filesystem because in place file modifications can potentially scatter a file all over a device.

Matt Zerella
Oct 7, 2002

Norris'es are back baby. It's good again. Awoouu (fox Howl)

bloodynose posted:

Another N40L question here:

I've got mine coming in on Monday, with 4x 2TB Drives (Samsung F4s) and a flash drive for the OS. My intent is to run SABnzbd, SickBeard, Couchpotato, and rTorrent on top of your typical NAS fileserving. I'm familiar with unix-based stuff, and the general consensus seems to be that ZFS/RAID-Z is super legit.

So the question: Is my best route, then, from a usability, speed, and safety standpoint (assume I'm also taking snapshots here), to go with FreeNAS setup, and setup RAID-Z1 on one big pool with the 4 drives? Anyone want to throw a better/different option in the hat before I roll this all out on Monday?

If you're familiar with Linux/unix you can roll with vanilla FreeBSD. Through the ports directory you can have sbanzbd/rtorrent/screen up and running very quickly and the snapshots and scrubs can be automated through cron. Setting up a zpool is very easy. You lose the pretty web interface but if you use your server for more than just regular file serving you won't miss it (plus you can run fbsd 9 which has a newer version of zfs).

Muslim Wookie
Jul 6, 2005

Longinus00 posted:

Defragging is important even in a COW filesystem because in place file modifications can potentially scatter a file all over a device.

LOL tell us more.

You'll have no issues with torrents on ZFS.

Bonobos
Jan 26, 2004

marketingman posted:

LOL tell us more.

You'll have no issues with torrents on ZFS.

Awesome, good to know as I was going to set up a separate drive just for torrents. Now that I know they can go to the same place as the rest of my data without affecting performance I won't bother.

Matt Zerella
Oct 7, 2002

Norris'es are back baby. It's good again. Awoouu (fox Howl)
As someone who used to sperg about zfs performance, I just throw everything at it now and stopped caring. Rtorrent, sabnzbd, time machine for a few laptops, xbmc share streaming. It's fine. Really, it is.

I got my biggest performance boost switching to a 5ghz dual band router in my apt complex where there are 100+ FiOS routers on 2.4 ghz.

Residency Evil
Jul 28, 2003

4/5 godo... Schumi
Is there an easy way to transition an ext4 formated partition over to FreeNAS 8? If I just boot my N40L from a USB stick will it automatically see everything?

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Bonobos posted:

I stand corrected. I am assuming these can be installed easily without the CLI?
The plug-in system of FreeNAS 8.2 is supposed to let you do that without CLI use, but it's still very much in beta, so I haven't tried. I know setting up Transmission for torrenting takes all of 5 minutes and there are step-by-step guides for doing it. Using the CLI won't kill you, I promise.

evil_bunnY
Apr 2, 2003

Longinus00 posted:

in place file modifications can potentially scatter a file all over a device.
Yes, it's copy on write. Leave a bit of free space and never worry about it.

Longinus00
Dec 29, 2005
Ur-Quan

marketingman posted:

LOL tell us more.

You'll have no issues with torrents on ZFS.

Who said there'll be any issue with torrents?

evil_bunnY posted:

Yes, it's copy on write. Leave a bit of free space and never worry about it.

It's definitely not in the "never worry about it" level but it's unlikely to be an issue for home use. ZFS has extra considerations because of the way the it's implemented (e.g. running out of specific block sizes which is why stuff like gang blocks is necessary).

Longinus00 fucked around with this message at 18:27 on Apr 15, 2012

Bonobos
Jan 26, 2004

DrDork posted:

The plug-in system of FreeNAS 8.2 is supposed to let you do that without CLI use, but it's still very much in beta, so I haven't tried. I know setting up Transmission for torrenting takes all of 5 minutes and there are step-by-step guides for doing it. Using the CLI won't kill you, I promise.

I'll definitely try, just that my unix / linux skills are severely lacking and am worried I'll bork the server. Hopefully those 8th grade MS-DOS skills will come in handy.

wang souffle
Apr 26, 2002
Can anyone convince me what to do with 8x 2TB drives in a ZFS setup? I currently have 5 of the drives in a RAIDZ1 setup and need a migration plan for existing data once I get 3 more.

1) 2x 4-drive RAIDZ1 groups in one pool.
2) 8-drive RAIDZ2 zpool.

Exactly how much more reliable is option 2 here? I'm considering option 1 as it seems possible to migrate the existing data without needing more drives.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Bonobos posted:

I'll definitely try, just that my unix / linux skills are severely lacking and am worried I'll bork the server. Hopefully those 8th grade MS-DOS skills will come in handy.
This guide is pretty step-by-step: http://sourceforge.net/apps/phpbb/freenas/viewtopic.php?f=104&t=10959

I'd suggest using nano instead of vi for the editing, though.

e; if pkg_add is throwing some cannot connect errors, make sure you assigned a default gateway and nameserver in FreeNAS's web setup. If you don't, while FreeNAS will still happily work on your internal network, it won't be able to connect out to the rest of the internet properly.

DrDork fucked around with this message at 21:02 on Apr 15, 2012

TLG James
Jun 5, 2000

Questing ain't easy
How does the VPN server work on synology's stuff? If I set it up before a trip and I'm in an area with filtered internet, can I use it to bounce off my internet at home?

BlankSystemDaemon
Mar 13, 2009



wang souffle posted:

Can anyone convince me what to do with 8x 2TB drives in a ZFS setup? I currently have 5 of the drives in a RAIDZ1 setup and need a migration plan for existing data once I get 3 more.

1) 2x 4-drive RAIDZ1 groups in one pool.
2) 8-drive RAIDZ2 zpool.

Exactly how much more reliable is option 2 here? I'm considering option 1 as it seems possible to migrate the existing data without needing more drives.
You can just add the three drives in a zraid1 vdev to your current pool - or throw in another drive and do zraid2 if data security means that much to you (remember, raid isn't a backup solution).

Tamba
Apr 5, 2010

wang souffle posted:

Can anyone convince me what to do with 8x 2TB drives in a ZFS setup? I currently have 5 of the drives in a RAIDZ1 setup and need a migration plan for existing data once I get 3 more.

1) 2x 4-drive RAIDZ1 groups in one pool.
2) 8-drive RAIDZ2 zpool.

Exactly how much more reliable is option 2 here? I'm considering option 1 as it seems possible to migrate the existing data without needing more drives.

In the first case, you lose half your data if two drives in the same RAIDZ1 die, in the second case, you can lose any two drives and still be fine (but then you lose everything if a third drives dies before you can rebuild)

adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer

Longinus00 posted:

It's definitely not in the "never worry about it" level but it's unlikely to be an issue for home use. ZFS has extra considerations because of the way the it's implemented (e.g. running out of specific block sizes which is why stuff like gang blocks is necessary).
Like he said, leave some free space and you don't have to worry about it. This is true for any COW filesystem.

Longinus00
Dec 29, 2005
Ur-Quan

adorai posted:

Like he said, leave some free space and you don't have to worry about it. This is true for any COW filesystem.

No, it's true for just about any filesystem. It's also a far way away from "its a copy on write file system, there is no point to defragging."

NickPancakes
Oct 27, 2004

Damnit, somebody get me a tissue.

LmaoTheKid posted:

If you're familiar with Linux/unix you can roll with vanilla FreeBSD. Through the ports directory you can have sbanzbd/rtorrent/screen up and running very quickly and the snapshots and scrubs can be automated through cron. Setting up a zpool is very easy. You lose the pretty web interface but if you use your server for more than just regular file serving you won't miss it (plus you can run fbsd 9 which has a newer version of zfs).

Cool, this is looking like the route I'm going to go. Since I have never actually worked with FreeNAS, does anyone want to recommend anything (scripts/special configuration included) that comes preinstalled with FreeNAS that would be handy to have installed on a FreeBSD-based box?

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

bloodynose posted:

Cool, this is looking like the route I'm going to go. Since I have never actually worked with FreeNAS, does anyone want to recommend anything (scripts/special configuration included) that comes preinstalled with FreeNAS that would be handy to have installed on a FreeBSD-based box?
FreeNAS 8.0.x really doesn't come with much pre-installed. The thing that gets everyone to use it is the webGUI front-end. If you feel comfortable with setting up your own RAIDZ pools and Samba/CIFS shares, you've got 90% of what most people use FreeNAS for already.

wang souffle
Apr 26, 2002

wang souffle posted:

Can anyone convince me what to do with 8x 2TB drives in a ZFS setup? I currently have 5 of the drives in a RAIDZ1 setup and need a migration plan for existing data once I get 3 more.

1) 2x 4-drive RAIDZ1 groups in one pool.
2) 8-drive RAIDZ2 zpool.

Exactly how much more reliable is option 2 here? I'm considering option 1 as it seems possible to migrate the existing data without needing more drives.
I found something with hard numbers. The number of disks is kind of crazy, but this puts a good comparison between RAID50 (1) and RAID6 (2) in my example above. Looks like RAIDZ2 is the way to go.

http://www.servethehome.com/raid-reliability-failure-anthology-part-1-primer/

wang souffle fucked around with this message at 01:52 on Apr 16, 2012

evil_bunnY
Apr 2, 2003

The failure rate of disk drives at 5 years is 10%+. If you're using big arrays and identical drives (and thus similar failure patterns), you're living on borrowed time without Raid6 or better (ideally something with end-to-end data integrity like RaidZ2 on ZFS).

RAID 5


RAID 6


This is on a big (20 drives) array, but you get the point.

Also shitton of media errors simply go undetected in traditional raid setups.

I don't even want to talk about non-parity raid levels.

evil_bunnY fucked around with this message at 12:37 on Apr 16, 2012

titaniumone
Jun 10, 2001

wang souffle posted:

Can anyone convince me what to do with 8x 2TB drives in a ZFS setup? I currently have 5 of the drives in a RAIDZ1 setup and need a migration plan for existing data once I get 3 more.

1) 2x 4-drive RAIDZ1 groups in one pool.
2) 8-drive RAIDZ2 zpool.

Exactly how much more reliable is option 2 here? I'm considering option 1 as it seems possible to migrate the existing data without needing more drives.

I had an 8 drive RAIDZ1 pool for about 2 years. It used WD Green drives and had performance issues, and I had 4 drives die at different times. I eventually bought 8 WD Black drives and replaced each drive one by one, rebuilding the array 8 times in a row.

I have never had an error or lost any data. ZFS is great.

IT Guy
Jan 12, 2010

You people drink like you don't want to live!
I have a question regarding ZFS snapshots.

I've scheduled the snapshots to take place at midnight every day and stay alive for 1 month.

If I look at the snapshots it will only allow me to revert to the latest one.

Say I wanted to revert 5 days ago, how does this work exactly?

IT Guy fucked around with this message at 19:49 on Apr 16, 2012

DEAD MAN'S SHOE
Nov 23, 2003

We will become evil and the stars will come alive

evil_bunnY posted:

If you're using big arrays and identical drives (and thus similar failure patterns), you're living on borrowed time

Please explain what you mean by failure pattern; do you mean to say that drives of the same model will fail at the same time?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

evil_bunnY posted:

The failure rate of disk drives at 5 years is 10%+. If you're using big arrays and identical drives (and thus similar failure patterns), you're living on borrowed time without Raid6 or better (ideally something with end-to-end data integrity like RaidZ2 on ZFS).

RAID 5


RAID 6


This is on a big (20 drives) array, but you get the point.

Also shitton of media errors simply go undetected in traditional raid setups.

I don't even want to talk about non-parity raid levels.

Your graph says 1% and you say 10%?

chizad
Jul 9, 2001

'Cus we find ourselves in the same old mess
Singin' drunken lullabies

Fangs404 posted:

FreeNAS 8 doesn't...yet. FreeNAS 8.2 allows plugins. It's in beta status right now, so you can try it out. There are already several plugins available, one of which being Transmission (torrents).

On another note, FreeNAS 8.0.4-p1 was released 2 days ago. It addresses a critical Samba vulnerability.

Damnit, my timing sucks. I'll have to double check when I get home, but I'm almost positive I upgraded to 8.0.4 like a day or two before 8.0.4-p1 came out.

adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer

Thermopyle posted:

Your graph says 1% and you say 10%?
That's data loss percentage. With a drive failure rate (at 5 years) of 10%, there is a 1% chance you will have a double disk failure at 5 years causing complete data loss.

wang souffle
Apr 26, 2002
Honestly, what is everyone using for backup of large amounts of data (10TB+) these days? Is tape a reasonable option for home use, or are external hard drives the only viable solution?

Galler
Jan 28, 2008


DEAD MAN'S SHOE posted:

Please explain what you mean by failure pattern; do you mean to say that drives of the same model will fail at the same time?

Two drives of the same type made during the same production run with the same usage pattern in the same conditions (ex. in the same RAID array) are (possibly) more likely to fail at roughly the same time.

So ideally you would want to have all your drives come from a different production runs to minimize the chance they will fail at approximately the same time (during rebuild).

Of course this isn't all that practical and I don't think it's something all that many people do but if you want to min/max your array it's something you can do I guess.

DEAD MAN'S SHOE
Nov 23, 2003

We will become evil and the stars will come alive
This sounds like bullshit and guesstimates until I see some empirical evidence from a significantly large sample, for example http://research.google.com/archive/disk_failures.pdf

Fangs404
Dec 20, 2004

I time bomb.

wang souffle posted:

Honestly, what is everyone using for backup of large amounts of data (10TB+) these days? Is tape a reasonable option for home use, or are external hard drives the only viable solution?

Do you really need to back all of that up? What I do is maintain an internal drive on my desktop where I store the important stuff, and I have CrashPlan back up that poo poo to the cloud. My NAS stores another copy of that stuff and also all my perishable items.

Adbot
ADBOT LOVES YOU

evil_bunnY
Apr 2, 2003

wang souffle posted:

Honestly, what is everyone using for backup of large amounts of data (10TB+) these days?
MD1000 filled with 2TB drives.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply