Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
IuniusBrutus
Jul 24, 2010

IOwnCalculus posted:

How much storage do you actually need?

Also, ECC-chat: i3 version of that Thinkserver with 4GB ECC is now only $200 at Newegg according to Slickdeals after a rebate and a coupon code.

Two or three terabytes would be adequate.

Edit: ugh. I'm stupidly tempted to buy that Lenovo and two 3TB red's.

IuniusBrutus fucked around with this message at 18:50 on Oct 10, 2014

Adbot
ADBOT LOVES YOU

IOwnCalculus
Apr 2, 2003





IuniusBrutus posted:

Two or three terabytes would be adequate.

Edit: ugh. I'm stupidly tempted to buy that Lenovo and two 3TB red's.

That would probably be the cheapest way into the roll-your-own route that still checks all of the boxes, but if 3TB is all you need and you already have other backups of your important poo poo, you'd probably be just as happy with any sort of commodity NAS instead.

SamDabbers
May 26, 2003



Yeah the TS140 is not hotswap, but you can put 4 drives in it (three 3.5" mounting points + one 5.25" to 3.5" adapter). They're good little machines; I've built a couple of them for clients.

suddenlyissoon
Feb 17, 2002

Don't be sad that I am gone.
Just said gently caress it and bought the 1513+. I don't need one more system to devote my time to like I would with a homebuilt NAS.

The pain of buying 2 wd red 4tb drives and knowing they're "only" being used as backups in SHR2 is awful though. ALL THAT SPACE, LOST!

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

suddenlyissoon posted:

Just said gently caress it and bought the 1513+. I don't need one more system to devote my time to like I would with a homebuilt NAS.
FreeNAS seems pretty maintenance free. It has a bunch of idiot bugs in relation to Samba with freshly created user accounts, but that's it.

Methylethylaldehyde
Oct 23, 2004

BAKA BAKA

Combat Pretzel posted:

In case of ZFS, it'll cache your data indefinitely, if it doesn't get flushed out by newer IO. If you want guaranteed correctness, ECC would be recommended, too. Writes get cached 30 seconds by default, altho FreeNAS has this tweaked to five seconds. Not sure about other distros.

I set up my new ZFS server with ECC ram specifically because of that. That and I wanted an ipmi card on it so I wouldn't have to go out to the garage and dig out the keyboard and monitor to argue with the thing.

Then I set up what is possibly the most Rube Goldberg storage setup possible.

ZFS raidz2 pools are carved up into ZFS volumes, then passed to the COMSTAR system and published as iSCSI targets.
A virtualized file server on another server picks up those iSCSI LUNs, and publishes them out as SMB shares to my home network.
A LTO tape deck is mounted to the Hyper-V host, and published via some wacky Tape Redirector server to a separate DPM 2012 VM, which periodically writes the contents of the server to tape.

Methylethylaldehyde fucked around with this message at 04:04 on Oct 11, 2014

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Methylethylaldehyde posted:

ZFS raidz2 pools are carved up into ZFS volumes, then passed to the COMSTAR system and published as iSCSI targets.
A virtualized file server on another server picks up those iSCSI LUNs, and publishes them out as SMB shares to my home network.
What functionality are you getting out of this that simply using SMB directly off your ZFS server wouldn't meet?

Methylethylaldehyde
Oct 23, 2004

BAKA BAKA

DrDork posted:

What functionality are you getting out of this that simply using SMB directly off your ZFS server wouldn't meet?

100% bullshit-less integration with my windows domain, and the ability for DPM to shadow volume snapshot the drives for tape backup.

Megaman
May 8, 2004
I didn't read the thread BUT...

Combat Pretzel posted:

FreeNAS seems pretty maintenance free. It has a bunch of idiot bugs in relation to Samba with freshly created user accounts, but that's it.

FreeNAS is so maintenance free it's not even funny, and it's so easy to set up. It's also ridiculously hassle free to upgrade. Just shut down the machine (copy the config first if you want to retain configuration), burn the new image onto the same or new key, pop it in and boot to it, copy over the config or auto import the old volume, done.

phosdex
Dec 16, 2005

It's actually easier, you can upgrade freenas from its webui.

suddenlyissoon
Feb 17, 2002

Don't be sad that I am gone.

Megaman posted:

FreeNAS is so maintenance free it's not even funny, and it's so easy to set up. It's also ridiculously hassle free to upgrade. Just shut down the machine (copy the config first if you want to retain configuration), burn the new image onto the same or new key, pop it in and boot to it, copy over the config or auto import the old volume, done.

So it would be worth more to build a freenas machine and do raidz2 (or ZFS2, whatever it's called) at 16tb's than to have a Synology machine that could theoretically reach 52tb's (with 2 expansion groups full of 4tb drives) or beyond? I can't seem to figure out what would be more beneficial, capacity or the energy savings of not running a HTPC 24/7 to do Plex. Everything I've found online indicates that it's next to impossible to expand a ZFS2 or RaidZ2 volume once it's created without first destroying it.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

Megaman posted:

FreeNAS is so maintenance free it's not even funny, and it's so easy to set up. It's also ridiculously hassle free to upgrade.
Some things don't work right, tho, even with the latest release. For instance, if I create an user for SMB, I have to change its password right after creation, otherwise it won't get added to the Samba database. I tore out many hairs trying to find out why the gently caress CIFS didn't work. Also, adding sysctls via the GUI doesn't work at all, at least not if you're trying to tweak ZFS. Which has dumb prefetching settings for a streaming server. I just upped the zfetch limit from one measly megabyte to 64MB, which makes more sense for watching movies.

DrDork posted:

What functionality are you getting out of this that simply using SMB directly off your ZFS server wouldn't meet?
There's applications that hate SMB. Lightroom for instance refuses to store and/or open its catalog on a Samba share. Same goes for lots of games. So iSCSI is the optimal way around that.

--edit: Whoops, you mean the spooling iSCSI ZVOLs through a virtual fileserver.

Combat Pretzel fucked around with this message at 22:16 on Oct 11, 2014

Hadlock
Nov 9, 2004

What's a fair friend price for two 1TB SATA drives, 2010 era? It looks like they go for about $60 new. Drives have been in storage since about 2011 and I happen to have two spare SATA ports on my file server. I'm thinking $40 for the pair? They'll be less reliable than what's in there right now but I'm only at about 33% capacity in a mirrored storage pool array.

Hadlock fucked around with this message at 22:36 on Oct 11, 2014

yomisei
Mar 18, 2011
How well (or bad) will a striped mirror (mirrored stripe?) of 4 3TB WD Reds do for vm disk and other hot data do on ZFS? Can a 256GB SSD for L2ARC and 120GB SSD for ZIL bring the performance to something snappy a bit close to SSD level?

Factory Factory
Mar 19, 2010

This is what
Arcane Velocity was like.

yomisei posted:

How well (or bad) will a striped mirror (mirrored stripe?) of 4 3TB WD Reds do for vm disk and other hot data do on ZFS? Can a 256GB SSD for L2ARC and 120GB SSD for ZIL bring the performance to something snappy a bit close to SSD level?

Badly. 5400 RPM drives have poo poo for random I/O performance. Cache will help with writes but not much at all with reads. Consider Red Pros or HGST Deskstar NAS drives.

Also, I think striping mirrored sets is more fault-tolerant. If you lose one disk from each set of two, striped mirrors still works when mirrored stripes fails.

Trammel
Dec 31, 2007
.

suddenlyissoon posted:

have a Synology machine that could theoretically reach 52tb's (with 2 expansion groups full of 4tb drives) or beyond?

Don't spread your raid across the main & expansion units. It's supported, and really easy to do, and works great. Except maybe, there's a power outage, and the expansion unit shuts down quicker than the main unit, or afterwards, you turn them on in the wrong order, or you accidentally dislodge that single, fragile, external cable linking the two units, and then the main unit kicks those 5 drives out of the shared array. Then your hosed until the nice Synology tech support man ssh's in to your server and performs some mdadm magic.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

suddenlyissoon posted:

So it would be worth more to build a freenas machine and do raidz2 (or ZFS2, whatever it's called) at 16tb's than to have a Synology machine that could theoretically reach 52tb's (with 2 expansion groups full of 4tb drives) or beyond? I can't seem to figure out what would be more beneficial, capacity or the energy savings of not running a HTPC 24/7 to do Plex. Everything I've found online indicates that it's next to impossible to expand a ZFS2 or RaidZ2 volume once it's created without first destroying it.
Depends on what you mean by "worth more." Don't be put off by RAIDZ's expansion limits; as Trammel notes, you don't really want to be splitting an array over separate hardware units to begin with, and you wouldn't really want to be running something like SHR (2-drive tolerance max) on a 13 drive system to begin with, right? With a custom box, when you want more drives you just add them until you run out of space. Then you buy a bigger case and keep going. So either way, you're probably going to be looking at breaking it up into a few smaller arrays, in which case you have to ask yourself:

Is the minimal extra time and effort it will take me to roll a ZFS box worth not having to pay $500 every time I want to add another 5-drive-bay? What about not having any official support?

Synologies are nice for their simplicity and ease of setup, but when you stop to think that you'll be paying $1000 just for the expansion bays they become a lot harder to seriously recommend. Especially since you can get a lot more powerful hardware out of a custom box. Building the box only takes a few hours at most unless you really don't know what you're doing. You obviously don't have anyone you can call up and complain to when it doesn't work right, though, which I can understand being a deal-breaker in a corporate environment where the extra cash isn't so much of a concern.

suddenlyissoon
Feb 17, 2002

Don't be sad that I am gone.

DrDork posted:

Depends on what you mean by "worth more." Don't be put off by RAIDZ's expansion limits; as Trammel notes, you don't really want to be splitting an array over separate hardware units to begin with, and you wouldn't really want to be running something like SHR (2-drive tolerance max) on a 13 drive system to begin with, right? With a custom box, when you want more drives you just add them until you run out of space. Then you buy a bigger case and keep going. So either way, you're probably going to be looking at breaking it up into a few smaller arrays, in which case you have to ask yourself:

Is the minimal extra time and effort it will take me to roll a ZFS box worth not having to pay $500 every time I want to add another 5-drive-bay? What about not having any official support?

Synologies are nice for their simplicity and ease of setup, but when you stop to think that you'll be paying $1000 just for the expansion bays they become a lot harder to seriously recommend. Especially since you can get a lot more powerful hardware out of a custom box. Building the box only takes a few hours at most unless you really don't know what you're doing. You obviously don't have anyone you can call up and complain to when it doesn't work right, though, which I can understand being a deal-breaker in a corporate environment where the extra cash isn't so much of a concern.

I've got no problems building it, I can do that all day. My problem is the, what seems to me, back and forth nature of freenas where every single thing you want to do requires you to reset permissions, users, etc. I'm just worried that I do one wrong thing and poof, all my data is gone.

Megaman
May 8, 2004
I didn't read the thread BUT...

suddenlyissoon posted:

I've got no problems building it, I can do that all day. My problem is the, what seems to me, back and forth nature of freenas where every single thing you want to do requires you to reset permissions, users, etc. I'm just worried that I do one wrong thing and poof, all my data is gone.

This point is actually irrelevant, since this is what backups are for. This can happen with any system, but moreso the synology. Synology is proprietary which means the following. Since the Synology formatted disks only work with Synology devices and ONLY the same exact synology model you purchased and ONLY with the configuration from that particular device (granted you backed it up before the time of crash), if something happens with the synology you're completely hosed unless all your ducks are in a row (you have the exact device/config/disks standing by). So you're better off going FreeNAS especially in this case. FreeNAS is not a proprietary device, so if the hardware goes who cares, you can put the same array in any other machine as long as the machine functions like a normal computer. If you're concerned about the data, any data from anywhere really, back it up onto offline/offsite disks, and with the online array use Z2 or Z3 (I will always recommend Z3)

Sub Rosa
Jun 9, 2010




Megaman posted:

Synology is proprietary which means the following. Since the Synology formatted disks only work with Synology devices and ONLY the same exact synology model you purchased and ONLY with the configuration from that particular device (granted you backed it up before the time of crash), if something happens with the synology you're completely hosed unless all your ducks are in a row (you have the exact device/config/disks standing by).

It's actually just mdadm under the hood, and the raid can be put back together under plenty of other linux systems. Pretty sure also isn't correct that you can't just move your disks to a new different model Synology and it will also find and recognize the volume.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

suddenlyissoon posted:

I've got no problems building it, I can do that all day. My problem is the, what seems to me, back and forth nature of freenas where every single thing you want to do requires you to reset permissions, users, etc. I'm just worried that I do one wrong thing and poof, all my data is gone.

Nah, you gently caress up some permissions or user settings or whatever, your data hasn't gone anywhere. You just lose some time as you go back and fix whatever settings you dicked up.

Megaman
May 8, 2004
I didn't read the thread BUT...

DrDork posted:

Nah, you gently caress up some permissions or user settings or whatever, your data hasn't gone anywhere. You just lose some time as you go back and fix whatever settings you dicked up.

Exactly. And since people only use their setups for themselves it usually just results in a chown -R and/or a chmod -R

phosdex
Dec 16, 2005

So does anybody have some good guesses on what kind of capacities we'll be seeing from SSD over the next year?

Last week I replaced the last 2 mechanical drives in my desktop with another ssd. (retired a raptor with over 70k hours and 0 errors). So now I'm all ssd/flash storage in my home except for freenas with some wd reds. If we see 2TB ssd in 2015 at reasonable prices I can see myself going that route.

phosdex fucked around with this message at 23:42 on Oct 12, 2014

theres a will theres moe
Jan 10, 2007


Hair Elf

phosdex posted:

So does anybody have some good guesses on what kind of capacities we'll be seeing from SSD over the next year?

Last week I replaced the last 2 mechanical drives in my desktop with another ssd. (retired a raptor with over 70k hours and 0 errors). So now I'm all ssd/flash storage in my home except for freenas with some wd reds. If we see 2TB ssd in 2015 at reasonable prices I can see myself going that route.

I don't have a prediction for ssd prices, but I am curious why would you want to do this? Power consumption and reliability?

Megaman
May 8, 2004
I didn't read the thread BUT...

Juice Box Hero posted:

I don't have a prediction for ssd prices, but I am curious why would you want to do this? Power consumption and reliability?

I'm curious as well, I'd understand for system disks, as no one should care about their boot disks being destroyed, and there is a huge performance boot going to SSDs. But I don't understand for long term storage disks. I wouldn't exactly get the power consumption argument, unless you really can't afford a few extra bucks for power, in which case you shouldn't be buying more parts, you should just use that money to pay for the extra power. And reliability, do/will SSDs be more reliable than mechanical disks at that time? Probably not.

phosdex
Dec 16, 2005

the answer is pretty much, for the hell of it.

I'm really just curious about whats in store for ssds over the next year.

phosdex fucked around with this message at 03:55 on Oct 13, 2014

suddenlyissoon
Feb 17, 2002

Don't be sad that I am gone.
So I'm about to pull the trigger on this and send back the 1513+ that I got from Amazon last week. Does this look like a good build for FreeNAS as a Plex server? I'd also be interested in running SABNZBD, CouchPotato, Sickbeard and Transmission on it as well. My big worry is accessing the files remotely and I couldn't find a whole lot of info on that. Is it because it's as easy as forwarding ports in my router for AFP, CIFS, etc? Is there really a lot of command line prompts or can most of what I need to do be set up in the GUI?

Xeon E3-1231 v3
ASRock E3C226D2I
Crucial 16gb DDR3-1600 ECC
6x WD Red 4tb (raidz2)
Fractal Design Node304
Seasonic 550w 80+ Gold

I guess if I bought all this, it would be my final decision. It's a lot easier to send back a Synology than to send back individual used computer parts.

SamDabbers
May 26, 2003



suddenlyissoon posted:

So I'm about to pull the trigger on this and send back the 1513+ that I got from Amazon last week. Does this look like a good build for FreeNAS as a Plex server?

You may consider the Lenovo server deal I posted on the previous page. Similar horsepower, similar price, but you get a sweet hotswap case with it.

SamDabbers posted:

For anyone looking to build a new NAS with similar requirements, take a look at the Lenovo TS440. It comes with four 3.5" hot swap bays, and can be upgraded to eight. You'll also have to buy drive sleds, because the ones included are just dummy placeholders.

yomisei
Mar 18, 2011

suddenlyissoon posted:

Xeon E3-1231 v3
ASRock E3C226D2I
Crucial 16gb DDR3-1600 ECC
6x WD Red 4tb (raidz2)
Fractal Design Node304
Seasonic 550w 80+ Gold

You might as well step down the PSU to a 250-300W one.

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost

suddenlyissoon posted:

I'd also be interested in running SABNZBD, CouchPotato, Sickbeard and Transmission on it as well. My big worry is accessing the files remotely and I couldn't find a whole lot of info on that. Is it because it's as easy as forwarding ports in my router for AFP, CIFS, etc? Is there really a lot of command line prompts or can most of what I need to do be set up in the GUI?
FreeNAS implements these services under jails and tells you the IPs that they're running under, and sometimes it's not 100% clear which port these guys are running on when you install it through the GUI. The bigger problems for me have been properly upgrading each of these services on a steady basis and I find it oddly harder than when I just put everything on the same system with everything running on different ports.

But really, forwarding the usual CIFS and AFP ports to the file server is dangerous for security reasons and you should be using something like a VPN setup (there's a FreeNAS plugin for one I believe) if you're not the most comfortable with stuff like SSH tunneled proxy setups. Otherwise, something like Owncloud can help a bit I guess.

suddenlyissoon
Feb 17, 2002

Don't be sad that I am gone.

necrobobsledder posted:

FreeNAS implements these services under jails and tells you the IPs that they're running under, and sometimes it's not 100% clear which port these guys are running on when you install it through the GUI. The bigger problems for me have been properly upgrading each of these services on a steady basis and I find it oddly harder than when I just put everything on the same system with everything running on different ports.

But really, forwarding the usual CIFS and AFP ports to the file server is dangerous for security reasons and you should be using something like a VPN setup (there's a FreeNAS plugin for one I believe) if you're not the most comfortable with stuff like SSH tunneled proxy setups. Otherwise, something like Owncloud can help a bit I guess.
Currently with my Synology I connect through AFP at work to my home unit. It prompts me for a user and pass...would it not do that due to the way freenas is configured? I can understand the risk when it wouldn't prompt but my 30+ character passwords would take ages to get through.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
FYI, if you want this device to be able to spin down, you need to drop one of the drives and add a SSD to create a separate zpool with, to put the system dataset on (and the jails). Otherwise it'll keep the drives spinning, say if you expect it to go into low power mode during the night.

My configuration is this:

ASRock E3C226D2I
Intel Xeon E3-1220V3
Kingston DDR3-1600 2x8GB ECC
bequiet! Pure Power L8
2x WD Red 3TB (mirror)
2x WD RE4 2TB (mirror)
1x Intel 320 40GB
Both NIC ports hooked up (1W for each active one)

The box draws around 54W idle with spinning disks, 33W with the disks spun down.

Combat Pretzel fucked around with this message at 19:07 on Oct 13, 2014

suddenlyissoon
Feb 17, 2002

Don't be sad that I am gone.

Combat Pretzel posted:

FYI, if you want this device to be able to spin down, you need to drop one of the drives and add a SSD to create a separate zpool with, to put the system dataset on (and the jails). Otherwise it'll keep the drives spinning, say if you expect it to go into low power mode during the night.

My configuration is this:

ASRock E3C226D2I
Intel Xeon E3-1220V3
Kingston DDR3-1600 2x8GB ECC
bequiet! Pure Power L8
2x WD Red 3TB (mirror)
2x WD RE4 2TB (mirror)
1x Intel 320 40GB
Both NIC ports hooked up (1W for each active one)

The box draws around 54W idle with spinning disks, 33W with the disks spun down.

At my current costs, that sort of wattage difference would cost me a $1.50 per month. Are you actually doing it to save money or to extend the hard drive life? I thought that spinning up & down was harder on the life of a hard drive than just staying spun up.

Or am I completely lost and you're actually talking about how you need a flash drive to run the system off of? I just didn't list that.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
3€/month here. Mostly for noise reduction until I've pulled the new cables so that I can move it to a different room.

As far as longevity goes, I don't know. During work days, it can be spinning idle for up to 16 hours straight a day, not sure if that's better. The timeout is pretty generous so that it doesn't spin down often at all.

suddenlyissoon posted:

Or am I completely lost and you're actually talking about how you need a flash drive to run the system off of? I just didn't list that.
No, I have an USB stick for that. The system dataset is hitting the first zpool you create, it stores configuration, log files, RRD database and so on. I have that on an SSD now.

However, FreeNAS 9.3 wants to use ZFS for the system partition, so that it can use snapshots and boot environments for updates. ZFS and USB sticks don't go well together, so a SSD may be recommended at some point. Version 9.2 and before don't have issues, because once installed, the UFS partition is mounted as read only.

Combat Pretzel fucked around with this message at 19:53 on Oct 13, 2014

BlankSystemDaemon
Mar 13, 2009



If any of you are in the market for some more serious storage, Asus will have you covered in the future.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

D. Ebdrup posted:

If any of you are in the market for some more serious storage, Asus will have you covered in the future.

And it supports ECC!

IOwnCalculus
Apr 2, 2003





Newegg Business already has it at $380 but unfortunately, it's all Marvell chipsets for the NIC and SAS - so BSD support is apparently poo poo.

Factory Factory
Mar 19, 2010

This is what
Arcane Velocity was like.
32 GB of RAM isn't enough for more than 32 TB of ZFS storage anyway, is it? You'd want to go with another softRAID.

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.

Factory Factory posted:

32 GB of RAM isn't enough for more than 32 TB of ZFS storage anyway, is it? You'd want to go with another softRAID.

What are people doing instead of RAID-Z anyway? I really like ZFS but want to be able to scale past 32TB as 4TB drives get cheaper.

Adbot
ADBOT LOVES YOU

Galler
Jan 28, 2008


Factory Factory posted:

32 GB of RAM isn't enough for more than 32 TB of ZFS storage anyway, is it? You'd want to go with another softRAID.

Are you an enterprise storage architect designing an enterprise storage solution? If yes, you're in the wrong thread. If no, don't worry about your quantity of RAM. Just get 8GB or maybe 16 if your NAS is doing half a dozen other tasks and has a bunch of capacity. Or get 32GB of RAM if it makes you happy, whatever, I won't judge you.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply