|
Questions for you storage gurus. I'll be building a file server for a friend (home use). After talking to him, we are looking for something that is redundant to cover failure of a hdd and easily expandable in the future. At first I was debating between a little Atom build with raid card, 3x2tb HDD for the storage array (Raid 5), then Windows 7 for front end. Windows 7 is preferred because he is not familiar with linux at all. (OpenIndiana looks awesome, but I don't want to leave him with a linux box) Another thought was similar build, but using a cheap AMD cpu/mobo, and using the onboard sata for the array. Last question comes down to drives. Suggestions for 2tb drives for a home use array? I dislike the whole TLER issue with the WD drives, and the RE drives are out of the price range.
|
#
?
Jan 26, 2011 07:14
|
|
|
|
| # ? May 15, 2024 09:25 |
|
|
Anyone have any stability issues with OpenIndiana? I just put together a storage box and it keeps crashing. OI installed fine, I joined it to the domain, and transferred a couple TB over to a dedup'd filesystem via CIFS. No problem. However, working with those files is causing frequent crashes, and I don't know if it's OI, dedup or hardware. I didn't have these issues when I tried out Solaris 11 Express, but I didn't use dedup and didn't transfer as many files. I can always crash the system by deleting a large directory tree. CIFS or locally with rm, it still crashes. All network connections die, the console hangs, and the disks appear to be idle. I can still ping the box though. Deleting a small directory tree works, but seems delayed. The last thing I tried was a zfs destroy on the shared filesystem, and after a few hours, that also hung the box. Any ideas? What should I be looking for? OI Build 148 2 x Intel SASUC8I (with LSI IT firmware) Supermicro X8SIA-F 8 GB ram 8 x 2TB HDS722020ALA330
|
|
#
?
Jan 26, 2011 09:18
|
|
|
What does iostat say? How about zpool status? Sounds like a bad hard drive in the array causing stalls and/or some memory or drive controller problem involved with the crashes.
|
|
#
?
Jan 26, 2011 17:50
|
|
|
necrobobsledder posted:What does iostat say? How about zpool status? Sounds like a bad hard drive in the array causing stalls and/or some memory or drive controller problem involved with the crashes. Yeah, I'd run memtest overnight, sounds like bad hardware failing under load perhaps (did you say the couple TB transfer was complete and it crashes trying to access those files?)
|
|
#
?
Jan 26, 2011 18:54
|
|
|
Moey posted:Questions for you storage gurus. Buy an off-the-shelf solution like the ones synology sells. It will be cheaper and less effort.
|
|
#
?
Jan 26, 2011 22:30
|
|
|
Moey posted:Questions for you storage gurus. My current go-to is the synology 411; its 350 bucks on ebay and is solid as a rock. good features. plus if the controller drops a good drive you can rebuild your poo poo with any 4-sata motherboard and a copy of ubuntu desktop.
|
|
#
?
Jan 26, 2011 23:02
|
|
|
dietcokefiend posted:Synology time!  Since you guys were mean I decided to spite all of you and buy the DS211+ instead. Got an awesome deal on it 
|
|
#
?
Jan 27, 2011 00:39
|
|
|
Off the shelf isn't as fun as building something, but seems like a better recommendation so I will not be lifetime tech support for this. DS411J looks like a good solution. http://www.newegg.com/Product/Product.aspx?Item=N82E16822108066 Looks like I can also expand the array on the fly by inserting larger disks. Has anyone tried that before? So if I start with a 4tb (3x2tb) raid 5 array, I can add a 4th 2tb disk to increase the size to 6tb on the fly? What about down the road, replacing those 2tb drives with 3tb drives one by one? This seems like a phenomenal solution. Edit: Pulling the trigger on this, going to start with 3x2tb WD Green drives and see how it works. I'll post back with updates if anyone is interested (could do some speed tests too). Moey fucked around with this message at 17:05 on Jan 27, 2011 |
|
#
?
Jan 27, 2011 01:29
|
|
|
movax posted:Yeah, I'd run memtest overnight, sounds like bad hardware failing under load perhaps (did you say the couple TB transfer was complete and it crashes trying to access those files?) Thanks - I kicked off memtest. Yeah, the transfer completed without any problems. I monitored io periodically, and it was slower than I expected (~40-50MB/s) but all the drives were consistent and blocked about the same amount. Traversing or deleting large directory trees is what appears to cause the crash.
|
|
#
?
Jan 27, 2011 01:33
|
|
|
Combat Pretzel posted:Run pfexec fmadm faulty and see whether there's a device loving up. Hey forgot to say thanks for this earlier. The issue fixed itself after about 30 hours but this is a real handy command to know.
|
|
#
?
Jan 27, 2011 02:31
|
|
|
I'm looking at the last few pages and I am totally lost. The OP hasn't been edited since 2008... is there a link to a more updated general info post? I kind of want a list of what I need to buy and best software solutions.
|
|
#
?
Jan 27, 2011 04:42
|
|
|
r2tincan posted:I kind of want a list of what I need to buy and best software solutions. Then you need to give a good idea of what you really want and are capable of computer/networking wise. People in this thread are running devices that are basically a hard drive with a network port, all the way up to enterprise-grade raid arrays with multipath I/O for their torrent server virtual machines. I'm guessing what you want is somewhere in the middle of that spectrum.
|
|
#
?
Jan 27, 2011 19:29
|
|
|
r2tincan posted:I'm looking at the last few pages and I am totally lost. The OP hasn't been edited since 2008... is there a link to a more updated general info post? what you need to buy: http://www.newegg.com/Product/ComboBundleDetails.aspx?ItemList=Combo.576656 install some OS on it, have a blast. I wish I'd seen this combo a year and a half ago before I started doing poo poo piece by piece and had just bought a goddamn 20TB pack.
|
|
#
?
Jan 27, 2011 19:37
|
|
|
Telex posted:what you need to buy: Doesn't support Solaris, worthless, etc etc.
|
|
#
?
Jan 27, 2011 20:41
|
|
|
I'm configuring a DS411j with 4x1T AV-GP's today. Nice improvements over previous Synology DS4xxx's i've used - dual fans, nice cages with 2.5" drive mounts.
|
|
#
?
Jan 27, 2011 20:46
|
|
|
Minus Pants posted:Thanks - I kicked off memtest. Yeah, the transfer completed without any problems. I monitored io periodically, and it was slower than I expected (~40-50MB/s) but all the drives were consistent and blocked about the same amount. Traversing or deleting large directory trees is what appears to cause the crash. That's a known dedup issue, backup your files to someplace else, delete the pools and rebuild them without dedup enabled.
|
|
#
?
Jan 27, 2011 21:35
|
|
|
FISHMANPET posted:Doesn't support Solaris, worthless, etc etc. I'm getting closer and closer to just saying "gently caress it" and custom rolling a 4k ashift'd zpool, because 7K2000s are never going to dip below $100 online. Those 5900rpm Seagates seem to be the least evil drive.
|
|
#
?
Jan 27, 2011 22:09
|
|
|
Jonny 290 posted:I'm configuring a DS411j with 4x1T AV-GP's today. Nice improvements over previous Synology DS4xxx's i've used - dual fans, nice cages with 2.5" drive mounts. I've also just finished setting up a DS411j today, with 4x2 TB 5900 rpm Seagates inside. Agree, I've been pretty pleased with the build quality of the unit and also the quality of the UI (and the price!). I also like the fact you can roll a straight Linux software raid so if the NAS dies I can just plug the drives in a normal computer. Transferring files to the NAS over my crappy wireless network is eye-wateringly slow at 1MB/s though, so might be time for me to upgrade or dig out the external drives for that initial sync.
|
|
#
?
Jan 27, 2011 22:28
|
|
|
Col posted:I also like the fact you can roll a straight Linux software raid so if the NAS dies I can just plug the drives in a normal computer. This is the killer app for Synology for me; I've already saved one customer's bacon this year because I was able to recover their financials off their blown RAID5 using an ubuntu desktop CD and a couple mdadm commands.
|
|
#
?
Jan 27, 2011 22:32
|
|
|
Is that possible with the Synology Hybrid RAID setup (rather than the regular RAID levels)? All they say about it is that it's "based on LInux!" but I'm wondering if they did custom stuff that won't be readable non Syno boxes. If anyone is incredibly stupid like me about miniaturization, they upgraded their 2.5" box to be more on par with their other ones: DS411slim. I think it's basically just a CPU and RAM bump (1.2 to 1.6ghz and 128 to 256MB). japtor fucked around with this message at 22:42 on Jan 27, 2011 |
|
#
?
Jan 27, 2011 22:37
|
|
|
movax posted:I'm getting closer and closer to just saying "gently caress it" and custom rolling a 4k ashift'd zpool, because 7K2000s are never going to dip below $100 online. Those 5900rpm Seagates seem to be the least evil drive. The seagates are 80 bucks each at amazon too so if you have Prime, free ship. They're not bad drives at all from what I can tell, they avoid the 4k sector issue and as soon as I can buy 5 more I'll have purged the 4k sector nonsense from my system and I can figure out something else to do with 12TB worth of that "advanced format" bullshit that doesn't involve a raid.
|
|
#
?
Jan 27, 2011 22:40
|
|
|
Unraid pushed out 4.7 release with 4k sector drive support. Awesome I just finished my setup a month ago with a bunch of EARS(jumpers huzza). Haven't made a jump at home to ZFS and stuck with unraid due to its easy upgrade/management. The thing just works fine with SIL card for extra drives. Migrated from WHS to it to at least get the parity protection and the snap-in for sabnzbd/sickbeard made it too easy. Network writes are ~60MB/s due to decent cache drive and overnight the drive moves it to the slow write parity pool. At work I'm looking at doing a FreeNAS/ZFS powered VMWare NAS datastore. Have a pe2950 quad core with 4x 500GB drives in a zraid1(thinking of zraid2). I was shocked that I could pull 128MB/s reads and 30MB/s writes from the VM running dd without any tuning. One downside is the controller is a PERC6 that has zero support for exposing the disks outside of a raid array to the OS. Ended up having to build 4virtual disks in the PERC6 of raid0 what a kludge.
|
|
#
?
Jan 27, 2011 23:03
|
|
|
Telex posted:The seagates are 80 bucks each at amazon too so if you have Prime, free ship. Which Seagate drives are you referring to? I assume this one? http://www.newegg.com/Product/Product.aspx?Item=N82E16822148413 Would that be a better choice for inside a DS411j over the WD20-EARS? EARS are on the compatibility list. Moey fucked around with this message at 23:13 on Jan 27, 2011 |
|
#
?
Jan 27, 2011 23:08
|
|
|
japtor posted:Is that possible with the Synology Hybrid RAID setup (rather than the regular RAID levels)? All they say about it is that it's "based on LInux!" but I'm wondering if they did custom stuff that won't be readable non Syno boxes. If they're doing their "hybrid raid' how I suspect they are, which is a combination of raid5 volumes and raid1 volumes to utilize the space, then glomming it all back together, it would be doable, but you'd have to get all the arrays assembled. fake edit: after reading, that is exactly what they are doing. So yes, trust that you can get SHR back, but I won't say that it won't be a much bumpier road. We don't mess around with upgrading drive sizes or anything though - if one of our customers wants more space we provision a new NAS with zero hour drives, and do a transfer. So I just do raid6 (if I get my way) or raid5(if sales gets their way). Jonny 290 fucked around with this message at 23:10 on Jan 27, 2011 |
|
#
?
Jan 27, 2011 23:08
|
|
|
Jonny 290 posted:I'm configuring a DS411j with 4x1T AV-GP's today. Nice improvements over previous Synology DS4xxx's i've used - dual fans, nice cages with 2.5" drive mounts. I'm very happy with my DS410 so far. It gave me some grief as I tried to replicate the setup I had going with another home server but after some more poking and clicking and fiddling around it's now handling my backups AND storing media so I can watch it on my iPod whenever I like. The setup process was super easy though I didn't see people warning about the SHR settings until too late. I don't care to rebuild the array at this point so I'm just going to run with it for now. Using the built-in web server and a PHP Autodirectory listing script I can get the same level of point-and-click access to my pre-converted movies that I had on my other server. I needed something to handle that because the Synology's web server is a bit fussy about displaying the root drive unless you have an index.xxx file in there AND it doesn't seem to auto-list subdirectories. I'm not going to bother dicking around under the hood to change that so the autolisting php script works fine for me. The setup was simple, it has way more power and space than I need right now and should be a real boon in the future when poo poo blows up on my main computer. Hooray for finally having a backup!
|
|
#
?
Jan 27, 2011 23:33
|
|
|
Moey posted:Which Seagate drives are you referring to? I assume this one? http://www.newegg.com/Product/Product.aspx?Item=N82E16822148413 yes, those are the seagates that are fairly quiet and fast enough in a raid. the EARS drives have 4k sectors instead of 512bytes. I don't know how the Synology raids work, but for me right now with a ZFS raid those drives have fairly awful as poo poo performance. I've replaced the switch, the network card and neither has improved a stuttering problem so I'm left to assume there is a fundamental issue with the drives despite formatting them as 4k sectors in a ZFS raid which leaves me to believe that the flag in FreeNAS that selects that doesn't do much at all. Abysmal performance. I can tell when files are on different drives, they scream on at 100MB/s read or write. From the EARS drives, I get around 30, but that's not sustained since it starts at 60 and has random pauses while transferring. From what I've read it's a sign of mis-aligned sectors when they have read issues and that's because of the "advanced format" so blah blah I hate the drives.
|
|
#
?
Jan 27, 2011 23:41
|
|
|
My random fun money this week is partially going towards a fistful of 36 gig 15k SAS Cheetahs. This should be a zippy little VM disk! Controller is ghetto fabulous Rocketraid 2640 X4. I know, not really NASy, but it is storagey. Right now my VM's run on a raid1 stripe across two U160 36g Cheetahs. Fun drives, good access time, cartoonishly loud.
|
|
#
?
Jan 28, 2011 00:40
|
|
|
Good access times for 1998 maybe...
|
|
#
?
Jan 28, 2011 02:13
|
|
|
I'm a firm believer in retrocomputing, what can i say.
|
|
#
?
Jan 28, 2011 02:43
|
|
|
Jonny 290 posted:Right now my VM's run on a raid1 stripe across two U160 36g Cheetahs. Fun drives, good access time, cartoonishly loud. Want a 18GB cache drive?   Click here for the full 1200x1600 image.
|
|
#
?
Jan 28, 2011 03:08
|
|
|
Telex posted:The seagates are 80 bucks each at amazon too so if you have Prime, free ship. No, they're still advanced format/4K internally I think? SmartAlign just takes care of lining poo poo up properly for WinXP I believe. I posted about 'em to the zfs-discuss mailing list, could never get a straight answer. I think that I'll get some Seagates, and just build a pool with the modified binary and ashift parameter. The ashift should (SHOULD) remain through the life of the pool, regardless of upgrades, so eventually, it'll end up OK. e: http://opensolaris.org/jive/thread.jspa?messageID=506787 is the thread I was on
|
|
#
?
Jan 28, 2011 03:42
|
|
|
movax posted:No, they're still advanced format/4K internally I think? SmartAlign just takes care of lining poo poo up properly for WinXP I believe. I posted about 'em to the zfs-discuss mailing list, could never get a straight answer. I was under the impression that the 32mb cache 5900 RPM drives were normal, at least that's what my documentation indicated, and the newer 64mb cache drives have the advanced format. I think even Seagate's pages are incorrect on this, or they've started shipping new models which sorta sucks. Or I'm totally wrong and I guess SmartAlign is loving awesome because I didn't do any formatting tricks with ZFS and they work fine and are fast as far as I'm aware. Now I'm confused and I'm not sure what I should do here. I really dislike having two different formats of drive in a system. Now I'm thinking I need to replace my hitachi drives too and go all advanced format seagate drives instead of this mix of bullshit 
|
|
#
?
Jan 28, 2011 04:22
|
|
|
If you run FreeBSD (or can use a FreeBSD livecd), you can get the ashift=12 without any modified binaries. If you're using the livecd, start the installer and select the Fixit option on the first menu. Load the zfs module: # kldload opensolaris # kldload zfs Check /dev to see where it filed your drives. If you loaded the ahci module (not on by default), they'll be /dev/adaX, otherwise they'll be /dev/adX. Then for every drive destined for your pool: # gnop create -S 4096 /dev/ada1 # gnop create -S 4096 /dev/ada2 etc. You will end up with new entries in /dev like ada1.nop, ada2.nop, etc. These are just your drives with FreeBSD faking a 4k sector size. Now just create your pool: # zpool create whatever raidz ada1.nop ada2.nop ... You can double-check it by running zdb | grep ashift When you reboot, the .nop's will vanish, but that's fine. ZFS will see the real drives, and the ashift value will stick. Keep in mind, if you later add/replace a (512-emulated) drive, you might have to gnop it before ZFS will let it in. And from the looks of zdb, each vdev might have its own ashift value but I'm not sure.
|
|
#
?
Jan 28, 2011 04:28
|
|
|
Void Chicken posted:And from the looks of zdb, each vdev might have its own ashift value but I'm not sure. Was just e-mailing the guy, he confirmed that ashift is per vdev. He has ST32000542AS, HD204UI and 7K2000s with ashifts of 12, 12 and 9 respectively all in the same pool. The 32MB ones are on Amazon, so maybe I won't need an ashift. Suppose I can do some preliminary poking with ZFS/something to figure out what exactly is up. Going to order 12 for now, see how it goes, and then order the last 6 drives (plus hot spares) when they go on sale again (like they always do).
|
|
#
?
Jan 28, 2011 05:19
|
|
|
Other than the 10-15% perf boost from using an appropriate ashift of 12, is there any downside to using ashift=9 of with 4k drives? My Solaris Express 11 server's been humming along fine without changing the ashift.
|
|
#
?
Jan 28, 2011 15:47
|
|
|
devilmouse posted:Other than the 10-15% perf boost from using an appropriate ashift of 12, is there any downside to using ashift=9 of with 4k drives? My Solaris Express 11 server's been humming along fine without changing the ashift. Nope, just performance. I'm probably going to re-roll with OpenIndiana; how do you like Solaris Express 11?
|
|
#
?
Jan 28, 2011 17:23
|
|
|
movax posted:Nope, just performance. I'm probably going to re-roll with OpenIndiana; how do you like Solaris Express 11? It's been fine so far. It was a toss-up between going with Solaris + Virtual Box (with a FreeBSD and Linux guests) or ESXi as a base, and ultimately went with Solaris because I didn't have to run Windows as a guest (plus, I hate dealing with pass-through and sharing the datastore between the guests). It's got 6x 2TB Samsung F4s running in a raidz2 and the benchmarks put it around 350-450MB/s with ashift=9. With ashift=12, I got a bit more perf, but didn't really feel like it was worth doing sketchy business with a recompiled zpool binary.
|
|
#
?
Jan 28, 2011 17:47
|
|
|
movax posted:Nope, just performance. I'm probably going to re-roll with OpenIndiana; how do you like Solaris Express 11? I'm running Solaris 11 Express, and I'm pretty happy with it so far. It's 5 or 6 revs ahead of OpenIndiana (snv 151a instead of 146? that OpenIndiana is on). My only complaint is that there's no easy way to get the SunStudio compiler installed. Which is strange, because I'm supposed to be testing out my code on this, and you can't compile anything the Sun (Oracle $&&#!@$#) way. gcc3.6 is in the IPS repo, but if you want to compile any python modules you're going to need to recompile Python, because Python modules build with the same flags that the Python binary was built with, and gcc and SunStudio use differnet flags (it all comes down to -KPIC in SunStudio vs something else in gcc).
|
|
#
?
Jan 28, 2011 18:31
|
|
|
Cool, thanks for the replies guys. Think I may try out Sol11 Express then too. Trying to decide if I want to buy new hardware or not. I've decided that I'm going to do what I should have done years ago and make a Linux VM for myself where I have all my servers running, and place it on my storage box as well. My desktop has enough horsepower to keep running it (more now with a 2600K + 16GB RAM) but consolidation is fun. I'm going to install new OS on a P45 + E6600 board, and see how that goes. I'm kind of tempted to get a new mobo/CPU combo (possibly H67 + i5/i7) so I get quad-core CPU, integrated graphics, and two x8 links to populate with USAS-L8is. The last 4 drives get run off the mobo. Any thoughts on Solaris compatibility with 6-Series chipsets? (I believe Nehalem and newer get awesome power management as well) I think it'll be overkill to get a server mobo/Xeons, because then I have to pay extra for ECC and stuff. Though maybe it's about time I got ECC for a server, who knows...
|
|
#
?
Jan 28, 2011 19:02
|
|
|
|
| # ? May 15, 2024 09:25 |
|
|
S11X should support 4KB sectors, AFAIK.
|
|
#
?
Jan 28, 2011 19:28
|
|
