Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Vidaeus
Jan 27, 2007

Cats are gonna cat.

MrMoo posted:

JBOD means Just a Bunch Of Disks (Drives), i.e. each disk is available as a separate share. You lose a disk you lose the share.

Some implementations may join all the disks to appear as one logical volume, lose one disk then you simply lose the files on that disk only. Synology appears to go this route with additional caveat:


http://forum.synology.com/wiki/index.php/How_to_Manage_the_Synology_RAID_Volume

Underneath Synology appear to be using a linear software RAID, which has the following warning:


http://tldp.org/HOWTO/Software-RAID-HOWTO-5.html#ss5.4

I suspected that you risk losing the JBOD "array" if one drive fails. Looks like I will be setting both drives up individually. Thanks for the quick response!

Adbot
ADBOT LOVES YOU

tonic
Jan 4, 2003

I bought a Rocket 620 PCI-E SATA3 2 port adapter so that I can add a few more hard drives to my Microserver N36L.

The adapter works fine and the drives can be added to my array, but the speed of the hard drives using the card is much slower than those directly connected to my motherboard. For example, I can write at about 110MB/sec to a 2TB Hitachi drive connected to the motherboard, but only about 35MB/sec to the same model drive connected via the adapter.

Am I wrong in assuming I should see the same speeds with drives connected via the adapter? Is there some setting I need to change to achieve this?

Also, the adapter is currently connected via the PCI-E x1 port, would switching it to the x16 make any difference?

BnT
Mar 10, 2006

tonic posted:

Also, the adapter is currently connected via the PCI-E x1 port, would switching it to the x16 make any difference?

Even the slowest PCIe v1.0 1x card should be given a 250MB/sec bandwidth lane by the motherboard. Assuming you have a PCIe v2.0 board, you should be getting a 500MB/sec lane in the 1x port. I highly doubt moving it would make any difference.

What OS/filesystem are you using? Are there any differences between the drives from the OS's point of view? For instance, does write caching get disabled when the drive is connected to the PCIe card? If not then I'd look at firmware/drivers next, I guess.

overdesigned
Apr 10, 2003

We are compassion...
Lipstick Apathy
So, apparently this is common knowledge, but Newegg is atrocious at packing and shipping OEM hard drives. I ordered five 3TB drives for the Synology DS1511+ that I got, and three of them were dead on arrival.

At least their RMA process is pretty painless, but ugh.

tonic
Jan 4, 2003

BnT posted:

Even the slowest PCIe v1.0 1x card should be given a 250MB/sec bandwidth lane by the motherboard. Assuming you have a PCIe v2.0 board, you should be getting a 500MB/sec lane in the 1x port. I highly doubt moving it would make any difference.

What OS/filesystem are you using? Are there any differences between the drives from the OS's point of view? For instance, does write caching get disabled when the drive is connected to the PCIe card? If not then I'd look at firmware/drivers next, I guess.

Yes, it is a PCIe 2.0 board. I'm running unRAID. I had to enable write caching in the BIOS for the onboard SATA slots, so perhaps it needs to be enabled on the card as well? Can't find much info on it though.

I guess I'll try moving it to the x16 slot to see if that has any effect.

Other than the issues with this card, I'm really loving my Microserver :holy:

Wheelchair Stunts
Dec 17, 2005
The more I dig into this the more it reminds me of car work. I've got to have more capacity, I need to push more iops, I need more data throughput.

:iiaca:

heeen
May 14, 2005

CAT NEVER STOPS
I want to share the following, for what it's worth:
Before I had 4 wd20ears drives, formatted by freenas to have a 2gb swap partition ans the rest used for zfs. This gave me writing speeds around 60mb/s with dd.
I upgraded this to 5 of the same drives, but while rebuilding the pool, I erased all partitioning, created 4k sector devices with gnop (gnop create -S 4096 /dev/adaX) and subsequently created the zraid1 using these adaN.nop devices directly. Write speed almost quadrupled to 220Mb/s! Read speed also increased to 300 Mb/s.
I have 5gb ram in my microserver and if any service was to use more than 5gb ram, swap wouldn't help it anyways so I can live without it.

Longinus00
Dec 29, 2005
Ur-Quan

heeen posted:

I want to share the following, for what it's worth:
Before I had 4 wd20ears drives, formatted by freenas to have a 2gb swap partition ans the rest used for zfs. This gave me writing speeds around 60mb/s with dd.
I upgraded this to 5 of the same drives, but while rebuilding the pool, I erased all partitioning, created 4k sector devices with gnop (gnop create -S 4096 /dev/adaX) and subsequently created the zraid1 using these adaN.nop devices directly. Write speed almost quadrupled to 220Mb/s! Read speed also increased to 300 Mb/s.
I have 5gb ram in my microserver and if any service was to use more than 5gb ram, swap wouldn't help it anyways so I can live without it.

Your wording is a little confusing, 2GB swap partition per drive?

heeen
May 14, 2005

CAT NEVER STOPS

Longinus00 posted:

Your wording is a little confusing, 2GB swap partition per drive?

Yes, freenas partitions drives for you when you create a volume through the web GUI. Each drive gets a 2GB swap partition.

jeeves
May 27, 2001

Deranged Psychopathic
Butler Extraordinaire
I have a problem with copying large files around on my NAS (FreeNAS, v8)

I get this issue when both copying via Teracopy or Window's built in copy engine. Basically it copies one or two huge 2GB+ files, but then the transfer rate goes from an average of ~40mb down to like... 720k a second. But I think it actually froze too, as it gets stuck on the same percentage for quite a while-- either copying at that speed makes the percentage never seem to move, or it actually froze. It copies tons of smaller (less than 1GB) files just fine, but will choke on huge files.

Here is a screenshot of Teracopy, which actually at least throws up an error message in the log:



Note one file out of dozen copied, but then it says "Network name not available" as an error.

This is copying from both a Win7 machine and Win2003 machine to the NAS, and the Win2003 machine has copied huge files to the Win7 machine just fine. So it is obviously something with the NAS.

Wiring? Version 8 sucking? Where could I check for an error log in FreeNAS/command line? (I'm not very good with BSD log commands)

Note: This is on a Proliant Microserver with 8GB of RAM running ZFS.

Genderfluid
Jun 18, 2009

my mom is a slut
Does anyone have any recommendations for good backup software for Windows? Basically, I have two 2tb hds mounted in an external enclosure, and I'd like each one to contain a mirror of my C drive. Any software that can do this, including incremental backups, file verions, etc. would be awesome. Thanks.

Modern Pragmatist
Aug 20, 2008

JaundiceDave posted:

Does anyone have any recommendations for good backup software for Windows? Basically, I have two 2tb hds mounted in an external enclosure, and I'd like each one to contain a mirror of my C drive. Any software that can do this, including incremental backups, file verions, etc. would be awesome. Thanks.

I've recently started using CrashPlan and am really liking it so far. It is free to use it to backup to a folder (your external) and super cheap to backup to the cloud if you wish.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Modern Pragmatist posted:

I've recently started using CrashPlan and am really liking it so far. It is free to use it to backup to a folder (your external) and super cheap to backup to the cloud if you wish.

Seconding CrashPlan.

teamdest
Jul 1, 2007

Thermopyle posted:

Seconding CrashPlan.

And a third from me.

IOwnCalculus
Apr 2, 2003





Fourthed. It really is excellent, and it's one of the few providers out there that has really good support for Linux as well. I have it running on my headless fileserver and I just had to tweak a config file on the desktop client to be able to access and manage it through the app.

heeen
May 14, 2005

CAT NEVER STOPS

some zfs guy posted:

> It's pretty simple. If you go direct to disk, ZFS manages the cache and can "hand off" write to the disk.
> If it's a partition, ZFS has no awareness of the cache and has to wait for the disk to return the write acknoloedgement

Don't use zfs on partitions (as freenas does by default)!

Longinus00
Dec 29, 2005
Ur-Quan

heeen posted:

Yes, freenas partitions drives for you when you create a volume through the web GUI. Each drive gets a 2GB swap partition.

That's mindboggling. How many gigs of swap did it make for you?

heeen
May 14, 2005

CAT NEVER STOPS
2Gb per drive on 4 drives makes 8Gb.

evol262
Nov 30, 2010
#!/usr/bin/perl

jeeves posted:

Note one file out of dozen copied, but then it says "Network name not available" as an error.

This is copying from both a Win7 machine and Win2003 machine to the NAS, and the Win2003 machine has copied huge files to the Win7 machine just fine. So it is obviously something with the NAS.

Wiring? Version 8 sucking? Where could I check for an error log in FreeNAS/command line? (I'm not very good with BSD log commands)

Note: This is on a Proliant Microserver with 8GB of RAM running ZFS.

Needs a lot more info.

You're not really going to find "BSD log commands" (dmesg is unlikely to help, and the logs are flat files other than that).

I presume you're using CIFS/SMB sharing from Windows? Check the Samba log.
Check the zpool.
Try NFS.
Try SFTP.
Don't assume it's a problem with the NAS. It might be. It could also be your network choking due to an old/lovely router/switch. It could be Windows crapping out. Especially with all the changed to the CIFS stack Microsoft makes (IIRC, Win7/2k8 would instantly crash a Samba server if they used SMB 2.0). Try other protocols, check logs.

evol262 fucked around with this message at 15:45 on Aug 30, 2011

Henrik Zetterberg
Dec 7, 2007

Ok, I just set up a raid5 (first time I've tried this) on my new desktop PC at work with 4x 2TB WD Blacks in addition to an SSD OS Boot drive. I'm using the on-board Intel RAID controller on the Intel DH67GD with Sandy Bridge.

I configured a raid volume through the RAID controller BIOS. This volume is showing up in the Intel Rapid Storage Tech application in Windows (7 Enterprise, 64-bit). I clicked initialize and it seems to be crawling. I am assuming it is currently building data parity, so the slowness is expected.

At this point, should the drives be showing up in My Computer, or do I have to wait until the initialization/parity is complete? I am not seeing anything except my OS disk.

When I go to Disk Management, I see my raid volume (5.6GB), but it's showing up as:


I'm kind of confused as to why it seems there's 2 unallocated partitions in Disk1, both of seemingly random size.

Is something dicked up, or should I just have some patience and wait the couple days for initialization to finish?

BnT
Mar 10, 2006

Henrik Zetterberg posted:

I'm kind of confused as to why it seems there's 2 unallocated partitions in Disk1, both of seemingly random size.

Is something dicked up, or should I just have some patience and wait the couple days for initialization to finish?

You need a GPT partition table. The default MBR partition table on Windows 7 is limited to 2TB of addressable space. If that doesn't work for you, you'll need to look up 'diskpart' probably. Also, it's probably going to take at least a day to finish initializing, but you should be able use it, it'll just be slow in the mean time.

Posts Only Secrets
Jan 22, 2005
Breaking the NDA...
Does anyone have a guide to install sabnzbd and sickbeard on freenas 0.7.2? Preferably something that you've used yourself with success. I've tried a bunch listed on the freenas forums and haven't been able to successfully get it working. At some point in the process there's a step that does not match up with what I have.

Im on 0.7.2 Sabanda (revision 6694).

Thanks

jeeves
May 27, 2001

Deranged Psychopathic
Butler Extraordinaire

evol262 posted:

I presume you're using CIFS/SMB sharing from Windows? Check the Samba log.
Check the zpool.
Try NFS.
Try SFTP.
Don't assume it's a problem with the NAS. It might be. It could also be your network choking due to an old/lovely router/switch. It could be Windows crapping out. Especially with all the changed to the CIFS stack Microsoft makes (IIRC, Win7/2k8 would instantly crash a Samba server if they used SMB 2.0). Try other protocols, check logs.

Yes, I am using CIFS. Where should I check the samba log? On FreeNAS or on my two Windows clients (2003+Win7).

I'd rather not try SFTP, as the whole point of the NAS is for steaming media which FTP can't do. I can give NFS a shot, but I use Samba to stream to a Mac HTPC, and have never ever had a problem with anything copying/streaming from my Win2003 machine to anything else. Yet anything large trying to copy to or from the NAS? Errors. Same huge file from Win2003->Win7/Mac, fine.

Thus, I don't really think it is a wiring/router issue.

Henrik Zetterberg
Dec 7, 2007

BnT posted:

You need a GPT partition table. The default MBR partition table on Windows 7 is limited to 2TB of addressable space. If that doesn't work for you, you'll need to look up 'diskpart' probably. Also, it's probably going to take at least a day to finish initializing, but you should be able use it, it'll just be slow in the mean time.

Ahh, perfect! Once I converted to GPT then quick-formatted the partition to NTFS, it let me assign a drive letter. Currently copying a few hundred thousand files at 7MB/s while still initializing. Looks like I was just missing the GPT part.

Thanks!

sixide
Oct 25, 2004
Anyone have experience with SnapRAID? I'm looking for some simple robustness for my large, limited importance data files and it seems pretty solid and easy.

Zero Trust
Dec 29, 2009

"the car's on fire, and there's no driver at the wheel..."
I'm looking to buy something that's more or less simple plug n play, directly to an ethernet port on my router. Is the WD My Book Live 3TB still choice for this?

http://www.amazon.com/Western-Digital-Personal-Cloud-Storage/dp/B0047FL85U/ref=sr_1_2?ie=UTF8&qid=1314862989&sr=8-2

Saukkis
May 16, 2003

Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.
Is any of the current 2TB eco-harddrives suitable for use with Linux software RAID? I vaguely remember that earlier WD Green Powers had some behaviours that made them not as good for RAID. I doubt the performance would matter much since the rest of the harddrives would be older models and the price difference to non-eco models is upto 40€.

Factory Factory
Mar 19, 2010

This is what
Arcane Velocity was like.

Saukkis posted:

Is any of the current 2TB eco-harddrives suitable for use with Linux software RAID? I vaguely remember that earlier WD Green Powers had some behaviours that made them not as good for RAID. I doubt the performance would matter much since the rest of the harddrives would be older models and the price difference to non-eco models is upto 40.

Samsung Spinpoint drives seem to work well. The F2 and F4 series are both low-power, lower-RPM drives. They're no worse for RAID than any non-WD-Green, non-enterprise/RAID edition drive.

movax
Aug 30, 2008

Saukkis posted:

Is any of the current 2TB eco-harddrives suitable for use with Linux software RAID? I vaguely remember that earlier WD Green Powers had some behaviours that made them not as good for RAID. I doubt the performance would matter much since the rest of the harddrives would be older models and the price difference to non-eco models is upto 40€.

Hitachi 5K3000s and Seagate Barracuda LPs should be fine, I confirm they work great in Solaris 11 under ZFS.

evol262
Nov 30, 2010
#!/usr/bin/perl

jeeves posted:

Yes, I am using CIFS. Where should I check the samba log? On FreeNAS or on my two Windows clients (2003+Win7).

I'd rather not try SFTP, as the whole point of the NAS is for steaming media which FTP can't do. I can give NFS a shot, but I use Samba to stream to a Mac HTPC, and have never ever had a problem with anything copying/streaming from my Win2003 machine to anything else. Yet anything large trying to copy to or from the NAS? Errors. Same huge file from Win2003->Win7/Mac, fine.

Thus, I don't really think it is a wiring/router issue.

Windows won't have useful logs. I'm pretty sure it's system.log or (more likely) daemon.log on FreeNAS, which you should be able to get to from the web interface.

NFS works fine to Macs.

Do you understand how to troubleshoot?

You're trying to figure out whether it's a problem with protocol, network, or operating system. So...

Try SFTP or NFS. If those work, it says there's a problem with SMB/CIFS in particular, but the NAS is fine.

If those don't work, it points to a problem with the NAS or the network.

In either case, you check the logs to see if the FreeNAS system things something went wrong.

What happens when your file transfers hang? What do you need to do to bring it back to life? Wait? Reboot?

Are you running ZFS compression (which is CPU-intensive for a relatively weak system like the N36L) or dedup (which is extremely memory intensive)? Try adding these to loader.conf:


vm.kmem_size="1536M"
vm.kmem_size_max="2048M"
vfs.zfs.arc_min="256M"
vfs.zfs.arc_max="1024M"
#vfs.zfs.prefetch_disable="1"
#vfs.zfs.zil_disable="0"
vfs.zfs.txg.timeout="30"
vfs.zfs.vdev.max_pending="35"
vfs.zfs.vdev.min_pending="4"
vfs.zfs.txg.write_limit_override=1073741824

I'd also consider NexentaStor if I were you.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Saukkis posted:

Is any of the current 2TB eco-harddrives suitable for use with Linux software RAID? I vaguely remember that earlier WD Green Powers had some behaviours that made them not as good for RAID. I doubt the performance would matter much since the rest of the harddrives would be older models and the price difference to non-eco models is upto 40€.

I've got a bunch of Samsung F4's running with Linux software RAID just fine.

Maniaman
Mar 3, 2006
I run a small local computer repair shop. Our "file-server" consists of an old Dell consumer-grade desktop running Linux with a 1TB drive and a 320GB drive.

I'm running out of space on it, and it has no redundancy what-so-ever.

I'd like to get a NAS and set everything up in a RAID array. We have a SBS 2011 system now set up with AD, and I'd love to have something that can use AD accounts and permissions for the shares.

Any recommendations? I'd like to have at least 2TB of storage, if not more.

I'm sure I'm very much overthinking this as well.

what is this
Sep 11, 2001

it is a lemur
My recommendation is to just buy a Synology storage unit and bind it to active directory.

I would get a 710+ or 1511+

indulgenthipster
Mar 16, 2004
Make that a pour over
I originally posted this in the system building thread, but they referred me over to you guys:

I'm helping a friend who is a photographer. He has backups of all his pictures at his house, but wants a cost effective off-site backup solution. This is because recently a local photographers house burned down and he lost all backups in the fire (he kept good backups, just not at other locations).

Since he currently has 10TB of data and growing, the only way I could think of keeping costs down is to build a server, stuff it with hard drives, and hook it up at another friends house (then help pay for electricity and internet).

This doesn't need to be RAID5 or offer mirroring or anything like that since this machine is just backups of backups he already has, just at another location. I'm looking at a 16TB starting size for the server, which will then be expanded.

So right now I'm looking at a computer case that can hold 24 drives, or your suggestions. All of the pre-built NAS storage either seems relatively small, or very expensive.

Any thoughts?

IOwnCalculus
Apr 2, 2003





Why not run RAID on it? You don't want to have to back up 2TB of data -again- when a drive dies.

Also, I'm doing the same thing, except with Crashplan - I have my server here at home, and one stashed at my mom's place, both running Crashplan. I also paid for their online storage so all of the important stuff gets backed up both to the server there, as well as to their storage.

Galler
Jan 28, 2008


Check out Lime Tech's unRAID servers. Not so much to buy any of them but to get an idea on the hardware required to make a storage array that big.

MachinTrucChose
Jun 25, 2009
Assuming I am hit by the RAID5 write hole, I understand there will be data loss, but what is the extent of this data loss? Just the file that was being written/pending writes, or all my data? Google has no answer, everyone seems to just parrot "data loss!" over and over.

If it's the former, then I can stop giving a poo poo about the write hole. I'm storing every file myself by copying it with Windows Explorer to the NAS over Samba. If I lose power or the write fails for some reason, I'll know from a Windows Explorer dialog, and can just repeat the file copy operation when the situation is fixed.

indulgenthipster
Mar 16, 2004
Make that a pour over

IOwnCalculus posted:

Also, I'm doing the same thing, except with Crashplan - I have my server here at home, and one stashed at my mom's place, both running Crashplan. I also paid for their online storage so all of the important stuff gets backed up both to the server there, as well as to their storage.

Crashplan seems way to good to be true. Would I seriously be able to store an "unlimited" amount (10TB+) for $3 / month?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Galler posted:

Check out Lime Tech's unRAID servers. Not so much to buy any of them but to get an idea on the hardware required to make a storage array that big.

Fun fact, I've used that SuperMicro 5-in-3 enclosure in my first build, and the Icy Dock 5-in-3 and a 3-in-2, and I have aboslutely no technical prefernce between the two. I used the Icy Dock in my new build because they looked cooler. When I fill up my existing drives not sure if I'll buy another 5-in-3 for the case or just swap out the old drives for bigger ones.

Once this server isn't enough, I'll go crazy on a case with 9 5.25 bays, fill it with 3 5-in-3 enclosures, and jam two mirrored 2.5 drives in the back in a PCI slot.

Adbot
ADBOT LOVES YOU

Ceros_X
Aug 6, 2006

U.S. Marine
Coolermaster and Xigmatek both have relatively cheap ($25ish) 4in3 units. You need a screwdriver with a magnetic head to install drives in the Xigmatek one, (I own one) but that shouldn't be too much of a problem in the long term.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply