Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

fatman1683 posted:

How would one go about building an eSATA DAS? I'd like to have an external RAID box connected to my ESXi host, but the cheap off-the-shelf options don't have any kind of useful management interface, and there's a huge gap between the $200 models and the $2000+ models where there's basically nothing.

Presumably you'd need some kind of controller card in the storage box that can act as a disk and present the array in the box to the host's controller, but I have no idea if such a thing exists as a standalone product, and I haven't found anything that advertises that capability.

Anyone know if this is possible?

Plenty of these do that:
http://www.newegg.com/RAID-Enclosure-Subsystems/SubCategory/ID-509

There are two ways of doing it. One is the way you describe where the external box builds a raid set and exposes a single volume to the machine. The other uses a port multiplier and passes all the drives through the single eSATA channel. You probably want the first one. I don't have any experience with any of them, but SANS Digital sure does make a wide variety.

Adbot
ADBOT LOVES YOU

PitViper
May 25, 2003

Welcome and thank you for shopping at Wal-Mart!
I love you!
I think that depends. Do you want the box to handle the RAID management, or one of the VMs? I'm running ZFS, and my case is out of internal bays. I've looked at a few of the Sans Digital boxes to just pass raw disks via port multiplication and eSATA so I can use ZFS to manage the disks.

Currently I'm trying to figure out if its more cost effective to buy one of the big Norco cases, or if one or two Sans Digital boxes are a more cost-effective option.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
I've got an LSI 1068E card, and I think it might be bad. Is there a good exhaustive way I can test it before I just give up and buy an M1015?

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
Someone tell me what the difference is between shutting down a computer properly and just yanking the power cable from a disk.

evol262
Nov 30, 2010
#!/usr/bin/perl

Combat Pretzel posted:

Someone tell me what the difference is between shutting down a computer properly and just yanking the power cable from a disk.

What is cache flushing, filesystem syncing, Alex?

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
I've disabled the write cache on the SATA port. I guess I should have specified the mechanical and maybe even electronic side of the action.

fatman1683
Jan 8, 2004
.

PitViper posted:

I think that depends. Do you want the box to handle the RAID management, or one of the VMs?

Basically I want to have a standalone box that creates the array and presents it to the hypervisor as a single volume. Then I can RDM it to the fileserver guest, and have a native filesystem on the storage box that isn't inside a datastore. This'll make recovery easier if the hypervisor ever dies, or if I just want to upgrade the hyp without having to do a host-side data migration.

And yes, I'm aware that there are off-the-shelf solutions that do this. However, the inexpensive ones I've found have no management at all, and basically only allow you to create a single array from a set of buttons on the front of the box. The expensive, enterprise-grade DASes do have management, naturally, but there's no middle-ground of reasonably priced storage enclosures with enough management to make them useful.

So if I could figure out how the 'device' side of a SATA interface is served and find a way to put a RAID array behind it, I could build my own storage server with an OS that presents to the hyp as a single volume over eSATA.

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
What is emptying the SATA command queue and backcurrent protection in hotswap SATA ports?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Are WD Reds still the hot poo poo when it comes to NAS drives? Newegg has a deal today only where they're $135 a piece.

Crossbar
Jun 16, 2002
Chronic Lurker

FISHMANPET posted:

Are WD Reds still the hot poo poo when it comes to NAS drives? Newegg has a deal today only where they're $135 a piece.
Thanks for the heads up, I just bought 4.

Anyone know of a mini-itx board with at least 6 sata ports that supports vt-d?

Crossbar fucked around with this message at 23:17 on Apr 16, 2013

IOwnCalculus
Apr 2, 2003





FISHMANPET posted:

Are WD Reds still the hot poo poo when it comes to NAS drives? Newegg has a deal today only where they're $135 a piece.

I don't think there's any other game in town. I just bought three 2TB ones yesterday when they were $100/ea after coupon; I'm not out of space, I just need to replace some of my 30,000-hour 1.5TB drives.

Tivac
Feb 18, 2003

No matter how things may seem to change, never forget who you are
I used to have one of the HP MediaSmart machines running WHSv1 & it was glorious. That box's mobo died on me & I have been pining away for a replacement ever since.

I really don't want to gently caress around w/ my data in Linux, and the drive pooling stuff in WHS was amazing. I know there's a few 3rd party options out there now for doing that, do any of them work?

Synology NAS boxes seem like they'd be nice but the custom filesystem & price are turning me away. Would a custom mini-itx system be reasonable?

Sorry, lots of somewhat undirected questions. From my perusing most of the folks in this thread are running linux-based systems so we'll see if this is even really the right place to be asking.

Splinter
Jul 4, 2003
Cowabunga!
I'm looking for backup software recommendations.

My house has combined HTPC/server (Win7) that all the roommates share. I want to start backing up my personal computers (Mac & Win7) to the server, but I'd like have the files on the server encrypted so my roommates can't browse through all my files while using the HTPC. I imagine the easiest way to do this is to have backup software running on my personal computer that encrypts the files as it tosses them onto the server. I'd also like the backup to either be scheduled (e.g. once a day/week) or manually triggered, rather than a live continuous backup (I don't want the backup service/program running in the background at all times).

What backup programs can do this relatively painlessly?

I'm also planning on having the server mirroring the encrypted backup online using either Backblaze or CrashPlan (both of which will encrypt the data again I believe). Are there any issues that could come up when using either service with already encrypted data?

IOwnCalculus
Apr 2, 2003





Crashplan can be used for free to back up between computers, so it will work for everything you're talking about - your friends can all sign up for their own accounts and then you can share your server as a backup destination. You won't be able to see their files, they won't be able to see yours.

McGlockenshire
Dec 16, 2005

GOLLOCKS!
e:f;b, but still:

Splinter posted:

I want to start backing up my personal computers (Mac & Win7) to the server, but I'd like have the files on the server encrypted so my roommates can't browse through all my files while using the HTPC.

Crashplan can do this. Set up the server with it's own Crashplan account, then send "friend" codes to the client machines, each of which is a different Crashplan account. Because every machine is a different user, nobody can see anyone else's files, and everything's encrypted anyway. The client sets the backup schedule, so there's no major concern about the server running the backup bits 24/7.

FCKGW
May 21, 2006

Tivac posted:

I used to have one of the HP MediaSmart machines running WHSv1 & it was glorious. That box's mobo died on me & I have been pining away for a replacement ever since.

I really don't want to gently caress around w/ my data in Linux, and the drive pooling stuff in WHS was amazing. I know there's a few 3rd party options out there now for doing that, do any of them work?

Synology NAS boxes seem like they'd be nice but the custom filesystem & price are turning me away. Would a custom mini-itx system be reasonable?

Sorry, lots of somewhat undirected questions. From my perusing most of the folks in this thread are running linux-based systems so we'll see if this is even really the right place to be asking.

I moved from a home built WHSv1 box to a HP N40L with WHS 2011 and Stablebit Drive Pool and couldn't be happier.

Splinter
Jul 4, 2003
Cowabunga!
Sounds like CrashPlan will work.

One question: does CrashPlan allow you to backup a "friends" backup online without paying for multiple computers? I gather I'd be running CrashPlan free on my personal machine, and CrashPlan+ on the server (since I want the online backups sent from the server). From the server, can I select my personal machine's backup as normal files to backup online, or will CrashPlan recognize it is another user's backup and require the family plan?

FCKGW
May 21, 2006

You can backup whatever with CrashPlan+. I'm doing the exact scenario you are explaining right now: 3 machines back up server, server backs up everything online.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Do I need anything special for a SATA to USB adapter to use with a 3TB hard drive? I was looking at this but a review says it didn't work with a 3TB drive.

Sub Rosa
Jun 9, 2010




Tivac posted:

Synology NAS boxes seem like they'd be nice but the custom filesystem & price are turning me away.
You don't have to use the custom filesystem. Also the custom filesystem is just ext4 on top of mdadm software raid anyway. It works by making multiple raid sets and concatenating them. For example with two 3TB, two 2TB, and two 1TB disks, you'll have a Raid 5 spanning all six, a Raid 5 spanning the four biggest, and a Raid 1 on the 3TBs. Presuming SHR1, anyway, with SHR2 it's the same concept but everything is Raid 6.

Also since it's basically a customized linux distribution they publish their source code modifications and people have been able to compile drivers for extra hardware and get it working on third party devices such as the MicroServer. Similar concept as the Hackintosh.

IT Guy
Jan 12, 2010

You people drink like you don't want to live!
never mind.

IT Guy fucked around with this message at 17:55 on Apr 17, 2013

Megaman
May 8, 2004
I didn't read the thread BUT...
How is everyone mostly backing up their multi terebyte freenas setups? Copying on to separate non raided disks and storing them in a closet? Backing up to s3? Both? I need to back up my freenas setup since I have 0 backups since I've created it and it's almost full, it's under 4 TB

IT Guy
Jan 12, 2010

You people drink like you don't want to live!

Megaman posted:

How is everyone mostly backing up their multi terebyte freenas setups? Copying on to separate non raided disks and storing them in a closet? Backing up to s3? Both? I need to back up my freenas setup since I have 0 backups since I've created it and it's almost full, it's under 4 TB

I have two identical servers sitting side by side using ZFS replication with FreeNAS. It's not offsite but then I'm not going to worry about movies/music/tvshows if I have a fire and offsite is just too expensive. I keep all important documents and stuff on Google Drive and Drop box.

Megaman
May 8, 2004
I didn't read the thread BUT...

IT Guy posted:

I have two identical servers sitting side by side using ZFS replication with FreeNAS. It's not offsite but then I'm not going to worry about movies/music/tvshows if I have a fire and offsite is just too expensive. I keep all important documents and stuff on Google Drive and Drop box.

I assume zfs replication is more of a wise choice than periodic rsync?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Megaman posted:

How is everyone mostly backing up their multi terebyte freenas setups? Copying on to separate non raided disks and storing them in a closet? Backing up to s3? Both? I need to back up my freenas setup since I have 0 backups since I've created it and it's almost full, it's under 4 TB

Crashplan.

movax
Aug 30, 2008

Megaman posted:

How is everyone mostly backing up their multi terebyte freenas setups? Copying on to separate non raided disks and storing them in a closet? Backing up to s3? Both? I need to back up my freenas setup since I have 0 backups since I've created it and it's almost full, it's under 4 TB

I don't :( But I need too, because my vdev utilization is all out of whack as well (due to adding one at a time):
code:
movax@megatron -  ~ [ zpool iostat -v                             ] 12:54:19 PM
               capacity     operations    bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
tank        28.9T  3.70T     60      2  7.48M   133K
  raidz2    10.9T  16.0G     22      0  2.77M  12.0K
  raidz2    10.9T  17.2G     22      0  2.78M  11.9K
  c3t1d0p1   276K  11.1G      0      0      0      0
  raidz2    7.20T  3.67T     15      1  1.93M   109K
cache           -      -      -      -      -      -
  c3t1d0p2  44.1G     8M      0      1  13.1K   214K
Just need somewhere to copy a fuckload of files :downs:

IT Guy
Jan 12, 2010

You people drink like you don't want to live!

Megaman posted:

I assume zfs replication is more of a wise choice than periodic rsync?

Apparently ZFS replication knows your file structure better since it's block-level rather than rsync where it's file-level. That gives it the benefit of knowing about the snapshots and whether to duplicate the data. For example, if you cloned a dataset, it shouldn't transfer it but rsync would. At least that's my understanding of it from the Google searching I did.

Secx
Mar 1, 2003


Hippopotamus retardus
Are the WD Reds really worth the premium for a NAS? I will run an Unraid setup and the drives will only store movies. So the only times the drives should be spinning is when I'm streaming them to my HTPC or I'm ripping/copying my movies over the network.

I was thinking of getting 4 of these Seagate external and just rip out the internal drive: http://www.newegg.ca/Product/Product.aspx?Item=N82E16822178117

Good idea or worst idea ever?

IT Guy
Jan 12, 2010

You people drink like you don't want to live!

Thermopyle posted:

Crashplan.

What plan do you use?

http://www.crashplan.com/consumer/compare.html

The Free has a big X through "Online storage" and the about page doesn't say anything about the limits.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

IT Guy posted:

What plan do you use?

http://www.crashplan.com/consumer/compare.html

The Free has a big X through "Online storage" and the about page doesn't say anything about the limits.

Crashplan+ Family Unlimited.

The storage is unlimited.

If you just want to backup your server online, I think Crashplan+ Unlimited will work.

The biggest issue is getting Crashplan running on a headless machine, but it's not too bad. Well, depending on your hardware, the memory usage of the client might be the biggest issue.

UndyingShadow
May 15, 2006
You're looking ESPECIALLY shadowy this evening, Sir

Thermopyle posted:

Crashplan+ Family Unlimited.

The storage is unlimited.

If you just want to backup your server online, I think Crashplan+ Unlimited will work.

The biggest issue is getting Crashplan running on a headless machine, but it's not too bad. Well, depending on your hardware, the memory usage of the client might be the biggest issue.

Has anyone figured out a way to actually get crashplan to backup a freenas server that doesn't involve symlink style hacks?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

UndyingShadow posted:

Has anyone figured out a way to actually get crashplan to backup a freenas server that doesn't involve symlink style hacks?

I'm sorry, I made an assumption that maybe isn't merited? I kind of just assumed that of course Crashplan would work on a freenas machine.

IT Guy
Jan 12, 2010

You people drink like you don't want to live!

Thermopyle posted:


If you just want to backup your server online, I think Crashplan+ Unlimited will work.


That price just seems too good to be true for some reason.

How much data are you backing up?

Would they seriously let me backup ~10TB?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

IT Guy posted:

That price just seems too good to be true for some reason.

How much data are you backing up?

Would they seriously let me backup ~10TB?

I've been slowly adding more storage to what I'm backing up since the seed backup takes forever. I'm probably at 4+ TB. I think someone in this thread said they're at 8TB.

Don Lapre
Mar 28, 2001

If you're having problems you're either holding the phone wrong or you have tiny girl hands.

IT Guy posted:

That price just seems too good to be true for some reason.

How much data are you backing up?

Would they seriously let me backup ~10TB?

Yes, its just gonna take a hell of a long time, and if you are on a residential internet account lots of them have monthly bandwidth limits.

And their loving seed drive service is limited to 1tb.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

movax posted:

Just need somewhere to copy a fuckload of files :downs:

We need a community NAS that we can mail around to each other for when we need to do upgrades/maintenance!

(I know this would never actually work, for so many reasons)

Lowen SoDium
Jun 5, 2003

Highen Fiber
Clapping Larry

UndyingShadow posted:

Has anyone figured out a way to actually get crashplan to backup a freenas server that doesn't involve symlink style hacks?

http://www.bionoren.com/blog/2013/03/freenas-crashplan/

I am actually going to try this later today

edit: I am never going to try this. It's more involved than I thought it was at first.

Lowen SoDium fucked around with this message at 19:18 on Apr 17, 2013

Mr Crucial
Oct 28, 2005
What's new pussycat?

Secx posted:

I was thinking of getting 4 of these Seagate external and just rip out the internal drive: http://www.newegg.ca/Product/Product.aspx?Item=N82E16822178117

Good idea or worst idea ever?

You'll void the warranty on the drives by ripping them out. Depending on the exact construction of the enclosure you may be able to put a faulty drive back inside and RMA it without them noticing, but its a bit of a risk to take to save yourself a bit of money.

poverty goat
Feb 15, 2004



Since I couldn't find anyone to tell me it wouldn't work, I grabbed a second dual-port PRO/1000 card from amazon for $50 to play with MPIO/iSCSI and see if I could make worthwhile gains over 1gbps for a single data stream. My desktop and my NAS are next to each other on the desk, and each is connected to the gigabit switch on a linksys e2000 with its onboard gigabit NIC (iSCSI doesn't touch this network). Each machine has an additional, identical dual-port Pro/1000 card, and the two machines are doubly crossed over with them with each pair on its own subnet. It wasn't too difficult to set up iSCSI via a guide, mount the disk in windows, and mash buttons until I figured out how to enable MPIO from that end.

(This is freeNAS with a 5 disk RAIDZ2 and fast SSDs for cache and ZIL.)

Also, anyone know if there's a way to do iSCSI without actually thick provisioning a virtual disk for it? Is there any way to just share my existing data as an iSCSI drive?

e: Things seem to be working more or less as intended. CrystalDiskMark on the mounted drive consistently reports 136MB/s for sqeuential reads no matter how I configure the test. This seems to be consistent with the speeds I'm getting on large (>4gb) transfers that aren't cached. Smaller transfers from the cache are clearly faster but I'm not sure what's the best way to benchmark it. I tried copying a 4gb windows .iso back and forth between samsung 830 and NAS and it kept completing in 6-7 seconds which isn't even possible inside of 2gbps. What the christ.

poverty goat fucked around with this message at 23:14 on Apr 17, 2013

Adbot
ADBOT LOVES YOU

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

gggiiimmmppp posted:

Also, anyone know if there's a way to do iSCSI without actually thick provisioning a virtual disk for it? Is there any way to just share my existing data as an iSCSI drive?

This is not possible. iSCSI is a block level protocol, it shares at the block level, not the file level.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply