Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

D. Ebdrup posted:

What disks are you connecting to those 16 ports? Because if it's just platter drives, you won't get anywhere close to the 6Gbps that SAS offers while using port multipliers on the two SAS connectors that the IBM ServeRAID M1015 has.

It'll be a mix of platters and flash. The backplane has 3 SFF-8087 ports (12 actual drives) but 4-port cards seem pretty expensive so I'll probably just grab 2 H700s and call it a day.

Adbot
ADBOT LOVES YOU

Red Dad Redemption
Sep 29, 2007

How tough is the learning curve on FreeNAS? I'd like to build an NAS, I'm OK with the recommended hardware (e.g., ECC RAM for ZFS) from a price perspective, and I've built PCs, but I've never done server admin or handled FreeBSD or Unix OSs. Should I go for it or is discretion the better part of valor on this one?

theperminator
Sep 16, 2009

by Smythe
Fun Shoe
FreeNAS is pretty easy to use, once you install it you manage it through a web interface so you should be fine.

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.
After yesterday's drive failure, I'm fast tracking my plans to do a FreeNAS build yet this year, and will probably be ordering parts late today or tomorrow. Wondering if folks would mind giving it a quick once-over and telling me if there's anything screamingly obvious that's a poor choice here.


PCPartPicker part list / Price breakdown by merchant

CPU: Intel Xeon E3-1220 V3 3.1GHz Quad-Core Processor ($192.99 @ SuperBiiz)
Motherboard: Supermicro X10SL7-F Micro ATX LGA1150 Motherboard
Memory: Crucial 16GB (2 x 8GB) DDR3-1600 Memory ($113.99 @ Newegg)
Memory: Crucial 16GB (2 x 8GB) DDR3-1600 Memory ($113.99 @ Newegg)
Storage: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($169.89 @ OutletPC)
Storage: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($169.89 @ OutletPC)
Storage: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($169.89 @ OutletPC)
Storage: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($169.89 @ OutletPC)
Storage: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($169.89 @ OutletPC)
Storage: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($169.89 @ OutletPC)
Storage: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($169.89 @ OutletPC)
Storage: Hitachi Deskstar NAS 4TB 3.5" 7200RPM Internal Hard Drive ($169.89 @ OutletPC)
Case: Fractal Design Define R5 (Black) ATX Mid Tower Case ($107.99 @ SuperBiiz)
Power Supply: SeaSonic 450W 80+ Gold Certified ATX Power Supply ($63.98 @ Newegg)
Case Fan: Fractal Design GP14-WT 68.4 CFM 140mm Fan ($14.99 @ Amazon)
UPS: APC BE750G UPS ($82.35 @ Amazon)
Other: Supermicro Mobo ($245.95)
Total: $2295.35
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2015-08-24 08:19 EDT-0400


Really digging that Supermicro board for the built in IPMI and the fact that it has a built in LSI 2308 that can be flashed to IT mode very easily. (Yes, I realize it's on there twice. For some reason, they can't price it, so I added a manual entry for it on Amazon.) Went Xeon because I plan to use this for transcoding and running a small farm of VMs for teaching myself work-related stuff, and VT-D might save my butt, plus there's been confirmation that despite Intel's ARK entries saying otherwise for a while, some i3 chips don't actually support ECC. And there's no boot drive in here because I've got a handful of extra flash drives, several nice USB3 ones, and a small, spare SSD that I've been looking for an excuse to use.

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll
Nap Ghost
Only suggestion I could offer there is to consider Toshiba hard drives. They seem to have a good track record of reliability so far at least anecdotally and are probably even cheaper than Seagate drives.

Krailor
Nov 2, 2001
I'm only pretending to care
Taco Defender

G-Prime posted:

Wondering if folks would mind giving it a quick once-over and telling me if there's anything screamingly obvious that's a poor choice here.

FYI - 122X xeons don't support hyperthreading so if that's something you want you might want to consider moving up to a 1230 v3. It would only increase your total cost by about 2% and provide some benefit if you plan on doing a lot of transcoding or VM work.

mAlfunkti0n
May 19, 2004
Fallen Rib
I finally bit the bullet and bought a QNAP box that has enough power to do VMs and stuff. I am migrating to a pretty much all Mac setup and wanted to have something for time machine, etc.

Two 4TB WD RED drives will provide enough storage and the crucial stuff I'll either send to crashplan or amazon glacier. Will be nice to part ways with the desktop that just sits here holding disks.

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.
Swapping to an E3-1230v3. I can't seem to find clear data on the Toshiba drives. PH3400U-1I72 appears to be absurdly cheap ($120 each for 4TB? The hell?). Anybody know why?

Edit: I found that the OEM part number is MD04ACA400, so there's more data on that at least, still not clear why it's so cheap...

G-Prime fucked around with this message at 02:51 on Aug 25, 2015

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

G-Prime posted:

Swapping to an E3-1231. I can't seem to find clear data on the Toshiba drives. PH3400U-1I72 appears to be absurdly cheap ($120 each for 4TB? The hell?). Anybody know why?

Their running average price is more like $135, which is in line with Seagate and WD's 4TB desktop (non-NAS) offerings. That they're down to $120 is only something in the last few months.

Seems like a great deal, though.

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.
Alright, yeah, couldn't pass that up. That's $400 saved over 8 hard drives. Ordered. Now I just have to get a gigabit switch, some more ethernet cables, a bunch of SATA cables, and some reasonable spot in my home to put the drat thing. Thanks for the help, folks.

Don Lapre
Mar 28, 2001

If you're having problems you're either holding the phone wrong or you have tiny girl hands.
Best part about a nas, stick that poo poo in a closet

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.
If there were any closet in my place that I could actually reasonably get power and an ethernet cable to, I'd do exactly that. In this case, I think I'm going to get a small shelf, less than 6 inches tall (just so it's off the floor) and keep it in my basement. That's where all the other PCs are anyway, so the fan noise should be no big deal. When we move out of here and buy something (next year, fingers crossed), I'm making a point of finding us a house where I can either set up a full rack, or have a dedicated wiring/server closet.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Anyone else using a Rackable SE3016 JBOD expander? Had a power outage today that gave me a brief heart attack. The UPS shut everything down and then brought everything back up when power came back, but the array itself doesn't auto power-on and I needed to manually hit the power button and bounce FreeNAS for my volumes to reappear.

Short of whipping out the soldering iron and going all mad scientist, I'd like to see if there's an easier way to make the thing auto-poweron when power is applied.

Falco
Dec 31, 2003

Freewheeling At Last

mAlfunkti0n posted:

I finally bit the bullet and bought a QNAP box that has enough power to do VMs and stuff. I am migrating to a pretty much all Mac setup and wanted to have something for time machine, etc.

Two 4TB WD RED drives will provide enough storage and the crucial stuff I'll either send to crashplan or amazon glacier. Will be nice to part ways with the desktop that just sits here holding disks.

Sounds pretty similar to what I'm looking to do. Which model did you end up going with, and why QNAP over Synology?

UndyingShadow
May 15, 2006
You're looking ESPECIALLY shadowy this evening, Sir
I'm building a do-everything VMware virtualized system that will run PFSense, various test servers, and a FreeNAS instance. The onboard HBA on the supermicro board will be flashed to IT mode and passed straight to the FreeNAS VM. The SSDs and the HDDs will be mirrored for VMs and the 6 data drives for FreeNAS will be purchased at a later time.

PCPartPicker part list / Price breakdown by merchant

CPU: Intel Xeon E3-1231 V3 3.4GHz Quad-Core Processor ($242.99 @ SuperBiiz)
CPU Cooler: Cooler Master Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler ($24.99 @ Newegg)
Memory: Crucial 16GB (2 x 8GB) DDR3-1600 Memory ($123.99 @ Adorama)
Memory: Crucial 16GB (2 x 8GB) DDR3-1600 Memory ($123.99 @ Adorama)
Storage: Samsung 850 EVO-Series 250GB 2.5" Solid State Drive ($89.99 @ Amazon)
Storage: Samsung 850 EVO-Series 250GB 2.5" Solid State Drive ($89.99 @ Amazon)
Storage: Toshiba 4TB 3.5" 7200RPM Internal Hard Drive ($119.95 @ Adorama)
Storage: Toshiba 4TB 3.5" 7200RPM Internal Hard Drive ($119.95 @ Adorama)
Case: Fractal Design Define R5 (Black) ATX Mid Tower Case ($107.99 @ SuperBiiz)
Power Supply: SeaSonic S12G 650W 80+ Gold Certified ATX Power Supply ($66.50 @ Newegg)
Other: SUPERMICRO MBD-X10SL7-F-O uATX Server Motherboard LGA 1150 Intel C222 DDR3 1600 ($249.99)
Other: Dell X3959 Intel PRO/1000 PT Gigabit Dual Port Network Server Adapter PCIe NIC ($19.90)
Total: $1380.22
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2015-08-25 17:11 EDT-0400

Is this sane?

phosdex
Dec 16, 2005

When I ran esxi on my x10sl7 with a freenas vm, any attempt to get iscsi working from esxi to the vm would hang all networking and require a full reboot.

UndyingShadow
May 15, 2006
You're looking ESPECIALLY shadowy this evening, Sir

phosdex posted:

When I ran esxi on my x10sl7 with a freenas vm, any attempt to get iscsi working from esxi to the vm would hang all networking and require a full reboot.

Hmm. A couple of questions:

The FreeNAS VM is only for data storage on the rest of my network. I wasn't planning on having of the VMs talk to the FreeNAS instance(they'll all use virtual disks on the SSD and hard drives.) Is this only for VMs trying to use iSCSI to the FreeNAS VM?

Is this related to the onboard LSI card? Would an external card work better? The IBM M1015's are hella expensive now (double what they used to be.)

phosdex
Dec 16, 2005

oh, well you can't use the software raid on the motherboard for esxi.

dox
Mar 4, 2006

EpicCodeMonkey posted:

Why the crap can't they use a real package manager that can fix this sort of thing, instead of making their own lovely home-grown system?

Check out the SynoCommunity repo- might have what you need.

mAlfunkti0n
May 19, 2004
Fallen Rib

Falco posted:

Sounds pretty similar to what I'm looking to do. Which model did you end up going with, and why QNAP over Synology?

I went with the TS-253 Pro base model (ordered 2x4gb modules to upgrade it). I liked the CPU it included as it can do some light transcoding for Plex and also can run a couple of basic linux VMs for me. The virtualization is what won me over since there are some basic services I'd like to run in a small linux VM and I didn't want to have to keep my old desktop around.

Now to decide what to do with said desktop .. I might run ESXi on it and actually use my VMUG licenses I bought.

UndyingShadow
May 15, 2006
You're looking ESPECIALLY shadowy this evening, Sir

mAlfunkti0n posted:

I went with the TS-253 Pro base model (ordered 2x4gb modules to upgrade it). I liked the CPU it included as it can do some light transcoding for Plex and also can run a couple of basic linux VMs for me. The virtualization is what won me over since there are some basic services I'd like to run in a small linux VM and I didn't want to have to keep my old desktop around.

Now to decide what to do with said desktop .. I might run ESXi on it and actually use my VMUG licenses I bought.

Is VMUG worth it just for having a single ESXi server to mess with?

frunksock
Feb 21, 2002

Wow, a lot of you also have an X10SL7. Has anyone else had problems getting a Si3132 to work?

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

frunksock posted:

Wow, a lot of you also have an X10SL7. Has anyone else had problems getting a Si3132 to work?

I tried to make an ESXI 5.5 work with my Si3132 controllers, didn't go well. Supposedly, 5.0 will still work with them... but I just gave up and went with Windows Server 2012 R2 & hyper-v.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
I had a generic Marvell controller and followed this guide to enable it in ESXi 5.5, might help for that case as well.

mayodreams
Jul 4, 2003


Hello darkness,
my old friend
In case someone else has a similar config, FreeNAS 9.3.1 moves the drivers from supporting FW16 to FW20 for the LSI 9211-i8 and the other rebranded cards. After updating, it gives a yellow warning that there is a driver and firmware mismatch.

I ended up just passing the card through to my Server 2012 R2 vm and updating it and everything was golden.

Extremely Penetrated
Aug 8, 2004
Hail Spwwttag.
Does anyone have an opinion on which NAS vendor has the best central management software? I'm looking at procuring a few hundred devices to replace tape backups on physical file servers with lovely WAN links and my primary concern is ease of deployment and management. I just want to be able to have my field techs plug it in, load a standardized config, and point the backups to the new target. And then forget about them until an SNMP trap comes in.

UndyingShadow
May 15, 2006
You're looking ESPECIALLY shadowy this evening, Sir

mayodreams posted:

In case someone else has a similar config, FreeNAS 9.3.1 moves the drivers from supporting FW16 to FW20 for the LSI 9211-i8 and the other rebranded cards. After updating, it gives a yellow warning that there is a driver and firmware mismatch.

I ended up just passing the card through to my Server 2012 R2 vm and updating it and everything was golden.

So you're saying if I have an IBM M1015 or similar I need to update the firmware on the card to FW20?

mayodreams
Jul 4, 2003


Hello darkness,
my old friend

UndyingShadow posted:

So you're saying if I have an IBM M1015 or similar I need to update the firmware on the card to FW20?

If you move to 9.3.1 you do. I've been doing the updates since 9.3.0 and the 9.3.1 roll up was when they implemented the driver change after a long soak on the beta path. FW20 came out like last year, so its not bleeding edge or anything.

8-bit Miniboss
May 24, 2005

CORPO COPS CAME FOR MY :filez:

UndyingShadow posted:

So you're saying if I have an IBM M1015 or similar I need to update the firmware on the card to FW20?

If it's crossflashed as a 9211-8i, yes. There may be slight variations if it's crossflashed as another model.

UndyingShadow
May 15, 2006
You're looking ESPECIALLY shadowy this evening, Sir

phosdex posted:

oh, well you can't use the software raid on the motherboard for esxi.

Yeah, I didn't realize this and somehow missed your comment. Thanks for that info.

I'm trying to decide if I should just use the onboard LSI 2308 for raid 1 mirroring and then buy a M1015 for passthrough freenas. Anyone have something similar? Will it work?

phosdex
Dec 16, 2005

When I ran esxi on my x10sl7 I just lived dangerously and didn't raid my datastores. I used veeam (free license for personal use, you'll just get hounded by their sales guys for let's see 8 months running now) to backup to a freenas cifs share.

UndyingShadow
May 15, 2006
You're looking ESPECIALLY shadowy this evening, Sir

phosdex posted:

When I ran esxi on my x10sl7 I just lived dangerously and didn't raid my datastores. I used veeam (free license for personal use, you'll just get hounded by their sales guys for let's see 8 months running now) to backup to a freenas cifs share.

That's what I'm looking at now. I was thinking about doing mirroring in the GUEST os's themselves (making 2 datastores available for each VM, each on a different physical drive.) But now I'm reading some scary things that suggest that I NEED a raid card with a huge cache and a BBU or my ESXi performance will be garbage. This is all very confusing.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

UndyingShadow posted:

That's what I'm looking at now. I was thinking about doing mirroring in the GUEST os's themselves (making 2 datastores available for each VM, each on a different physical drive.) But now I'm reading some scary things that suggest that I NEED a raid card with a huge cache and a BBU or my ESXi performance will be garbage. This is all very confusing.

Wait, you mean doing something akin to software raid between two vmdk files on different datastores? That sounds awful. You don't have to have a card with BBU or giant cache, for home use you're probably not going to notice much of a difference. Depending on RAID layout, VM count, etc.

mAlfunkti0n
May 19, 2004
Fallen Rib

UndyingShadow posted:

Is VMUG worth it just for having a single ESXi server to mess with?

I bought the subscription when I had my R610 here and did nested hypervisors. I can do the same thing on my i5, having 3 hosts fully licensed and vcenter is nice since you can play around with everything .. oh and it's nice when the mega corp you work for doesn't have a lab, so I do a lot of testing here rather than in prod at work.

On another note, while the QNAP TS-253 pro is nice I will send it back with the ts-453 arrives .. two bays is a bit too limiting. Other than that, the QNAP setup is very nice, extremely quiet and plenty fast for my needs.

UndyingShadow
May 15, 2006
You're looking ESPECIALLY shadowy this evening, Sir

mAlfunkti0n posted:

I bought the subscription when I had my R610 here and did nested hypervisors. I can do the same thing on my i5, having 3 hosts fully licensed and vcenter is nice since you can play around with everything .. oh and it's nice when the mega corp you work for doesn't have a lab, so I do a lot of testing here rather than in prod at work.

Wait, so you can run hypervisors within hypervisors? Man, I've got a lot of stuff to mess with.

Moey
Oct 22, 2010

I LIKE TO MOVE IT

UndyingShadow posted:

Wait, so you can run hypervisors within hypervisors? Man, I've got a lot of stuff to mess with.

Inception!

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Moey posted:

Inception!

They do get 20x slower each level though.

mAlfunkti0n
May 19, 2004
Fallen Rib

UndyingShadow posted:

Wait, so you can run hypervisors within hypervisors? Man, I've got a lot of stuff to mess with.

Sure can, it's called "nested". You need to make sure that when you create the VM that will run the hypervisor you expose the virtualization to it as well.

http://vmw.re/1F3zkAE

Skandranon posted:

They do get 20x slower each level though.

One can argue that at this point nested virtualization is only for lab type stuff to get to know the product, so speed isn't really that large of a problem.

mAlfunkti0n fucked around with this message at 19:50 on Aug 28, 2015

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.
Think I'm finally done with my build. Did my RAM burn-in last night, and everything went swimmingly. Put in the hard drives today...... and errors on two of the drives. Did some digging, swapped the SATA cables, all better now. 7/9 working cables from Monoprice is, truthfully, better than I expected. Now, we begin the long, brutal process of doing burn-in on the drives.

Adbot
ADBOT LOVES YOU

mAlfunkti0n
May 19, 2004
Fallen Rib
Decided against the QNAP and it is going back. The proprietary weird linux build installed can make for some headaches. I have an i5 w/16GB ram laying around that I was going to sell but I've since installed unRAID on it and really like it. Docker containers, KVM for virtualization and the storage system works well.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply