|
Docjowles posted:A+ rant, would Yeah, no exaggeration sadly. Fortunately the other 2/3 of the estate is HDS/EMC and not half as much trouble.
|
#
?
Jun 30, 2014 23:06
|
|
|
|
| # ? May 21, 2024 15:48 |
|
|
Richard Noggin posted:The amount of time that a host spends in maintenance mode is very, very small. Wouldn't you rather take that other $5k that you would have spent on a server and put it into better storage/networking? How many small clusters do you have running essentials plus? Can you add those hosts to a Standard vCenter server or are you running a vCenter server per cluster? We are not too strapped for budget here (or at my old place) so that hasn't been a problem. What if one of your hosts blows a motherboard? Putting all that faith in the remaining host? Edit: Also Zephirus, I will never complain about storage again.
|
|
#
?
Jun 30, 2014 23:42
|
|
|
Richard Noggin posted:The amount of time that a host spends in maintenance mode is very, very small. Wouldn't you rather take that other $5k that you would have spent on a server and put it into better storage/networking? I'd rather not lose half of my compute and all of my redundancy when I have a component in a server die. There are arguments for both scale up and scale out, but three nodes is the minimum that I would deploy for production.
|
|
#
?
Jun 30, 2014 23:52
|
|
|
Richard Noggin posted:The amount of time that a host spends in maintenance mode is very, very small. Wouldn't you rather take that other $5k that you would have spent on a server and put it into better storage/networking? How much does the VNXe3150 run around? What separates it from the Dell MD3820i that I posted? Is it the dual storage processors? Adding this feature to the dell brings it up to $16,500. That's with 12 900TB 10K RPM drives. Is this more in line with a disk array that would be worth investing in?
|
|
#
?
Jul 1, 2014 00:01
|
|
|
sudo rm -rf posted:How much does the VNXe3150 run around? What separates it from the Dell MD3820i that I posted? Is it the dual storage processors? Adding this feature to the dell brings it up to $16,500. That's with 12 900TB 10K RPM drives. Is this more in line with a disk array that would be worth investing in? The VNXe will be much more feature rich. Also, don't buy an array without dual controllers. Non-disruptive failover is the biggest advantage of centralized storage.
|
|
#
?
Jul 1, 2014 00:07
|
|
|
Richard Noggin posted:Just to give everyone an idea, here's our "standard" two host cluster: I tend to agree that running multiple small clusters is non-optimal. You have to reserve 1 host's worth -- or 1/n of the workload where n = number of hosts -- of capacity in the cluster in case of failure. The bigger your clusters get, the more efficient you are. It is not efficient to run a bunch of 2-node clusters at 50% util, compared to one big cluster at (n-1)/n percent util. You also lose efficiency from all those unnecessary array controllers and switches. This is not how anyone sane scales out.
|
|
#
?
Jul 1, 2014 00:43
|
|
|
Zephirus posted:Storage trainwreck. This post needs to be in the OP. I lost it when I saw the puke-green CLI interface.
|
|
#
?
Jul 1, 2014 11:58
|
|
|
KS posted:I tend to agree that running multiple small clusters is non-optimal. You have to reserve 1 host's worth -- or 1/n of the workload where n = number of hosts -- of capacity in the cluster in case of failure. The bigger your clusters get, the more efficient you are. It is not efficient to run a bunch of 2-node clusters at 50% util, compared to one big cluster at (n-1)/n percent util. We don't run multiple small clusters. This would be per-client. Our clients, just like a lot of posters here, don't have unlimited resources, so it's a bang for the buck type of thing. The choice between better hardware and a server that sits idle all day long is pretty much a no-brainer.
|
|
#
?
Jul 1, 2014 12:35
|
|
|
Richard Noggin posted:We don't run multiple small clusters. This would be per-client. Our clients, just like a lot of posters here, don't have unlimited resources, so it's a bang for the buck type of thing. The choice between better hardware and a server that sits idle all day long is pretty much a no-brainer. If you don't have the equivalent, resource wise, of one server sitting idle then your cluster isn't going to survive a host failure gracefully. The problem is that in your cluster that means 50% of resources need to be reserved, whereas for larger clusters it's a smaller percentage for n+1 protection. Your logic seems exactly backwards to me.
|
|
#
?
Jul 1, 2014 18:17
|
|
|
NippleFloss posted:If you don't have the equivalent, resource wise, of one server sitting idle then your cluster isn't going to survive a host failure gracefully. The problem is that in your cluster that means 50% of resources need to be reserved, whereas for larger clusters it's a smaller percentage for n+1 protection. In large environments, yes, the design does not make sense. The majority of our clients don't fall into that segment. Given a fixed budget for a high-availability setup, we have simply chosen to go quality over quantity. The workload can happily run on one host, so we have redundancy covered. For workloads that can't, then yes, three hosts makes sense.
|
|
#
?
Jul 1, 2014 18:51
|
|
|
NippleFloss posted:If you put some SSD in your ESX hosts and configure it as read cache your budget SAN config will go a lot further. This will be my next blog post. Debunking the hype behind vSAN. In all honesty it's an over priced product that is way more over hyped than it should be. It's cool and all but, the limitations and cost of long term overship far outweigh the cost of a real SAN. sudo rm -rf posted:How much does the VNXe3150 run around? What separates it from the Dell MD3820i that I posted? Is it the dual storage processors? Adding this feature to the dell brings it up to $16,500. That's with 12 900TB 10K RPM drives. Is this more in line with a disk array that would be worth investing in? Just order this https://drive.google.com/file/d/0Bz6GQvhNTfo7Tkg1dmxfSjFrdnc/edit?usp=sharing If you separated your boot/data drives then IOP's shouldn't be a huge issue for your environment. It's dual controller and 2GB cache shared between read/writes it should handle your environment well. Dilbert As FUCK fucked around with this message at 01:20 on Jul 2, 2014 |
|
#
?
Jul 2, 2014 01:04
|
|
|
Dilbert As gently caress posted:This will be my next blog post. Can you compare it or show other parallels with vSAN and EMC ScaleIO and other scale out storage systems as well?
|
|
#
?
Jul 2, 2014 01:20
|
|
|
Wicaeed posted:Can you compare it or show other parallels with vSAN and EMC ScaleIO and other scale out storage systems as well? Uhh does ExtremeIO count? That's what I am working with right now, and actively pursuing to counter because how immature the product is... But sure. I don't know what parallels I can show, but within the next week, check howtovirtual.com for some kind of post.
|
|
#
?
Jul 2, 2014 01:22
|
|
|
Dilbert As gently caress posted:Uhh does ExtremeIO count? That's what I am working with right now, and actively pursuing to counter because how immature the product is... But sure. I don't think ExtremeIO and ScaleIO really do the same thing? ExtremeIO seems to be a fairly straight forward flash based array while ScaleIO is more a software defined, scale out storage system. For those of you with Nimble devices: Does Nimble have something similar to Equallogic SynchRep? SynchRep allows you to synchronize writes to two volumes across two (or more) separate Equallogic arrays. It allows for a fairly high degree of failover and I'm currently looking at it for a MSSQL cluster. Was curious if Nimble does anything like this.
|
|
#
?
Jul 9, 2014 22:49
|
|
|
My VNX 5200 showed up yesterday and I already wish I had a 2nd one to replace our VNXe 3300. The VNXe was easier to set up initially, but the stripped down interface started to bother me over time. Especially the missing performance related info/reporting.
|
|
#
?
Jul 10, 2014 14:52
|
|
|
Cavepimp posted:My VNX 5200 showed up yesterday and I already wish I had a 2nd one to replace our VNXe 3300. The VNXe was easier to set up initially, but the stripped down interface started to bother me over time. Especially the missing performance related info/reporting. Did you purchase the monitoring suite for the VNX? Without it you don't get that info. I also really hate how EMC charges an arm and a leg just to be able to view performance info.
|
|
#
?
Jul 10, 2014 15:55
|
|
|
Richard Noggin posted:Did you purchase the monitoring suite for the VNX? Without it you don't get that info. I also really hate how EMC charges an arm and a leg just to be able to view performance info. I had to go double check when you posted this, but yes, we did. Phew. I hate the nickel and dime stuff usually, but in this case I'm perfectly happy to pick and choose the things we actually need instead of overpaying for some huge bundle.
|
|
#
?
Jul 10, 2014 17:45
|
|
|
Wicaeed posted:I don't think ExtremeIO and ScaleIO really do the same thing? ExtremeIO seems to be a fairly straight forward flash based array while ScaleIO is more a software defined, scale out storage system. You can synchronize volumes to another array, yes.
|
|
#
?
Jul 10, 2014 18:53
|
|
|
Just spent the past week dealing with building at setting up some dell MD's, Gotta say the 3620i 10G is a tight little box, even without the SSD caching this box is awesome. Stupid simple to setup and runs like a champ, my biggest gripe still remains no software button to initiate a shutdown just switches on the back of the box, but whatever. It also sucks that there is no raid 50 or raid 60 but for the price that's a minor complaint.
|
|
#
?
Jul 10, 2014 19:08
|
|
|
Nitr0 posted:You can synchronize volumes to another array, yes. Is this more of a take a snapshot and then replicate the info setup, or some thing where any writes made to one array are also replicated to the second array and verified before any confirmation is actually sent to the machine that did the writing?
|
|
#
?
Jul 10, 2014 19:45
|
|
|
Second scenerio
|
|
#
?
Jul 10, 2014 20:09
|
|
|
We discovered recently that our oracle array has disks which are consistently at or above 95% utilization. It's amazing that the array was able to mask this kind of problem from us for long enough to get to this point. To make sure I wasn't lying to my users I've been running in VDI since I noticed the issue while I work on quotes for more storage, and other than the big logon push in the morning the system stays pretty drat usable.
|
|
#
?
Jul 10, 2014 22:39
|
|
|
adorai posted:We discovered recently that our oracle array has disks which are consistently at or above 95% utilization. It's amazing that the array was able to mask this kind of problem from us for long enough to get to this point. To make sure I wasn't lying to my users I've been running in VDI since I noticed the issue while I work on quotes for more storage, and other than the big logon push in the morning the system stays pretty drat usable. We've been burned a few times by Oracle storage (7120) and it got sent down to the minors where it can be a non-critical unit. We have a VNX 5200 instead, and love it. A shame too - the 7120 was ridiculously expensive for what we got.
|
|
#
?
Jul 10, 2014 23:06
|
|
|
Richard Noggin posted:We've been burned a few times by Oracle storage (7120) and it got sent down to the minors where it can be a non-critical unit. We have a VNX 5200 instead, and love it. A shame too - the 7120 was ridiculously expensive for what we got.
|
|
#
?
Jul 11, 2014 01:37
|
|
|
Does anyone use LSI MegaRAID Storage Manager any more? We have probably 500 or so endpoints that currently use this software. I was wondering LSI has a product that can be used to administer (and deploy configs to) all of these endpoints en masse. Googling hasn't really been of any help thus far.
|
|
#
?
Jul 11, 2014 21:34
|
|
|
Wicaeed posted:Does anyone use LSI MegaRAID Storage Manager any more? What information are you looking from the cards exactly? Like drive failure, write cache, utilization?
|
|
#
?
Jul 13, 2014 01:48
|
|
|
Anyone gone Netapp -> Nimble? We've historically been a Netapp shop, but we've got a small site where we inherited an Equallogic that's on it's way out and we're considering plopping a Nimble there to replace it instead of the gimped out 2520 config we would need to end up at a similar price/performance point. Mainly just curious what people's thoughts are from a "oh god I miss this SO MUCH" standpoint.
|
|
#
?
Jul 14, 2014 18:02
|
|
|
Maneki Neko posted:Anyone gone Netapp -> Nimble? We've historically been a Netapp shop, but we've got a small site where we inherited an Equallogic that's on it's way out and we're considering plopping a Nimble there to replace it instead of the gimped out 2520 config we would need to end up at a similar price/performance point. What Nimble are you looking at and what's the 2520 config look like?
|
|
#
?
Jul 14, 2014 18:28
|
|
|
I'm bored so I'm rewriting stuff T=Enterprise Storage Megathread: Just add Flash 1.What is storage and why is it important 2.How is some storage different than others 3.Delivering storage to servers 4.Data Management 5.Backup data Storage is the lifeblood of all business networks, without utilizing some form of storage all the servers, switches, and IT personnel are nothing. Storage is not only one of the most important parts of networks but it is also the most overlooked, and improperly managed or designed. This can cause issues maintaining space for company data, providing data fast enough for company needs, or redundancy against drive failures. Improperly configuring storage will essentially cause your whole company and systems to feel the bottleneck of it, and with seconds adding to dollars it is imperative to have the solution that fits best for your company. 2. What storage is right for me? There are various kinds of storage, some locally built into the server, some specially made servers dedicated to running storage services, and even systems that aggregate large amounts of storage from various servers and services. To know which one you need you need to ask yourself a few basic questions:
At the basic level some of these questions can be answered by determining your RAID level, regardless if you you DAS or SAN you probably will still run into something regarding raid levels; however as time goes with the flood of SSD technology into storage, the RAID level can become less well known or even configurable. When calculating raid levels it is important to understand the impact on IOPS that it has. code:Calculating RAID RAID 0 - This is a suicide option that should only be used for non critical data, it carries no write penalty and has better performance the more disks you add; the catch being if you lose one disk all you data is lost. A use case for this might be if you have a system that needs to dump out large amounts of data very fast, that is not mission critical or important to anyone. RAID 1 - This option provides redundancy of 2 disks by mirroring data between them, this allows for a disk to fail while everything continues to operate normally, it has a write penalty of 2 meaning that data must be written twice(or each write to their own drive) before the write is completed. Performance is mainly that of a single drive however you do gain redundancy if a drive fails. A use case would be for an OS volume, where the loss of a drive will not cause the system to go down. While size is not increased over the volume size, raid one is best used where data needs to be kept available. RAID 10 - RAID 10 combines the benefits of RAID 0 and RAID 1 in one raid type. This is done by creating multiple raid 1 volumes and then striping data on top of them. It allows for multiple drive failures, HOWEVER, it does not allow for two drive failures in the same raid 1 volume meaning; RAID 10 ----------------------- |A1xA2| |B1xB2| |C1xC2| ----------------------- in this raid 10 set comprising of raid 1 A,B,C volumes if A1, B2, and C1 failed the volume would still be operational; however if A1 and A2 failed the volume would be lost due to how the RAID 0 striping is occurring over the volumes. Additional a Raid 10 volume will only have 50% usable space as the mirror of the drives will reduce the total usable similar to raid 1, however the individual raid 1 volumes are aggregated similar to how RAID 0 increases space. RAID 10 has multiple use cases however it works best for databases, high transactional storage needs such as busy OS drives, and VDI. RAID 5 - One of the most commonly used raid due to it's reputation of being data redundancy, while providing increased storage space for data. It while it does provide redundancy and increased storage space, the draw back is that it comes with additional write penalties meaning data written to disk will take longer as data must be written in parity to each disk in the volume. Example: Raid 5 ------------ |_||_||_||_| |_||_||_||_| |_||_||_||_| |_||_||_||_| ------------ If a VM decides it needs to write "X" to disk it must write it to all disks meaning it must complete a write penalty of 4 in this situation, this is what allows RAID 5 to be fault tolarant but also what causes it to be slower than some other kinds of RAID levels. ------------ |X||_||_||_| |_||X||_||_| |_||_||X||_| |_||_||_||X| ------------ Here we see X being written to the raid 5 Array. While RAID 5 is best used with things like Exchange DB's, File Servers, Application servers(non OS or program partition) that are primarily read intensive and where larger size is required and RAID 10 carries too large a capacity loss. Raid 5 can sustain 1 Drive failure at a time, and requires an overhead of 1 disk size. RAID 6 Raid 6 is very similar to raid 5 but has a dual parity meaning that it writes an additional parity bit to another disk, while this is slower than RAID 5 it provides protection against 2 drive failures. To help counter the penalty of RAID 5 and 6 many raid controllers will often have options of Caching in the 512/1/2/4/8 GB range allowing for data to be written to a flash or ram based cache until the storage volumes are ready to receive the data. While this will mask much of the operations in smaller environments cache exaughstion is real and can make your performance suffer as you will be bound to your spindles. NOTE: While there are additional RAID configurations such as RAIDZ or RAID 1E, these are the most common configurations that you will come across. Picking a storage architecture While the list can go on for a while, the things you need to find out is what will suite you application and business continuity needs, most of which are detailed below. Direct Attached Storage(DAS)- Directly attached storage is by far the simplest to setup and configure, there is no network resources utilized it attaches straight to the host and generally has the lowest access times. However, with DAS it does limit your ability to protect data for high availability, potentially has higher operational costs vs a NAS/SAN, and server faults can provide additional downtime. Vendors are determined by the HW manufacturer of the server Virtual Storage Area Network(vSAN)- Somewhat of a newcomer to the field this method uses setup and utilizes resources of local storage to provide a larger more aggregated raid setup allowing for a highly available solution for customers who can not find it feasible for a NAS but need the storage redundancy and availability. Vendors include: EMC, VMware, HP Network Attached Storage(NAS)- A NAS is generally an appliance working at the file level providing things such as NFS services, generally more geared to BASET networking and not block level protocols; a NAS may not also be highly available on the storage controller level. Generally a NAS is cheaper than a SAN, and is more appealing to small cluster environments or the SMB market space. Vendors of NAS include: NetApp, Dell, Synology, HP EasyStore Storage Area Network(SAN)- A SAN is generally an appliance operating on the block level of storage utilizing protocols such as ISCSI, FC, FCoE; this generally is higher performing than a NAS in terms of latency and performance. A SAN may also be comprised of multiple Storage appliances working together to provide more control, space, and availability of storage. Vendors of SAN's include: EMC, Dell, 3Par, NetApp, Nimble Spindles or Flash Flash is a reality, it's here to stay and it is where storage is going to go, even with 16TB drives coming out flash is more compact, less power, cooling, and to some extent less prone to failure. This isn't to say we won't see traditional storage of spinning drives go anywhere soon, currently spindles are very reliable have a very cheap cost per GB/TB, and can work together with flash to provide additional performance. It is becoming more common to see storage arrays utilizing a small pool of SSD's such as 4 200GB SSD's in RAID 10 that cache Writes and Reads. This allows end users to get data faster that is considered "hot" meaning the storage array is looking on the slow end disks for blocks that are commonly read and copying them into cache so users can get "SSD like performance" without the cost of an all flash array. The problem comes in where consumers or vendors under estimate or under spec how much data is used by a company on a day to day process, or when the storage controller will stop considering the block "hot" and take it out of cache. This can lead to a problem known as cache exhaustion, in which SSD or RAM Cache is depleted and the storage controller must send requests to the back end disks for data bypassing any caching. While this is not a very common problem for most people it can make your life harder if you have users common to a performance level and then the storage dips in performance. Hybrid arrays exist from things like Nimble, Dell, HP, NetApp, EMC; most vendors offer a SSD/HDD array type. Things like ExtremIO, NetApp Pure, Violin utilize RAM and SSD caching and other techniques to improve performance and allow for things such as much faster deduplication and even inline deduplication, compression, and the like. Delivering the storage to servers and services -wip- What I got thus far any suggestions or feedback? Dilbert As FUCK fucked around with this message at 22:20 on Jul 14, 2014 |
|
#
?
Jul 14, 2014 22:08
|
|
|
Dilbert As gently caress posted:RAID 5 - One of the most commonly used raid due to it's reputation of being data redundancy, while providing increased storage space for data. It while it does provide redundancy and increased storage space, the draw back is that it comes with additional write penalties meaning data written to disk will take longer as data must be written in parity to each disk in the volume. ------------ |X||X||X||P| ------------ and you need to write this on top of it (where X is a totally changed segment, x is a partially changed segment, and P is the new parity for the stripe): ------------ |_||X||x||P| ------------ you're actually doing the following: ------------- |R||W||RW||W| ------------- So because you need to rewrite one full segment and one partial segment of the write, instead of doing two writes onto the stripe, you're doing two reads and three writes. Worse yet, you can't linearize the whole thing like you can if you were the write to every disk -- you need to wait on those reads to complete before you can start writing back. So if you're doing this synchronously without caching, your latency is more than double what it would be if you were able to take advantage of a write-back cache. Vulture Culture fucked around with this message at 23:54 on Jul 14, 2014 |
|
#
?
Jul 14, 2014 23:43
|
|
|
NippleFloss posted:What Nimble are you looking at and what's the 2520 config look like? Nimble CS220 and I went back to our Netapp VAR to get some 2552 configs, I dunno why they quoted out the 2520 initially. Regardless of the model numbers though, mainly just curious what I'm going to be missing (if anything) going from Netapp to Nimble for vmware storage.
|
|
#
?
Jul 15, 2014 00:18
|
|
|
Maneki Neko posted:Regardless of the model numbers though, mainly just curious what I'm going to be missing (if anything) going from Netapp to Nimble for vmware storage.
|
|
#
?
Jul 15, 2014 00:54
|
|
|
drat, Just got the quote from Nimble today, and while it's not 22k for a CS220G, it's almost exactly the same price as an Equallogic PS6210X + a little bit more for support. If we weren't dead set on using SyncRep for our new MSSQL cluster I think I could make a good business case for a new SAN vendor. Unfortunately everything I've read says that Nimble doesn't have a comparable technology to SyncRep, which might be a deal breaker for us. Goodbye, pipe dream 
|
|
#
?
Jul 15, 2014 01:02
|
|
|
Wicaeed posted:drat, Why not just use SQL 2012 availability groups and have much more transparent failover with any storage you choose?
|
|
#
?
Jul 15, 2014 03:26
|
|
|
Yeah, I don't see why people still insist of storage level replication for everything. There is a lot out there in the ways of application level replication or clustering these days.
|
|
#
?
Jul 15, 2014 03:33
|
|
|
NippleFloss posted:Why not just use SQL 2012 availability groups and have much more transparent failover with any storage you choose? I really don't know. For some reason we decide to separate our billing DB into read servers and write servers. It seems incredibly backwards, but I'm not a DBA.
|
|
#
?
Jul 15, 2014 06:22
|
|
|
Wicaeed posted:I really don't know. For some reason we decide to separate our billing DB into read servers and write servers. You can do that with availability groups. Secondary copies are read only by default. Separating reporting or backup on to a read only copy of a DB is common, and trivially easy with native SQL 2012 tools. Even as an employee of a storage vendor with a pretty robust replication suite I still recommend that our customers use native 2012 replication rather than our tools.
|
|
#
?
Jul 15, 2014 06:48
|
|
|
NippleFloss posted:You can do that with availability groups. Secondary copies are read only by default. Separating reporting or backup on to a read only copy of a DB is common, and trivially easy with native SQL 2012 tools. Even as an employee of a storage vendor with a pretty robust replication suite I still recommend that our customers use native 2012 replication rather than our tools. Probably more for the DBA thread, but correct me if I'm wrong, don't MSSQL HA Availability Groups use Cluster Shared Volumes for storage? Or can they use attached local storage as well? The presentation I've been given says we are going to be using our SAN for write-intensive storage in a 2 node availability group, and then using another 2 nodes availability group for the read-intensive operations. Either way we are potentially wasting 50% of our resources since: A) We have 4 nodes to use, and all 4 nodes have the same storage on them B) Only two of those nodes (the read-intensive workloads) will be used in an availability group C) Two nodes will have all of their local storage going to waste
|
|
#
?
Jul 15, 2014 07:01
|
|
|
Wicaeed posted:Probably more for the DBA thread, but correct me if I'm wrong, don't MSSQL HA Availability Groups use Cluster Shared Volumes for storage? Or can they use attached local storage as well? code:http://blogs.msdn.com/b/sqlcat/archive/2013/11/20/sql-server-2012-alwayson-high-availability-and-disaster-recovery-design-patterns.aspx You never want all four nodes having the same storage. At most, you would have two HA clusters that each shares the same volumes between the cluster members, and use log shipping to your DR site. Vulture Culture fucked around with this message at 07:11 on Jul 15, 2014 |
|
#
?
Jul 15, 2014 07:09
|
|
|
|
| # ? May 21, 2024 15:48 |
|
|
Misogynist posted:In SQL Server 2008, a common topology was something like this: DR site, what's that?  This company has had to learn some hard lessons, and apparently is still learning them.
|
|
#
?
Jul 15, 2014 07:37
|
|