Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
mayodreams
Jul 4, 2003


Hello darkness,
my old friend

Nitr0 posted:

"but I know the mass storage is 7200k drives in a raid6."

Oh man oh man oh man. No wonder VM blames storage whenever someone phones in with vcenter problems.

Nexenta uses zfs, so in reality it is raidz2, not really raid6. This data store is for secondary vmdk's and iscsi luns for file servers, so performance isn't key.

I agree that we would be better off with a Netapp or something, but we can't get the execs to see the value in that, as painful as that is.

Adbot
ADBOT LOVES YOU

adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer

mayodreams posted:

Nexenta uses zfs, so in reality it is raidz2, not really raid6. This data store is for secondary vmdk's and iscsi luns for file servers, so performance isn't key.

I agree that we would be better off with a Netapp or something, but we can't get the execs to see the value in that, as painful as that is.
I'm sure your nexenta device can perform the way you want it to, but you are going to have to add more ram, an l2arc if you don't have one, a slog if you don't have one, and most importantly more spindles. ZFS can perform extremely well, even in a raidz or raidz2 configuration, but mostly because of it's scaling features. The design of the filesystem was first and foremost for data integrity and scaling, performance was a secondary concern. The slog and arc really help, but there comes a point when more spindles, and specifically more vdevs, are the only way to improve performance.

CtrlMagicDel
Nov 11, 2011

parid posted:

June was a bad month for my VMware clusters. We had a series (at least 3) of network outages that prevented many of my hosts from talking to their storage (all NFS). Every time this happens I have to take time to get vCenter back up before we can dig in fixing the rest of the VMs. In on case, this was an extra hour of delay.

I'd love to fix the root of the problem (the network) but its out of my hands with another team. What I do have control over is how we have vCenter implemented. Right now the vcenter/sso server is physical. I had plans to virtualize it very soon. The SQL server for it is already a VM.

Making vCenter a VM is fine for many failure scenarios, but the kind of disruptive network failure were seeing more of would be challenging to deal with. I would like to find a way to make it multi-site. Or at least have some kind of cold standby if the app in the primary datacenter fails.

What is everyone doing to ensure the availability of their vCenter? It looks like VMware is dropping heartbeat soon. I see vCenter supports SQL clustering and microsoft cluster services now. Is this additional headache worth considering?

I'm kind of in the same boat as you with a physical vCenter and no HA/DR solution beyond "rebuild vCenter". I was actually arguing for us to buy Heartbeat until I found out at the VMUG that is was going end of sale.

We have two datacenters that share networks across a MAN, each of which is the failover DR site for the other and I'd like to solve both the HA and DR problems. My current plan is to virtualize the existing vCenter next time we upgrade and purchase a second vCenter to run at the other data center and run them in linked mode, and migrate all the hosts at the secondary site to the local vCenter. The only downside I can find is that then we can't vMotion between data centers (not really a big deal) and that I'm guessing we'd have to take some downtime to migrate VM's?

parid
Mar 18, 2004

CtrlMagicDel posted:

I'm kind of in the same boat as you with a physical vCenter and no HA/DR solution beyond "rebuild vCenter". I was actually arguing for us to buy Heartbeat until I found out at the VMUG that is was going end of sale.

We have two datacenters that share networks across a MAN, each of which is the failover DR site for the other and I'd like to solve both the HA and DR problems. My current plan is to virtualize the existing vCenter next time we upgrade and purchase a second vCenter to run at the other data center and run them in linked mode, and migrate all the hosts at the secondary site to the local vCenter. The only downside I can find is that then we can't vMotion between data centers (not really a big deal) and that I'm guessing we'd have to take some downtime to migrate VM's?

Two vCenters isn't a terrible idea. Have you joined the 6 beta yet? This idea might be even better in the future.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

adorai posted:

I'm sure your nexenta device can perform the way you want it to, but you are going to have to add more ram, an l2arc if you don't have one, a slog if you don't have one, and most importantly more spindles. ZFS can perform extremely well, even in a raidz or raidz2 configuration, but mostly because of it's scaling features. The design of the filesystem was first and foremost for data integrity and scaling, performance was a secondary concern. The slog and arc really help, but there comes a point when more spindles, and specifically more vdevs, are the only way to improve performance.

Note on Nexenta, if you plan to install bare metal do not use broadcom NIC's....

Holy poo poo I had no loving clue how annoying it could be. I think they recommend it in a VM anyways, since it supports the VMXnet and if you have Direct IO passthrough to a raid controller or the like.

Next week I am going to see what the performance metrics are like, probably post some of the results.

parid posted:

Two vCenters isn't a terrible idea. Have you joined the 6 beta yet? This idea might be even better in the future.

That feature won't work with a MAN, unless you are talking about somethingelse.

CtrlMagicDel posted:

I'm kind of in the same boat as you with a physical vCenter and no HA/DR solution beyond "rebuild vCenter". I was actually arguing for us to buy Heartbeat until I found out at the VMUG that is was going end of sale.

We have two datacenters that share networks across a MAN, each of which is the failover DR site for the other and I'd like to solve both the HA and DR problems. My current plan is to virtualize the existing vCenter next time we upgrade and purchase a second vCenter to run at the other data center and run them in linked mode, and migrate all the hosts at the secondary site to the local vCenter. The only downside I can find is that then we can't vMotion between data centers (not really a big deal) and that I'm guessing we'd have to take some downtime to migrate VM's?
I mean what do you do for site redundancy? SRM has a 15 minute RTO, is that window to short? I'll give you DRS and vMotion are nice but is 15 minutes that critical?

Dilbert As FUCK fucked around with this message at 01:58 on Jul 6, 2014

CtrlMagicDel
Nov 11, 2011

parid posted:

Two vCenters isn't a terrible idea. Have you joined the 6 beta yet? This idea might be even better in the future.

Yeah, I joined the beta right after posting that, so obviously I'm now thrown for another loop and will have to do some testing of that functionality.

Dilbert As gently caress posted:

That feature won't work with a MAN, unless you are talking about somethingelse.

Maybe MAN is the wrong terminology, they are different physical locations but share the same network (like my hosts at both datacenters are on the same subnet range). I'm kind of assuming this is something weird my company does that no one else is doing.


Hahaha

Dilbert As gently caress posted:

SRM has a 15 minute RTO, is that window to short? I'll give you DRS and vMotion are nice but is 15 minutes that critical?

They quoted us the prices for SRM this year and it basically got laughed out of the budget. I cobbled together a PowerCLI script that re-adds all of our VM's to inventory and powers them back on in the event of a DR scenario.

parid
Mar 18, 2004

CtrlMagicDel posted:

They quoted us the prices for SRM this year and it basically got laughed out of the budget. I cobbled together a PowerCLI script that re-adds all of our VM's to inventory and powers them back on in the event of a DR scenario.

Your environment sounds very similar to mine. The only difference so far is that we have synchronous storage in both sites. Unfortuneatly we don't have a 3rd site quorum. The switch is manual. Once we throw it on the storage, we can let HA do the rest.

Keep in touch on what you decide to do :). Right now were going to virtualized the vcenter and fence it tightly with drs group host affinity. I might have to manually fix but at least I have the best shot possible to do so.

In version 6, two vcenters sounds like and neat solution.

CtrlMagicDel
Nov 11, 2011

parid posted:

Your environment sounds very similar to mine. The only difference so far is that we have synchronous storage in both sites. Unfortuneatly we don't have a 3rd site quorum. The switch is manual. Once we throw it on the storage, we can let HA do the rest.

Keep in touch on what you decide to do :). Right now were going to virtualized the vcenter and fence it tightly with drs group host affinity. I might have to manually fix but at least I have the best shot possible to do so.

In version 6, two vcenters sounds like and neat solution.

Our hosts don't share the same storage between data centers so I'm guessing we are probably pretty similar. We replicate our datastores between the two data centers. Our recovery is currently pretty manual for DR.

I'm still not sure what exactly the implication of multiple vCenters is going to be now but I haven't done a ton of reading of the beta documentation. Will look forward to discussing it more after I have read up more and after it can be discussed publicly. On that note, anyone going to VMworld? We got a free ticket from one of our vendors so it sounds like I'll be attending and would love to meet some wise virtualization goons.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

Well shoot if srm or the like is out of the question that limits you quite a bit. Are you able to use vsphere replication at least?

CtrlMagicDel
Nov 11, 2011

Dilbert As gently caress posted:

Well shoot if srm or the like is out of the question that limits you quite a bit. Are you able to use vsphere replication at least?

We have array based replication but it isn't really automated at the moment or integrated with vcenter. I'm planning on arguing that we need another vcenter at our other site up and running for DR purposes as I think that will be reasonably enough priced I should be able to get that ordered. If we virtualized vcenter and all the db's it would be a step In the right direction but it would still require some recovery time id rather not have to spend.

Edit: fix stupid autocorrect from my phone because our new firewall blocks SA because it is "questionable".

CtrlMagicDel fucked around with this message at 22:15 on Jul 7, 2014

some kinda jackal
Feb 25, 2003

 
 
Can I disable an alarm that was defined at the VCSA level for a particular datastore?

I'm running one lab ESXi host which I set up explicitly for one VM, so I allocated it like 95% disk. Obviously this triggers the "datastore usage on disk" alarm, and I dutifully acknowledge it, but it seems to trigger every now and then. Possibly every time the machine or VCSA is rebooted? I just overhauled the cabinet my lab servers are in so maybe that's why it's triggering again. Anyway, I can't find a way to disable it for that one particular VM and maybe it's not possible, which isn't the end of the world.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

Martytoof posted:

Can I disable an alarm that was defined at the VCSA level for a particular datastore?

I'm running one lab ESXi host which I set up explicitly for one VM, so I allocated it like 95% disk. Obviously this triggers the "datastore usage on disk" alarm, and I dutifully acknowledge it, but it seems to trigger every now and then. Possibly every time the machine or VCSA is rebooted? I just overhauled the cabinet my lab servers are in so maybe that's why it's triggering again. Anyway, I can't find a way to disable it for that one particular VM and maybe it's not possible, which isn't the end of the world.

Quick and easy fix for a lab is to drop the datastore in its own folder then disable alarms on the folder level

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
Where the gently caress does vmware keep the startup config scripts for the VDP? I want to comment out the nslookup for reverse DNS lookup it does, since the site I am at won't have it for "security reasons".

Dilbert As FUCK fucked around with this message at 17:10 on Jul 9, 2014

1000101
May 14, 2003

BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY FRUITCAKE!

Dilbert As gently caress posted:

Where the gently caress does vmware keep the startup config scripts for the VDP? I want to comment out the nslookup for reverse DNS lookup it does, since the site I am at won't have it for "security reasons".

Assuming you mean the appliance it won't be a config file.

Add a host file entry for itself that includes the FQDN.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

1000101 posted:

Assuming you mean the appliance it won't be a config file.

Add a host file entry for itself that includes the FQDN.

I'll try that again but I think it is performing a nslookup which will bypass the /etc/hosts entry.


I probably forgot to restart the network service, after adding into hosts, last time now that I think about it.

three
Aug 9, 2007

i fantasize about ndamukong suh licking my doodoo hole

Dilbert As gently caress posted:

I'll try that again but I think it is performing a nslookup which will bypass the /etc/hosts entry.


I probably forgot to restart the network service, after adding into hosts, last time now that I think about it.

/etc/nsswitch.conf configures the order of lookup. By default, the VCSA is configured like a normal linux system to use 'files' (aka hosts file) then 'dns'.

You don't need to restart the network service after modifying a hosts entry.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

three posted:

/etc/nsswitch.conf configures the order of lookup. By default, the VCSA is configured like a normal linux system to use 'files' (aka hosts file) then 'dns'.

You don't need to restart the network service after modifying a hosts entry.

Yeah I tried it anyways by restarting, I think it's looking for a specific return from an nslookup command, all vdp appliances still fails to get past it because no reverse lookup zone.

Just going to use Veaam, I like VDP but Veeam is fine for a smallish environment of 250-300 vms.

Serfer
Mar 10, 2003

The piss tape is real



Dilbert As gently caress posted:

It's also a bit of an annoyance when people don't optimize their 2008r2 templates for a virtual environment.
What constitutes optimizing 2008r2 for virtual environments?

evol262
Nov 30, 2010
#!/usr/bin/perl

Dilbert As gently caress posted:

Yeah I tried it anyways by restarting, I think it's looking for a specific return from an nslookup command, all vdp appliances still fails to get past it because no reverse lookup zone.

Just going to use Veaam, I like VDP but Veeam is fine for a smallish environment of 250-300 vms.

/etc/hosts|nsswitch will only work if it's doing gethostbyaddr(), and can by bypassed pretty trivially. But it can also be checked pretty trivially, with "python -c 'import socket; socket.gethostbyaddr('some.address')"
"getent hosts some.addr" will also use gethostbyaddr()

But bind-utils (nslookup, host, etc) intentionally bypass the hosts file and normal name resolution.

The application may also be caching.

Why don't you just setup up a trivial DNS server?

adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer

Serfer posted:

What constitutes optimizing 2008r2 for virtual environments?

excellent question, I definitely await the answer.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

Serfer posted:

What constitutes optimizing 2008r2 for virtual environments?

There are many ways depending on what kind of environment you run but these guides are pretty good:

http://support.citrix.com/article/CTX131577
http://jeremywaldrop.wordpress.com/2008/10/28/how-to-build-a-windows-2008-vmware-esx-vm-template/ (don't agree with the 64MB video card 16/32 works well, but whatever)

It's not the biggest thing in the world as IOPS are more plentiful but for solid disk arrays that have only Controller cache, it's a minor annoyance at least to me.

evol262 posted:

/etc/hosts|nsswitch will only work if it's doing gethostbyaddr(), and can by bypassed pretty trivially. But it can also be checked pretty trivially, with "python -c 'import socket; socket.gethostbyaddr('some.address')"
"getent hosts some.addr" will also use gethostbyaddr()

But bind-utils (nslookup, host, etc) intentionally bypass the hosts file and normal name resolution.

The application may also be caching.

Why don't you just setup up a trivial DNS server?


Trivial DNS did cross my mind but I didn't want network team/others to freak out when I fire up a DNSVM to get past the option. I'm going to dig more into it this weekend see if I can bypass it.

Dilbert As FUCK fucked around with this message at 23:46 on Jul 10, 2014

Erwin
Feb 17, 2006


I don't see anything in there about video cards, nor do I see much specific to virtualization. Everything he does is either something you'd do to an image for physical servers or something rear end in a top hat sperglords do, like the classic start menu. And turning off the Windows firewall? Come on.

Docjowles
Apr 9, 2009

Serious question: I see that that article is 6 years old. In 2014 can most of that poo poo be scripted? When we create a golden image or template for our Linux guests, all we do is have the VM PXE boot and run the same Kickstart procedure we use for physical servers, plus a post-install script that does standard templatey stuff like delete the SSH keys and hardcoded MAC addresses. If someone on my team presented me with a 71 item checklist of how they prepared a VM to deploy I'd say "that's great. come back to me when you've automated the entire thing." I'm a zealot about config management tools and getting rid of terrible checklists is one of their many merits :catholic:

Just wondering as I haven't done real Windows work in years. I don't mean to troll or start an OS holy war. The fact that the document leads with "VMware Virtual Center 2.5" suggests that the content is old as hell.

Internet Explorer
Jun 1, 2005





I'm no virtualization Guru but I can't think of any changes I make to 2008 R2 other than installing the VMware / XenServer tools.

Serfer
Mar 10, 2003

The piss tape is real




So maybe one good piece of advice (indexing), a whole bunch of OCD stuff, and two pieces of bad advice (disable firewall and ipv6).

three
Aug 9, 2007

i fantasize about ndamukong suh licking my doodoo hole
It's a 6 year old blog post.

Erwin
Feb 17, 2006

Docjowles posted:

Serious question: I see that that article is 6 years old. In 2014 can most of that poo poo be scripted?

Yeah, between group policy, PowerShell desired state configuration, and maybe SCCM, you don't really need to do anything besides install Windows and maybe Windows updates if you want the post-clone process to go quicker. I think it's a case of Dilbert saying something incoherent, people asking for clarification, and him hastily blurting out the first Google result without checking the date or content.

Moey
Oct 22, 2010

I LIKE TO MOVE IT

Erwin posted:

Yeah, between group policy, PowerShell desired state configuration, and maybe SCCM, you don't really need to do anything besides install Windows and maybe Windows updates if you want the post-clone process to go quicker. I think it's a case of Dilbert saying something incoherent, people asking for clarification, and him hastily blurting out the first Google result without checking the date or content.

You need to add in an additional floppy drive.

Pile Of Garbage
May 28, 2007



Moey posted:

You need to add in an additional floppy drive.

Well yeah, where you going to mount your RAMDISK?

some kinda jackal
Feb 25, 2003

 
 
What are my options for moving VMs between hosts without shared storage? I pretty much want the least hassle.

I had to do an emergency server buildout today because our file server was making GBS threads itself in front of my eyes. I took a lab machine, slapped a few 1TB drives in there, and built a new 2012R2 server, etc. I specifically did this on top of ESXi so I would have a VM that I could "move" later, though I didn't really put much thought into how I would move it because of the general "oh my god the world is ending" atmosphere. I'm picking up a much beefier server on Monday to be this guy's new home, but once it arrives I'll have to find a way to get it over there.

My plan was just to enable SSH on both machines and scp the files between the two overnight. Unless someone has a better idea that's probably what I'll end up doing.

Much as I'd love shared storage, it's really out of my realm of experience and the last thing I want to do is start half-assing a SAN to host our critical doodads.

Erwin
Feb 17, 2006

VeeamZIP: http://www.veeam.com/virtual-machine-backup-solution-free.html#veeamzip

some kinda jackal
Feb 25, 2003

 
 
Sounds promising. Thanks!

Moey
Oct 22, 2010

I LIKE TO MOVE IT
Seconding Veeam Zip.

evol262
Nov 30, 2010
#!/usr/bin/perl

Martytoof posted:

What are my options for moving VMs between hosts without shared storage? I pretty much want the least hassle.

I had to do an emergency server buildout today because our file server was making GBS threads itself in front of my eyes. I took a lab machine, slapped a few 1TB drives in there, and built a new 2012R2 server, etc. I specifically did this on top of ESXi so I would have a VM that I could "move" later, though I didn't really put much thought into how I would move it because of the general "oh my god the world is ending" atmosphere. I'm picking up a much beefier server on Monday to be this guy's new home, but once it arrives I'll have to find a way to get it over there.

My plan was just to enable SSH on both machines and scp the files between the two overnight. Unless someone has a better idea that's probably what I'll end up doing.

Much as I'd love shared storage, it's really out of my realm of experience and the last thing I want to do is start half-assing a SAN to host our critical doodads.

If you're not totally tied to VMware (not sure from your comment of using esxi just to move it), libvirt can migrate VMs between machines with no shared storage.

adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer
Why does he disable the serial and parallel port in the bios instead of just deleting them from the VM? Seems like a waste for the hypervisor to emulate them just to disable them.

some kinda jackal
Feb 25, 2003

 
 

evol262 posted:

If you're not totally tied to VMware (not sure from your comment of using esxi just to move it), libvirt can migrate VMs between machines with no shared storage.

Sorry, I should have been more clear. I'm slowly implementing a vSphere environment so I'm tied to ESXi in that regard. I mentioned building it out on ESXi because the file server I'm building is a physical server, and I was aiming to not just create another temporary physical server to replace it, but rather a guest on a proper host.

1000101
May 14, 2003

BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY FRUITCAKE!

Martytoof posted:

Sorry, I should have been more clear. I'm slowly implementing a vSphere environment so I'm tied to ESXi in that regard. I mentioned building it out on ESXi because the file server I'm building is a physical server, and I was aiming to not just create another temporary physical server to replace it, but rather a guest on a proper host.

vCenter can make that move for you as well.

some kinda jackal
Feb 25, 2003

 
 

1000101 posted:

vCenter can make that move for you as well.

If they're not on shared storage? I'm looking at my lab vCenter and I don't see the option. I can move from datastore to datastore on the SAME machine, but I dunno. I might be missing something though.

jre
Sep 2, 2011

To the cloud ?



Martytoof posted:

If they're not on shared storage? I'm looking at my lab vCenter and I don't see the option. I can move from datastore to datastore on the SAME machine, but I dunno. I might be missing something though.

Is the vm running ? The option might not show until you shut it down.

Adbot
ADBOT LOVES YOU

some kinda jackal
Feb 25, 2003

 
 
Oh yeah, that's it. OK thanks, looks like I can save myself some hassle and just have vSphere do it. Excellent :)

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply