Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
luminalflux
May 27, 2005



Bob Morales posted:

So apparently Apple's 10.8.2 update breaks VirtualBox on Mac laptops.

Good thing I'm still on Lion at work :smug:

https://discussions.apple.com/thread/4311118?start=0&tstart=0

Wow, that's going to seriously suck for basically everyone that relies on Vagrant to do devops stuff.

Adbot
ADBOT LOVES YOU

luminalflux
May 27, 2005



Bob Morales posted:

Solution 1: Use Parallels or VMware
Solution 2: Don't upgrade your work machine the day an OS update comes out

Vagrant is still unfortunately Vbox-only, but work is underway to allow other hypervisors.

I generally never use .0 releases of OSes

luminalflux
May 27, 2005



VMware's done a lot lately with Postgres, like vFabric for example. I really hope they ditch the need to use an external database that costs a fuckton for vCenter, get VUM working in the appliance and basically let me ditch the only windows server in my whole company that I have no other need for.

luminalflux
May 27, 2005



Today's fun: I logged in and realized vCenter was down. Apparently the vCenter database swelling to double the size after the 5.1 upgrade, past the SQL Server 2005 Express license limit making the vCenter server kill itself.

Wait, didn't 5.1 install SQL Server 2008 Express? Why yes it did, but you have to migrate the data yourself apparently (I missed this step in the upgrade). So, a migrate later, change the DSN, realize that management studio isn't available, fire up vCenter again, uninstall 2005.

Now update manager won't start. Oh yeah, that needs a new DSN, pop that in. Wait, still not working? Huh? Oh yeah, you need to create a 32 bit DSN instead of the normal 64, didn't you read the documentation?

VUM is at least running but not being able to download patches. Of course, finding logs means generating a 500MB large zip "log bundle" and grepping through since VUM doesn't use Windows built-in event logging, and adding a utility like tail -f never occurred to anyone (or telling me where the logs for the VUM server are).

How the hell do windows admins manage their daily work without drinking themselves to death is beyond me (being a unix guy). Add in a bunch of frustration due to my RDP client being a dick, I gave no fucks to go off drinking at 5pm (5 course microbrew/food pairing tasting menu).

luminalflux
May 27, 2005



FISHMANPET posted:

I was planning on leaving the actual server at 5.0, just install 5.1 for vCenter, since vMotion and VUM can't really take care of your vCenter server, and I don't want to deal with the stuff luminalflux did.

Updating ESXi was smooth. Right now it's only VUM being a jerk, so it gets to sit in the corner for a few days until it can behave.

luminalflux
May 27, 2005



Oh wow.

After having some issues with SQL Server 2005 Express and vCenter 5.1 after upgrading (Apparently I was still running 2005 instead of the 2008 that was upgraded to...), I upgraded the database, which apparently borked since the schema was all wrong and no stored procs were loaded. Thinking "Nah I'll just nuke it and reinstall from scratch" (since that was what #vmware was basically saying), I did a fresh install of vCenter on a new database.

Now I can't connect to my existing ESXi 5.1 hosts. Or rather, I can connect, they'll show up for about a minute, and then go back to being disconnected. Not sure what to look for in the logs (since they're chatty as gently caress), and what looks like the big culprit isn't giving me any good results in google (besides making sure that the ports are open and restarting the Mgmt agent on the hosts).

Does

quote:

[10512 error 'vpxdvpxdVmomi' opID=01E5A73D-00000092-bd] [VpxdClientAdapter] Got vmacore exception: Server closed connection after 0 response bytes read;
SSL(TCPClientSocket(this=000000001a155c20, state=CONNECTED, _connectSocket=TCP(fd=-1), error=(null)) TCPStreamWin32(socket=TCP(fd=4944) local=10.x.x.1:55044, peer=10.x.x.17:443))

in vpxd.log ring a bell for anyone? Otherwise it looks like I'm going to spend monday with support to get my management up and running again (production hosts are running but I have no way of doing stuff with them)

luminalflux
May 27, 2005



Will do, I did the restarting thing but that didn't help.

edit: Nope, same thing happened again :(

luminalflux fucked around with this message at 15:37 on Sep 30, 2012

luminalflux
May 27, 2005



It seems that the for some reason HKEY_LOCAL_MACHINE\SOFTWARE\VMware, Inc.\VMware VirtualCenter\heartbeatPort was configured to port 922 instead of 902, and 922 wasn't open while 902 was. vpxd was pushing this down to the agents upon reconnect, while vCenter was listening on the default port. Naturally no heartbeats were getting through, which forced the disconnects.

At least I hope, but it hasn't disconnected yet, let's let this run for a day or so.

edit: I would have caught this a lot earlier, if I were looking at the correct file. Apparently on the host there's both /etc/opt/vmware/vpxa/vpxa.cfg from an older ESXi version and /etc/vmware/vpxa/vpxa.cfg which is the running config...

luminalflux fucked around with this message at 09:37 on Oct 1, 2012

luminalflux
May 27, 2005



I probably did a lot of stupid stuff while reinstalling when the database was in the wrong state, is my only guess.

luminalflux
May 27, 2005



Currently I'm running 3 networks: VM network as a vDS, management and iSCSI on normal vswitches. Is there any reason I should move iSCSI to a vDS in the future as we keep expanding this setup? Is there any reason not to?

luminalflux
May 27, 2005



I use CoRD but gently caress that when can I access VM consoles via the web client on mac

luminalflux
May 27, 2005



So if I'm storing my Postgres databases on iSCSI-backed vmdk's, I'm doing it wrong?

luminalflux
May 27, 2005



itskage posted:

/\ You don't have to, but it is best practice for performance.

Got any numbers? Right now I kinda like the fact that it's easy to grow the vmdk to add add storage space instead of screwing around with the LeftHand. in fact, our LeftHand is only used for vmware storage

luminalflux
May 27, 2005



My databases aren't big either - if anything I'd probably consolidate a bunch of databases into one VM or a physical machine if VMware overhead was the problem

luminalflux
May 27, 2005



Don't know which is the correct thread (until someone starts a "gently caress HP" thread), but the HP Insight Control for vCenter installer borked in some way when installing it on my vCenter server. Can't install it, can't uninstall it.

Thanks HP.

luminalflux
May 27, 2005



Number19 posted:

Just wait until it starts leaking memory randomly and your vCenter grinds to a halt.

I'm pretty sure that's the responsibility of the LeftHand management tool.

luminalflux
May 27, 2005



Kerpal posted:

Anyone know about hot-adding CPUs in Debian Linux?
This is what I do on CentOS:

Hot-add the vCPUs.
Run dmesg to see that they've been discovered.
Run # echo 1 > /sys/devices/system/cpu/cpu1/online to enable the cpu

Hot-adding CPUs is no problem. Hot-adding memory from under 4GB to over might be an issue since bounce buffers are disabled in the IOMMU by default if under 4GB is probed on boot.

luminalflux
May 27, 2005



partprobe will pick up changed size of the device. Then its just a matter of either adding a new LVM physical volume to the VG or editing the partition table with fdisk to add more space at the end.

luminalflux
May 27, 2005



Soekris and duct tape.

luminalflux
May 27, 2005



Erwin posted:

Since it's a "mid-tower colocation special" I assume that means you don't have the option of out of band management anyway. The only chicken and egg scenario happens when ESXi dies enough to bring down Monowall or the management VM. Hopefully your colo could lend a hand in that situation.

According the specs on hp.com, ML110 G7 has iLO. It might be shared with the 1st NIC port though, haven't really touched that series and it seems a dedicated iLO port is an option.

luminalflux
May 27, 2005



Corvettefisher posted:

Generally for the best networking you'll want to use the VMXNET3 when possible, it offers many performance improvements over the E1000

Which specifically, and is this true for all OSes?

luminalflux
May 27, 2005



Interesting. I only found benchmarks for Windows unfortunately, doesn't look like anyone has hard numbers for vmxnet3 vs e1000 for Linux. I just checked, most of my stuff seems to be running on e1000 with a few newer VMs running vmxnet3, and I haven't seen the issues with vmxnet3 on Linux the forums seem to be crying about.

And yeah, e1000 is the tulip/lance of 2010's. (I had to get separate 4x1G E1000 since OpenBSD supports that but not the Broadcoms that HP puts in their new servers)

Edit:

Misogynist posted:

One other neat thing about VMXNET2/3 is that they use shared memory for network communication with the hypervisor, so if you have two VMs on the same host communicating over the same portgroup they can throw traffic at each other as fast as they can write to memory. Depending on how you can collocate your VMs, you can really kick up the responsiveness of a few applications.

brb converting ALL the VMs to vmxnet3

(seriously a large chunk of my what my webapp servers do is network chatter)

luminalflux
May 27, 2005



skipdogg posted:

Anyone have any experience with running ESXi from a SD Card on HP DL3xx G8 servers?

I chucked in a random SanDisk and it works fine on both my G7s and G8s

luminalflux
May 27, 2005



I store my logs on local storage, but VM swap is stored with the VM iirc. Since our SAN is 2x1GbE we also use local storage for stuff we don't need to vMotion (test machines) or stuff that would be detrimental to SAN performance (Syslog for 100+ VMs)

luminalflux
May 27, 2005



Is there an easy way to test PXE install of ESXi into a VM? We're acquiring new hypervisors and I'd like to not have to screw around with install media, but I'd like to try it out on a VM first.

luminalflux
May 27, 2005



Moey posted:

What are you installing it on to?

Currently I just prep thumb drives with unetbootin and ESXi then pop that into the server and run from there.

Installing the HP ESXi 5.1 image on a DL360 G7 and a G8.

I never got UNetBootin to create working thumbdrives on Mac, nor did I get dd:ing the 5.1 image to an SD-card and shoving it in to work. Last install I took a 4.1 SD card I had lying around and used VUM to bring it up to the 5.1 HP install.

cf: I meant PXE-installing ESXi onto a VM in ESXi, is that possible?

luminalflux
May 27, 2005



HP charges extra for iLO Advanced which gives you that feature (at least on rackmounts, dunno about blades), I don't have it licensed on the machines I'm installing onto.

edit: Yeah it's possible, just need to set a flag on the hypervisor.

luminalflux fucked around with this message at 13:08 on Apr 26, 2013

luminalflux
May 27, 2005



Storage vMotion to another datastore, convert to thin instead of "use source type" or whatever it says.

luminalflux
May 27, 2005



I'm probably going to whenever I get around to setting up a CA so any writeup would be helpful

luminalflux
May 27, 2005



Corvettefisher posted:

Hooray vSphere 5.1 U1 supports SQL 2012 now!

I'm still waiting on Postgres support for external databases so I can ditch SQL Server entirely.

luminalflux
May 27, 2005



Misogynist posted:

If PowerCLI is too much trouble for you, you're going to have a lot of fun trying to work with the SOAP APIs.

There's a ruby wrapper (rbvmomi) if that's your thing, it's used in the Chef vSphere plugins.

luminalflux
May 27, 2005



vSphere 5.5 is released :toot:

Now to wait until others have discovered all the upgrade issues before upgrading myself.

luminalflux
May 27, 2005



Yeah, that's really welcome for us since we RDP into the vCenter server to run the desktop client, as everyone here is on a mac.

luminalflux
May 27, 2005



FISHMANPET posted:

So it seems to me that when a new version of vSphere comes out, you upgrade right away, whereas with other software you wait a little while for bugs to be shaken out before you upgrade?

It depends on the featureset. I skipped ESXi 5.0 because of the vRAM idiocy, and went 4.1 to 5.1 as soon as 5.1 was out. vCenter 5.5 has features I need.

luminalflux
May 27, 2005



Martytoof posted:

I have a chassis full of BL25p blades. Do I win the prize for "most anemic VM environment"?

A joint venture we're involved in got a rack of BL360p G1 blades and wondered if it was economical to run VMware on them. They apparently like copying my setups and since I run ESXi Ent+ on DL360 Gen8s, they also needed to run it - on EOL'd hardware that was surplus from some other part of the company.

(They also tried to copy my bandwidth - since I was only pushing 5 Mbit in production traffic on my sites, they ordered that much bandwidth for hosting and their office. Capped of course, whereas my production can burst to 100 and our office has well over 1 Gbit to the world)

luminalflux
May 27, 2005



How painful is it to migrate from vCenter Server to VCSA?

luminalflux
May 27, 2005



evol262 posted:

Archipel looks cooler.

Anyone tried it?

luminalflux
May 27, 2005



Is there an easy way to get serial console access on ESXi / vCenter 5? I don't need a VGA console access usually, just serial to my linux guests would be dandy.

Otherwise I RDP in to a windows macine from my mac to run vCenter client and the experience sucks rear end since it tends to repeat keys, especially when typing the root password.

luminalflux
May 27, 2005



sudo rm -rf posted:

How lovely would performance be if I tried to setup a iSCSI shared vmfs datastore for clustering on a disk array limited to two GB nics? It's a Synology RS814.




Guess where we went from 2 Lefthands with 2x1 GbE (Network RAID 10) to adding 2 4x1GbE nodes? This is 7 ESXi hosts (each with 2x1 GbE) and about 90 guests using shared storage.

It's all about workload, it worked pretty well with light loads but lately some guests were crapping out due to high queue depth and latency (100s in loadavg with basically no CPU load)

Adbot
ADBOT LOVES YOU

luminalflux
May 27, 2005



complex posted:

That's some sweet graphing. How do you pull those stats?

The data is from SNMP (LEFTHAND-NETWORKS-NSM-CLUSTERING-MIB::clusClusterStatsQDepthRead in this example). The graph is from Munin, I either wrote or found an SNMP plugin for Munin that grabbed cluster-wide read/write queue depth, bytes written/read and write/read latency. I'm not sure of the license on it, but the heavy lifting is:

code:
my $read_ops  = $session->get_single(".1.3.6.1.4.1.9804.3.1.1.2.12.48.1.20.1") || 'U';
my $write_ops = $session->get_single(".1.3.6.1.4.1.9804.3.1.1.2.12.48.1.21.1") || 'U';

my $read_bytes  = $session->get_single(".1.3.6.1.4.1.9804.3.1.1.2.12.48.1.22.1") || 'U';
my $write_bytes = $session->get_single(".1.3.6.1.4.1.9804.3.1.1.2.12.48.1.23.1") || 'U';

my $read_depth  = $session->get_single(".1.3.6.1.4.1.9804.3.1.1.2.12.48.1.24.1") || '0';
my $write_depth = $session->get_single(".1.3.6.1.4.1.9804.3.1.1.2.12.48.1.25.1") || '0';

my $read_ms  = $session->get_single(".1.3.6.1.4.1.9804.3.1.1.2.12.48.1.26.1") || '0';
my $write_ms = $session->get_single(".1.3.6.1.4.1.9804.3.1.1.2.12.48.1.27.1") || '0';

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply