Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
So this continuously pisses me off. Everytime I try and upload a file to my datastores on my ESX 4.1 servers, the vSphere client crashes. OK, fine, I'll SCP the file over. Oh wait, everything is owned by root:root with 755 perms.

My guess is that it's crashing on a permission denied error, so how do I fix that? Do I modify the datastore somehow? Do I have to give myself additional permissions? Should the changes I make be on the server via ssh (I doubt it) or from the vSphere client?

Adbot
ADBOT LOVES YOU

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Moey posted:

I was thinking about getting this book, but one if the reviews keeps bringing up that it is quite far out of date.

Would this be a good recommended read still, or is there something better out there? (Already have "Mastering vSphere 5" and "HA and DRS technical deepdive")

It's mostly about concepts and not specifics. So there's a chapter on how to size your VMs, a chapter on storage concepts, network concepts, etc etc. I've got it already, and it was a pretty quick read. If nothing else, it can be good to put in front of a boss to show them how stupid their idea is.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Kachunkachunk posted:

I think domain authentication of ESXi boxes is purely for tracking and ease of credentials management, really. If you fire a user, you don't have to change all the server passwords, for example.

Not saying I like AD authentication of ESXi, though.

Can users be managed through VCenter Server (or whatever the gently caress it is)?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Mausi posted:

Physical dependency cards, low latency systems, flakey RDMs, Anything that licenses by MAC address, non-stop systems requiring more than 1 cpu. Of course most people don't have these issues.

You can change the MAC in the OS to get around MAC address licensing, can't you? Of course if you're starting out virtual then just be sure that VM always has the same MAC.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Mausi posted:

You can change it in the .vmx file as well. However it becomes a problem when you inadvertently change the MAC and haven't saved it anywhere which VMware newbies regularly do. This was/is often a problem for people new to virtualisation especially when P2Ving from an old environment.

I remember you can only set certain ranges in the GUI, does editing the VMX let you use any MAC?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Kachunkachunk posted:

Also... FFFFFFFFFFFUUUU-
Okay, so the Linux kernel bug detailed here may likely affect anyone running ESX 4.0 U1 and earlier, later this week. ESXi users, or at least anyone running patch-6 for ESX 4.0 and later should be fine, however.

In case some of you do not already know, ESX "Classic" runs with a Console OS, which is basically a repurposed derivative of RHEL 3 or 4... along with all of the applicable bugs and quirks that may accompany such a release (resolved with patches).

If Red Hat bug 479765 triggers on an ESX box's Console OS, it will probably result in a Lost Heartbeat purple screen eventually (vmkernel/hypervisor continues to run but notices the Console OS has stopped responding, then panics).

Patch up by Saturday if you're running such an old release.

Wha, patch-6? What does that mean? I'm working on patching my 4.0 ESX hosts (currently RTM or Update 1) and it looks like Update 4 is the latest, with 4 updates relased in addition after update 4. What does "patch" specifically mean in this context, and what does the 6 mean?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Kachunkachunk posted:

Specifically known as "P06," which is only pissing me off, since I also have trouble figuring out exactly what release level that is.
Rest assured, it's probably still something that predates Update-2, even. You'll surpass that with a regular update rollout.

Update-x is a roll-up package, like a Service Pack. It may also include other specific fixes and stuff. P0x would be a specific minor rollup that looks like an individual patch. Installing Update-4 will net you all the benefits and fixes from prior updates/releases cumulatively.

Edit: I think this is the relevant kernel update for ESX 4 that one would need: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1013127, otherwise known as ESX400-201005001.

Also see: http://scientificlinuxforum.org/index.php?s=2c11935c7c49be320ff30d9a09376a6e&showtopic=1695&st=0&#entry11777 and/or http://blog.toracat.org/2012/06/leap-seconds-who-cares/ for some information on what kernel revision needs to be met or exceeded.

Ugh, thanks VMWare, can't wait to get a VCenter Server and not have to worry about all this.


Moey posted:

I would assume the 6th patch level for ESX 4, so get to 4.1

Not sure on what the status is of current maintenance, do we need to be current to go to 4.1?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Kachunkachunk posted:

No vCenter? Okay, so you'll probably be working on this in a maintenance window or something.

Get Update-4 for 4.0 and interactively install it on each box using the command-line. Won't take you very long for each host, but it will require rebooting and downtime (especially since you don't have vMotion without vCenter?).

If you do go to 4.1 and later, you generally do not need to update your existing install before it upgrades. Otherwise the worst case is to install over it and retain your local VMFS partition (in case you have VMs there).

I found this page: https://my.vmware.com/web/vmware/details/esx41u2/dHdlYnRoKmRidGRkKg==

And it lists an update to install before going from 4.0 to 4.1. So I'd install that and the the Update 2.

E: And this confirms that the same key is good: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1024256
I've got a downtime on one of the servers tonight. I was going to patch to 4.0u4, but I guess I'm going to 4.1u2!

FISHMANPET fucked around with this message at 22:53 on Jun 26, 2012

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
I'd put in a vote for vSphere Client is complete poo poo at rendering anything.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Just the vSphere client, or something else? I've got no problems running the client on Win 7 Enterprise x64.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Well poo poo, maybe when I actually have a vCenter server I'll be able to tell them apart.

But why would you install it on a desktop OS?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Holy poo poo, if I'd known I was going to be presenting my 2 month old VMWare implementation plan in front of my boss' boss, I would have worn nicer pants.

Also, can anyone critique my physical layout?


The switches are 10G, all the connections between hosts and storage is 10G, connection back to the router close is currently 1G, but we might go to etherchannel and do double or quadruple ports there.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Mausi posted:

No separate vMotion?

I think Vmotion is getting lumped in with Management traffic, or FT, I don't remember which. I made the plan in my head 2 months ago, and then pretty quickly the boss' boss said "nope, gonna be years before we do that" and now today he's ready to start looking at it. So I gotta kinda scramble to remember what I did last time.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
As I went through it last time I was reading through Masterving VMware vSphere 5 and VMware vSphere Design (which, though it's for 4, the same concepts apply). It was much more of a possiblity back then so I was planning it for real, not just for funsies, but I'm going to be poring my material again to make sure I don't gently caress it up.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Corvettefisher posted:

Yeah you can do vMotion on Management seeing how management usually is idle or very low bandwidth traffic.

Are you doing it like this
(crappy ms paint drawing)


Management Primary nic0, Fixed failover to standby adapter nic1
vMotion Primary nic1, Fixed Failover to standby adapter nic0

This eliminates and congestion while addressing failure scenarios.

Yeah, that'll probably be what we do on the ports where two services are sharing the same pair.

Mausi posted:

Also, looking at your whiteboard there, it implies that you're going to run 4x 10GbE connections to each host, which is ridiculous.
Presumably you're running a single 10GbE to each host from each physical switch, then vLan segregating your traffic types? At which point carving off vMotion or anything else is an arbitrary task.

Actually the plan has 8 10GBe to each host :catstare:. Using Dell 8024F switches and Intel DP 10GB SFP+ NICs, the price per connection is around $600, which I don't think is bad at all. Also, as far as I know, there isn't a way to segregate bandwith on VLANs if I segment out the traffic on a 10GB link, though if that's changed that would be awesome.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Well apparently I just got super carried away, based on your reactions, so that's good to know.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Since Intel released the E5-2400 series, Dell has also released the R320, R420, and R520. They're signifigantly cheaper than the equivalent E5-2600 models, and when I talked about the $10k per node cost of the R720 my boss was kind of shocked, so getting an equivalent R520 for 2/3rds the cost is certainly appealing.

I know I'm looking at only one QPI with the E5-2400, but VMware is really good at not scheduling a VM on two separate physical CPUs, so that's not too big of a problem. It's tri channel memory rather than quad channel but I don't think that increase is worth the extra cost. To get the full advantage of triple channel memory I'd have to load up all 12 slots with 16GB sticks, which gives 50% more memory than I'm entitled too, but we'd have enough memory to bring down a machine for maintenance and not have to worry about it.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
So is there a reason to choose NFS datastores over something block based? It can be as good as block as far as I can tell, but it seems like it just started as an afterthought and snowballed from there, and when starting from scratch there's no reason to choose NFS if you have iSCSI or FC.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Reading through Scott Lowe's vSphere 5 the end of the Storage chapter says "Why would you use each of the 3 kinds of datastores (iSCSI, FC, NFS)" and the only reason I can think to choose NFS is you don't have block available, or someone is holding a gun to your head and forcing you to for reasons you have no control over.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
So onto another question, to hard drive or not to hard drive?

I guess the root of my question is what happens to an esxi host running off of SD card but with swap space on a local hard drive if the hard drive dies (and therefore swap space disappears).

And then any thoughts on installing on flash media vs installing on a hard drive?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
So here's a new diagram, not testing the configuration maximums:


We're looking at Dell servers, and they all have the ability to run ESXi off of SD cards. In fact, it looks like if we get VMware directly through Dell, it's required (at least it is when you configure the server, who knows what our Rep can do).

It makes sense to have one hard drive for swap and other scratch stuff. I know that I can't join an HA cluster without a configured swap space. So I'm wondering what happens to a running server if the local hard drive dies and it loses its swap space. Is it worth it to spend the extra for a second hard drive and possibly a RAID card. And at that point is it worth it to spend the extra for SD card (which can come out to more than the price of a hard drive).

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

KS posted:

You can buy 4gig SD cards at staples for $7. Dell's customized ESX is downloadable as well, so preinstall only saves you a few minutes, and those SD cards are ridiculously expensive.

Not sure what swap space you're talking about. If you're talking about swap to host cache, you're better off with a single SSD. You can forego this if you have headroom on the hosts. It's a fairly new feature and only kicks in for a situation that you want to avoid anyways.

If you're talking about virtual machine swap files, and you store them separately from the VM for some reason, you want that to be redundant and fast. However, you really probably don't want to be doing this either unless you're replicating the VMs and want to avoid replicating the swap file. Even then, you're better off putting swap on shared storage for vmotion purposes.

I'm talking about Hypervisor swap. Normally I wouldn't care because if the hypervisor is doing swapping then holy gently caress hold onto your butts, but I've read that you can't join a server to an HA cluster unless it has local hypervisor swap, so I'm not sure what the loss of that swap would do to a machine.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

KS posted:

I think that's really outdated -- you had to turn it on back in 2.5. Nowadays even on an SD card it sets up a 1GB swap space on the card and can join fine, but god help you if it ever gets into a swapping situation. Have never run into that error from 4.0 till now.

This is the kb, it says it still applies to 5.0. Though if it only wants 1 GB, I could easily dump that on the SAN super cheaply, and not have to worry about disks at all.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Misogynist posted:

Hypervisor swap is so completely uncommon since they implemented memory compression in 4.1 that seriously, gently caress yourself and your career if you let your environment get so oversubscribed that you're swapping.

I'm well aware that it's a situation that I should strive to avoid, but ESXi requires it for HA (which we'll be using), and I can't find anything that tells what happens when that swap space disappears in an HA cluster.

Corvettefisher posted:

Before we go make more recommendations which license are you getting?

We'll be getting Enterprise licenses. There's a possibility that we might get some Standard for a separate cluster, but that's for instruction, not infrastructure production.

I've looked over the Enterprise plus and I don't think our enviroment will be big enough to justify the added cost for stuff like storage DRS and VDS.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
My boss asked me if we should run the vCenter on Oracle and I almost instinctively did a spit take, though at the time all I had was my gut to say that was a bad idea. But it sounds like it's still a good idea? I'm guessing since MSSQL is the most frequently used it's the best?

Another question he had that I couldn't answer about networking. He was wondering if traffic between two hosts when those hosts are on separate vlans is able to be routed on the vSwitch or if it has to go out to the router and back in.

On that same note, is there any kind of deep dive book for networking? I was hoping the vSphere Clustering Deepdive was part of a series but it appears not, and I'd really like to be able to dig into some of this stuff deeper.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

1000101 posted:

If you have 2 VMs on the same host but different VLANs then the traffic will have to go out to some device to route the traffic from one VLAN to the next. If this device is a VM that's doing the routing it could stay in the host. Otherwise yeah it's going to go out to some real router on the network.

Would the Cisco Nexus 1000V (or whatever their vSwitch replacment is) be able to do that?

On the other hand, I don't really know how much traffic there would be between VLANs that it would matter, but I'm sure I can partially blame our hosed up VLANs for it.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
What in the gently caress. So what do we pay for now? Just X per CPU? Is there going to be a core limit on CPUs?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Just one more thing...

ESXi for ARM! Now you can run ESXi on all your mobile devices!

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

evil_bunnY posted:

You pay per socket now, and each socket license entitles you to an amount of vRAM usage. It's retarded.

I meant "now" as in going forward with 5.1, not this exact instance now. I'm well aware of the pricing now, I'm just trying to think back to 4.0. Wasn't there a core limit per CPU or something?

I guess we'll just have to wait until Monday.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Current project, replace an ESX3 machine with ESXi5 :feelsgood:

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Moey posted:

Was this one of those stories where a server gets bricked inside of some old closet and is just recently discovered?

For content:

Are those licensing changes they announced effective immediately on ESXi 5.x, or does it come into play once 5.1 is released?

If it is in effect now, I am tempted to update my hosts from 4.1 to 5.0 sooner than later.

No, it's a single shot that was set up in a research lab a million years and nobody ever looked at it again. I'm hoping to buy a second server and setup vCenter for them, since during experiments there has to be 100% uptime.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
Can I join my ESX3 server into a cluster with an ESXi5 and vMotion the guests from the ESX3 server to the ESXi5 server? Once they migrate off the ESX3 server will be retired, but if I can do the migration seamlessly I'll be a superstar hero.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
So ESX3 supports virtual machine hardware version 4, as does ESXi5. And ESX3 supports VMFS3, which ESXi5 does as well, so I should be fine on that front (with the caveat that I should upgrade the hardware to version 8 when I can, and svMotion everything to a VMFS5 volume). Is there anything else I need to be concerned about?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
What exactly are CPU requirements for FT? I'm looking at running an FT VM on a host with a 55xx chip and another host with an E5-24xx chip. Does that use EVC to properly mask the correct CPU bits so it all runs at the 55xx level? I can't find anything definitive in the recommendations, other than that the CPUs need to be compatible, but there's no explanation of what that means.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

Mierdaan posted:

I swear, VMware's documentation is actually really good!

That hasn't been updated since October, so it doesn't include any modern CPUs (like the E5-24xx I'll be using), and also isn't very clear. Do all the CPUs have to be under the same bold heading, or do all the CPUs have to be on that list?

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
That's about EVC for vMotion, nothing about FT. And C'mon VMWare, I'd like to know if this will work before I spend $50k on it, not after. That Site Survey tool is pretty stupid in that context.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
So the reason I ask, is a coworker who is a programmer but knows just enough to be dangerous did some research on my plans to use FT between the R710 and the R520, and found a vSphere 4 FT FAQ that included the following:

quote:

ESX host processors must be VMware FT capable and belong to the same processor model family. VMware
FT capable processors required changes in both the performance counter architecture and virtualization
hardware assists of both AMD and Intel. These changes could only be included in recent processors from both
vendors: third-generation AMD Opteron™ based on the AMD Barcelona, Budapest and Shanghai processor
families; and Intel® Xeon® processors based on the Penryn and Nehalem micro-architectures and their
successors. For details please refer to http://kb.vmware.com/kb/1008027.

So I understand what EVC and what it does, but EVC has been around for a long time yet the FAQ makes no mention of it and basically says the CPUs have to be in the same family. So either that's still true, in which case it would be nice to find a vSphere 5 FT FAQ that mentions that, or it's not true and EVC can keep CPU properly masked, in which case it would be nice to find a document that says "Hey, we removed this limitation!"

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

madsushi posted:

Just so we're all clear here, you know that FT is not usually a good idea? Unless you happen to have a 1-core VM that for some reason needs an insane amount of uptime, the overhead and configuration you'll have to deal with for FT is usually wasted.

Yeah, I'm well aware. This is for a research lab that does experiments that cost hundreds of thousands of dollars to run. If a server collecting data goes down during an experiment then you've lost that data and have to rerun the experiment. I'm hoping what they need protected is 1 core, and if not I'll figure out how to reduce it to 1 core.

And if it ends up being that we need two identical servers then we can buy another server, I just need to know if that's necessary or not.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams
In unrelated news, curse you VMware for moving Storage vMotion into Standard, it meant I had to defend myself against a claim of being lazy from my director for wanting DRS in Enterprise (though I think it may have been a devil's advocate thing from the director, but the managers didn't get that memo, so gently caress that discussion).

E: And unrelated, to possibly play devil's advocate with myself, any thoughts on getting production vs basic SNS vs only getting the first required year and flying by the seat of your pants?

E2: And to clarify, I'm actually working in two seperate VMware deployments currently, so if some of my questions seem like they don't all mesh together, it's because they don't. Like saving money on support for one cheap department vs dumping money into hardware for another rich department.

FISHMANPET fucked around with this message at 22:34 on Aug 30, 2012

Adbot
ADBOT LOVES YOU

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams

evil_bunnY posted:

And then not getting updates? Sounds good mate.

That's kind of what I thought about going the no support route, but any thoughts on Production vs Basic?

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply