Alfajor posted:

I've been working on setting up syslogs from a VMware 6.5 environment, and having a bitch of a time with the ESX part (vCenter is going well).
In the ESX hosts, I set the variable "Syslog.global.logHost" to poing to my syslog server, enable the firewall rule on the host, and messages pour out. However, the bulk of the messages are "debug", and I couldn't find a way to get them to step down to "info" or even "warning".
So I gave up, and reset "Syslog.global.logHost" to blank, disabled the firewall rule for syslog on the host, and even did a "esxcli system syslog reload"... and messages are still flowing to my syslog endpoint, even 2 hours later.

Now I'm assuming that I'm missing something obvious, and the same solution to make syslogs stop going out might be the same as changing the effective logging levels.
Any ideas?

Happiness Commando posted:

I did a firmware update on one of my Dell servers. vMotion to the host now takes about half an hour per VM. Away from it takes the normal one or two minutes

VMware support was useless, unsurprisingly. I should have checked with Dell first. I will probably do that Monday...

BangersInMyKnickers posted:

What NICs are you using? If you say X710 you win the poo poo NIC Lottery.

BangersInMyKnickers posted:

What NICs are you using? If you say X710 you win the poo poo NIC Lottery.

TheFace posted:

I didn't see this answered, but there's a config.xml file for hostd, and vpxd.cfg (also xml file), and you can control the log level there. You're probably set to verbose. For putting them back local, did you change Syslog.global.LogDir at all when you pointed to the syslog server?

Alfajor posted:

I did mess with config.xml and vpxd.cfg, and didn't see the changes of what was pouring into syslog.
The .logDir variable is empty, not populated. I do wonder if that needs to be set so that it can be "shipped" from there... but I don't know if there's really a thing.

At any rate, I ended up disabling everything, and after a few hours, the messages just stopped coming in to GrayLog.
Which was a decision, because even with all the debug data (300k a day per host!), I couldn't really find any useful metrics to dashboard. Just grabbing events from vCenter is enough apparently, so wooohooo, I've moved on from this mysterious syslog gray area.

evol262 posted:

Your cards are dead. Lots of SD cards (and SSDs) go to read-only once new blocks can't safely be written.

All the information you're talking about is only living in memory. It's probably desperately trying to log this fact and failing

Wibla posted:

Seems like it wouldn't be very hard to move to M.2 SSDs for this kind of poo poo.

evol262 posted:

M2 SSDs solve the problem in the same way that SD and internal USB ports used to, except faster and with write leveling (which most SD/USB don't have, since there's no enough IO to worry about it). They'll still eventually go read-only, though I've never seen even a first gen SLC SSD poo poo out this way. One of the earlier MLC vendors known for gaming equipment cheaped out on controllers and had this happen a lot. I don't remember the vendor, but someone here will

evol262 posted:

I don't remember the vendor, but someone here will

anthonypants posted:

Does ESXi say anything about having trouble writing to the SD cards, or can you plug a LiveCD/LiveUSB into the server and see if you can write to the SD cards?

evol262 posted:

Your cards are dead. Lots of SD cards (and SSDs) go to read-only once new blocks can't safely be written.

All the information you're talking about is only living in memory. It's probably desperately trying to log this fact and failing

cheese-cube posted:

Pretty sure your hypervisor boot volume shouldn't be seeing significant amounts of write. You probably just got lovely flash.

Edit: I recently saw 1/5 failure rates on cards for Cisco UCS blades the year before last so don't exclude hardware failure.

Vulture Culture posted:

ESXi doesn't read or write the media once it's running, so if this is for a colo use case instead of something you're turning on and off five times a day, does it actually matter?

k-uno posted:

I'm completely new to VMs, and I have a question about setting them up with socket affinity. I'm a physics prof and want to set up a two-socket workstation for research. The code I run (mostly Mathematica) is very CPU and memory bandwidth intensive, parallelizes very well up to a point, but has core count limitations (won't scale above 16 threads per instance). With one CPU in the system this is no big deal, but in a two socket machine it runs like poo poo by default since it doesn't know how to intelligently distribute data between the memory banks of the two sockets, and the bandwidth loss and latency increase that results from that kills performance. I want this to be a local system since while I have access to a cluster, I do a lot of restarting and tinkering with the code as I'm running it, and the hassle of submitting things into a queue gets really annoying in that case.

I had posted about this in the Intel thread, and it was recommended that I find ways to manually set socket affinity at the OS level, so specific threads are pinned to specific cores. However, it occurred to me that an even better solution could be to build a two socket machine (with at least 16 cores/socket), and run VMs; I would like to run two copies of Windows 10 simultaneously, and set them up so that each copy only accesses one socket and one memory bank. In principle this could get around core count limitations and NUMA issues, which could be a big performance boost if the overhead for running VMs isn't too bad. And in the relatively rare situations where I need communication between that many processes, or more than one bank's worth of RAM, I could just run a single instance with all the resources. However, I've never set up VMs before so I don't really know what I'm doing. Is what I want to do easy to accomplish? If so, what software should I use? VMware or Hyper-V?

Potato Salad posted:

Intel's X cpus are basically xeons with no ecc support (no difference), smaller L3 caches (not too consequential if you're constantly loading/unloading huge vectors from main memory anyway), and include overclocking for CPU and RAM. These have worked miracles for *some* chem simulation, but everyone (rightly) reaches for xeons first without testing out the "I need to decode two streams of 8K video pive for my Plex player while playing 80 Skyrim mods" basement dweller black magic option for their specific small sim use case.

BangersInMyKnickers posted:

This is not true, it will opportunistically write logs to SD if it is large enough (I think 8gb+ is the delineation, but its been a while).

Moey posted:

Syslog server Elastic stack is the solution here.

Wicaeed posted:

Has anyone taken a crack at writing a PowerCLI script to fully Provision a new VM from a template?

I had one at a previous job but I've since lost it, but I remember it wasn't as simple as calling Create-VM and then calling it a day.

Even something as simple as targeting a single datastore for a new VM was complicated because a Datastore & Datastore-Cluster are two distinct objects that need to be evaluated when choosing the datastore.

BallerBallerDillz posted: