movax posted:

So I've given up on ESXi and trying out Proxmox now.

One thing I can't figure out... in ESXi, you spawned vmkernel NICs for access to management interface. How does that work on Proxmox? I can't seem to find an obvious place to control where the management interface listens.

It is a DeskMini 310 w/ single NIC I'm finally going to colo, so in my ideal world, I'd like too:

Untagged <--> eno1 <--> vmbr0 <---> pfSense VM WAN Interface
VLAN 10 <--> eno1.10 <--> vmbr0.10 <--> Proxmox management interface

And then vmbr1 is Proxmox, pfSense LAN, TrueNAS and my Linux VM. This will stop me from getting locked out of Proxmox if pfSense is down / let me use it for initial setup at home.

Mr Shiny Pants posted:

Clustering physical machines and running microservices on top of them is Erlang in a nutshell. Erlang is just one VM (BEAM) that can run actors (microservices that pass messages) with supervisors (if you use OTP). You need to program in Erlang but from a technical standpoint they are roughly comparable at what they accomplish. Erlang is elegant though (like really elegant), Kubernetes is not IMHO.

All that is old is new again.

It was more of a reply to: This Kubernetes thing, at least outside of the alluded "hyperscaling setting", feels like someone's pulling a huge practical joke on me/us.
I got that feeling as well.

BlankSystemDaemon posted:

This is just rumours, but with Facebook having moved WhatsApp from the FreeBSD+Erlang setup that Jan Koum originally set up to their own in-house PHP on Linux solution with Twine (their scale-out container-management solution), they've supposedly had to throw 100-200x the number of machines to handle the same level of traffic.

Actuarial Fables posted:

Essentially, any IP address you configure under Networking is a management interface.

The documentation has an example of what you're trying to do with the NIC, under Host System Administraton > Network Configuration > 802.1q. I don't think you can make this kind of configuration through the GUI, so you'll have to edit /etc/network/interfaces manually.

code:

Example: Use VLAN 10 for the Proxmox VE management IP with VLAN aware Linux bridge

auto lo
iface lo inet loopback

iface eno1 inet manual


auto vmbr0.10
iface vmbr0.10 inet static
        address  10.10.10.2
        netmask  255.255.255.0
        gateway  10.10.10.1

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes

movax posted:

Bob Morales posted:

It's like Microsoft buying HoTMaiL all over again

underlig posted:

I didn't know 2012r2 with Hyper-V was that common anymore, but apparently KB5009624 breaks Hyper-V
"could not be started because the hypervisor is not running"

We're running Esxi instead, but that's also making the beginning of tyol 2022 hard,

Bob Morales posted:

Hyper V isn't already broken?

Bob Morales posted:

I have two Hyper-V VM's in the same cluster that both had drive errors (VHD's on the same physical drive array)

One Linux, one Windows (actually our main file server). Had to do file system checks and reboot them and everything seems to be fine now. Do have a couple corrupt Excel files and such reported but was able to pull them from backups.

Anything that could have caused this? They are running on a Dell VRTX that isn't reporting any issues with the storage systems. Weird coincidence?

Mr Shiny Pants posted:

Running ReFS by any chance? There are some problems with the latest round of updates it seems; these also include problems with Hyper-V.

Happiness Commando posted:

WSL uses Hyper-V. Were you using Hyper-V to run your Ubuntu VM, and did the WSL installer change some Hyper-V setting or create some network bridge or something?

SlowBloke posted:

If he used virtualbox to run the vm, adding wsl2 will make it inoperable.

Pile Of Garbage posted:

With VMware is there a way to exempt a VM from DRS but at the VM level instead of the cluster? I know you can do it at the cluster level but is there a tag or something I can set on a VM that will make DRS ignore it?

I'm building VM templates with Packer and as part of the build it mounts a floppy to the VM containing the autounattend.xml and some scripts. The issue is that if the VM is migrated via vMotion the floppy drive gets automatically disconnected from the VM. This either causes the build to stall or only complete partially due to missing scripts. This issue seems to arise fairly frequently because DRS appears to be hyper-aggressive on clusters I'm deploying to despite being configured with default settings.

Pikehead posted:

You could also put the floppy image on a shared datastore or library, so vmotioning isn't an issue.

(I believe this will work - I haven't had to mount a floppy image in .. well, ever).

Internet Explorer posted:

Yes. Old UI, but should be fairly similar.
https://www.yellow-bricks.com/2018/03/28/disable-drs-for-a-vm/

Pikehead posted:

You could also put the floppy image on a shared datastore or library, so vmotioning isn't an issue.

(I believe this will work - I haven't had to mount a floppy image in .. well, ever).

Cyks posted:

In vcenter you set it up as an affinity rule.

https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.resmgmt.doc/GUID-FF28F29C-8B67-4EFF-A2EF-63B3537E6934.html

Specifically VM to Host group affinity rule.
Actually just doing manual for the individual vm as the other poster said is easier I’m overthinking it too early in the morning.

Pile Of Garbage posted:

That's still at the cluster level though and requires that the VM already exist. I'm deploying new VMs with Packer using the vsphere-iso builder which doesn't exactly give you options to mess with things in the environment other than the VM it is deploying.

Packer creates the .flp file on the fly and uploads it to the same datastore as the VM so it's already on shared storage. From what I observed the floppy image isn't being unmounted but rather the floppy drive is being disconnected from the VM when it vMotions. As part of the build Packer also mounts two ISOs to the VM on separate CD-ROM drives and those are unaffected by the vMotion.

Again that would require configuration at the cluster level and can't be done prior to the creation of the VM. I'd really like to avoid having to orchestrate something alongside Packer to do this.

Maybe I should just figure out what's going on with DRS. I've seen it vMotion a new VM three times in the space of two minutes basically immediately after the VM was created. Surely that shouldn't happen as Packer is just pointing at the cluster and relying on DRS initial placement to choose a host. Perhaps that initial placement recommendation from DRS is cooked?

Pile Of Garbage posted:

That's still at the cluster level though and requires that the VM already exist. I'm deploying new VMs with Packer using the vsphere-iso builder which doesn't exactly give you options to mess with things in the environment other than the VM it is deploying.

Packer creates the .flp file on the fly and uploads it to the same datastore as the VM so it's already on shared storage. From what I observed the floppy image isn't being unmounted but rather the floppy drive is being disconnected from the VM when it vMotions. As part of the build Packer also mounts two ISOs to the VM on separate CD-ROM drives and those are unaffected by the vMotion.

Again that would require configuration at the cluster level and can't be done prior to the creation of the VM. I'd really like to avoid having to orchestrate something alongside Packer to do this.

Maybe I should just figure out what's going on with DRS. I've seen it vMotion a new VM three times in the space of two minutes basically immediately after the VM was created. Surely that shouldn't happen as Packer is just pointing at the cluster and relying on DRS initial placement to choose a host. Perhaps that initial placement recommendation from DRS is cooked?

Pikehead posted:

If the floppy is on the same datastore as the vm then I wouldn't expect issues, but obviously there are. I can't think why it would disconnect - would there be anything in the esxi logs as to what's going on?

Pikehead posted:

With regard to a very aggressive DRS - that's again a bit unexpected - I thought DRS by default runs every 5 or 15 minutes and not at the frequency you're seeing. Is the vms being powered on each time? - That would possibly make sense, as when DRS is fully automatic it's involved each time a vm is powered on.

Pile Of Garbage posted:

I'll have a look at the logs if any when I'm back at work next week. It is quite strange as normally I'd expect it to cause vMotion to fail outright.

Pile Of Garbage posted:

For the Packer build the VM is only powered-on the one time. In the guest OS it does an unattended install of Windows Server 2019 so there are a couple of soft reboots and then at the end it powers off so that Packer can emove the CD-ROM and floppy drives and convert it into a template.