Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Matt Zerella
Oct 7, 2002

Norris'es are back baby. It's good again. Awoouu (fox Howl)

Potato Salad posted:

you could use proxmox at home but you won't be learning any skills that are useful for your career imo

Nah, you can use Terraform with it OOTB and packer. Plenty of stuff to learn.

Adbot
ADBOT LOVES YOU

Potato Salad
Oct 23, 2014

nobody cares


well yeah

so use a modern hypervisor

there's a free gitlabs tier, put up some teraform and go learn proper cdci

Potato Salad
Oct 23, 2014

nobody cares


does.... anyone use proxmox in enterprise, at scale? maybe it's me who is out of touch

SlowBloke
Aug 14, 2017

Potato Salad posted:

does.... anyone use proxmox in enterprise, at scale? maybe it's me who is out of touch

Our hpc setup runs a proxmox partition for stuff that wouldn't fit in k8s. I've heard it's not that weird as a setup if you don't need to run windows loads.

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.
ESXi is clearly hosed under Broadcom, if IBM hadn't simultaneously been wrecking Red Hat I'd actually be betting on Red Hat Virtualization/ oVirt picking up some market share.

KVM itself is completely rock solid and has been for ages. There's not going to be any real replacement for how big ESXi was because operating an on-prem virtualization farm will become lostech and everyone will just pay AWS more money.

Edit: if anything in the longer term I'd bet that the groups that continue to run on-premises will move to a bare-metal kubernetes solution without any hypervisor. You can run a KVM guest as a kubernetes pod if you need actual VM isolation.

Twerk from Home fucked around with this message at 23:40 on Jan 5, 2024

SlowBloke
Aug 14, 2017

Twerk from Home posted:

There's not going to be any real replacement for how big ESXi was because operating an on-prem virtualization farm will become lostech and everyone will just pay AWS more money.
Does that mean that us VCP holders need to go get word of blake membership cards?

Kreeblah
May 17, 2004

INSERT QUACK TO CONTINUE


Taco Defender

Potato Salad posted:

you could use proxmox at home but you won't be learning any skills that are useful for your career imo

I'm really not worried about picking up VMware skills. I've already done things like written a custom VM build system to connect to it using the SOAP and REST APIs and build VM templates and deploy them to VMs, developed an integration for it for our CMDB, etc. Like I said, I'm using the free ESXi license because I already use it at work and I'm familiar with it. Though, these days, I do a lot less with VMware directly, since I've been doing a lot of security policy work.

SlowBloke posted:

At this point in the post acquisition stage, getting esxi skills today would be like like getting novell groupware knowledge in the early two ks. Broadcom is going to kill esxi no if no buts, it's just a matter of time.

Also, this. I don't see VMware doing well under Broadcom with the changes they've announced, between the licensing changes and selling off all the workstation products.

Mainly, I'm just wondering whether Proxmox is reliable and maintained well enough to replace ESXi at home at this point. I've never run into it in the wild, so I have zero experience with it, but at least on paper, it sounds like the platform I'd want to move to (free, supports Windows, Linux, and FreeBSD guest VMs, has some sort of web UI for easy admin tasks, has an API, supports ZFS for VM storage, supports device passthrough so I can pass my GPU through to my Plex VM).

Potato Salad
Oct 23, 2014

nobody cares


I meant go learn KVM

nobody should be learning more VMware rn

proxmox in enterprise? ehhhhhhhhhhh sounds like there's one poster who has it for a kinda standalone thing, that's hardly a vote in confidence for any notion that serious businesses use proxmox in robust devops stacks

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.
I have been skimming the Proxmox support forums, it's a mix of hobbyists that have no idea what they're doing running into predictable problems with their stupid home setups, and pros with what sounds like a reasonable, modern environment experiencing a bunch of problems with HA.

Here's an example of the latter: https://forum.proxmox.com/threads/unexpected-fencing.136345/


quote:

I have a 28 node PVE cluster running PVE 7.4-16. All nodes are Dell R640/R650 servers with 1.5 or 2TB of RAM and Intel Xeon Gold CPU's. They all have 2 x 1GB NIC's and 4 x 25 GB NIC's.

HA sounds completely busted for them. Apparently that's around the effective max size of a proxmox cluster right now before it collapses under its own weight, judging by responses.

SlowBloke
Aug 14, 2017
If I had extra metal and azure credits, I would test Azure HCI before investing time in KVM but alas I have neither.

Potato Salad
Oct 23, 2014

nobody cares


can you do azurestack HCI homelab for a good price

Kreeblah
May 17, 2004

INSERT QUACK TO CONTINUE


Taco Defender

Potato Salad posted:

I meant go learn KVM

nobody should be learning more VMware rn

proxmox in enterprise? ehhhhhhhhhhh sounds like there's one poster who has it for a kinda standalone thing, that's hardly a vote in confidence for any notion that serious businesses use proxmox in robust devops stacks

Eh, if I were thinking about going with KVM directly, I'd just use bhyve instead. I like FreeBSD more than I do Linux, plus it natively supports ZFS.

Twerk from Home posted:

I have been skimming the Proxmox support forums, it's a mix of hobbyists that have no idea what they're doing running into predictable problems with their stupid home setups, and pros with what sounds like a reasonable, modern environment experiencing a bunch of problems with HA.

Here's an example of the latter: https://forum.proxmox.com/threads/unexpected-fencing.136345/

HA sounds completely busted for them. Apparently that's around the effective max size of a proxmox cluster right now before it collapses under its own weight, judging by responses.

Hmmm. Good to know. I'd only have it on a single node, but that's still good info to have. I've mostly found the people with broken setups who don't know what they're doing, so it's good info seeing what actual problems it has. Thanks!

Wibla
Feb 16, 2011

Kreeblah posted:

Also, this. I don't see VMware doing well under Broadcom with the changes they've announced, between the licensing changes and selling off all the workstation products.

IMHO, VMware is hosed long-term, but Broadcom is going to squeeze existing customers for a while before they can switch to more cost-effective solutions.

Kreeblah posted:

Mainly, I'm just wondering whether Proxmox is reliable and maintained well enough to replace ESXi at home at this point. I've never run into it in the wild, so I have zero experience with it, but at least on paper, it sounds like the platform I'd want to move to (free, supports Windows, Linux, and FreeBSD guest VMs, has some sort of web UI for easy admin tasks, has an API, supports ZFS for VM storage, supports device passthrough so I can pass my GPU through to my Plex VM).

Proxmox is fine for home use. Don't enable clustering just for shits and giggles, read the docs first. I would consider HA an advanced feature.

Potato Salad posted:

can you do azurestack HCI homelab for a good price
:frogout:

Wild EEPROM
Jul 29, 2011


oh, my, god. Becky, look at her bitrate.
h
y
p
e
r
-
v

BlankSystemDaemon
Mar 13, 2009



Potato Salad posted:

well yeah

so use a modern hypervisor

there's a free gitlabs tier, put up some teraform and go learn proper cdci
Has the dust settled after Hashicorp changed the license of Terraform to a business source license?

If I was a new company looking at it, I'd be very careful, since they've not only shown that they can change the license, but will do so, if it benefits them and nobody else.

Twerk from Home posted:

ESXi is clearly hosed under Broadcom, if IBM hadn't simultaneously been wrecking Red Hat I'd actually be betting on Red Hat Virtualization/ oVirt picking up some market share.

KVM itself is completely rock solid and has been for ages. There's not going to be any real replacement for how big ESXi was because operating an on-prem virtualization farm will become lostech and everyone will just pay AWS more money.

Edit: if anything in the longer term I'd bet that the groups that continue to run on-premises will move to a bare-metal kubernetes solution without any hypervisor. You can run a KVM guest as a kubernetes pod if you need actual VM isolation.
Unless a company can make actual use of the elasticity of the Amazon butt, there's little reason to pay for it as it isn't actually cheaper if you've got a steady-state production system (ie. one that just periodically needs to expand).

Selfishly, I hope bhyve will get the little bit of TLC it needs to get to the point that it can be used in production by everyone.
As it is, the main pain-point is being able to transfer guests between hosts - but that's being worked on, and since most of the things, that people want the high-availability for, can be done further up-stack, it's production ready for the vast majority of use-cases.

RVWinkle
Aug 24, 2004

In relating the circumstances which have led to my confinement within this refuge for the demented, I am aware that my present position will create a natural doubt of the authenticity of my narrative.
Nap Ghost
I think the best part of proxmox is that they have their own backup service (PBS) so you don't have to deal with all the bullshit like paying extra just so backups are supported and then paying extra for 3rd party software. Backup is fundamental and VMWare treats it like some fringe benefit.

Kibner
Oct 21, 2008

Acguy Supremacy
What would be a recommended hypervisor running on Linux that will support Windows VMs? This is for home usage. KVM or Proxmox seem like the two mentioned most recently on this page but I do not know enough about the benefits/drawbacks of both to make an informed decision.

I am looking to switch from Windows to Linux (haven't decided on distro, yet) for my main machine. However, I need to bring over a Windows VM with the move. It is currently running under Hyper-V. I don't plan on spinning up other VMs on a regular basis for this machine. I know as part of the move, I will also need to figure out some way to convert the Hyper-V VM to whatever hypervisor I end up deciding on.

RVWinkle
Aug 24, 2004

In relating the circumstances which have led to my confinement within this refuge for the demented, I am aware that my present position will create a natural doubt of the authenticity of my narrative.
Nap Ghost
KVM is the Linux hypervisor and proxmox is a web interface for managing KVM. There are other front ends but they weren't very mature when I tried them like 4 years ago.

Eletriarnation
Apr 6, 2005

People don't appreciate the substance of things...
objects in space.


Oven Wrangler
Yeah, as I understand it Proxmox is fundamentally Debian with KVM, LXC, various storage/networking features and a nice web interface for managing them. It can do a lot out of the box but also you can pretty much just install whatever you want with apt if there are any features you need in the base OS instead of in a container/VM.

There appear to be a few straightforward methods to migrate a VM guest from Hyper-V: https://pve.proxmox.com/wiki/Migration_of_servers_to_Proxmox_VE#HyperV

Installation is very easy and it runs well even on pretty old hardware, so I'd recommend giving it a try if you have a spare machine.

However, it doesn't have a desktop environment by default so if you are looking for a desktop Linux distro that can also host VMs then I'd just get Ubuntu/Fedora and run virt-manager to manage KVM.

Eletriarnation fucked around with this message at 17:11 on Jan 9, 2024

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



I've been pretty happy with Proxmox for my home lab. It might not see a ton of use in the corporate space, but it's a pretty high speed low drag way to slot a web managed hypervisor into your home network. And you can just ssh in and give qm commands if you want instead.

Kibner
Oct 21, 2008

Acguy Supremacy
Ahh, that makes a lot of sense. Thanks for the explanations and conversion link! I’ll have to play around a bit on a separate machine to get things writing how I want before making the move.

Kibner
Oct 21, 2008

Acguy Supremacy
I'm sure this is nothing new to you guys, but I found this video on the subject of KVM and figured I would link it here since it is by someone I know has been talked about positively before in these forums:

https://www.youtube.com/watch?v=BgZHbCDFODk

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



Huh, this is a neat experiment from the Podman crew. Super new, but might be useful for temporary vms for different arches or if you need a special kernel and are famliar with the general syntax of podman/docker.

https://github.com/containers/crun-qemu

Doesn't seem like the intended use case is long-running vms though.

Moey
Oct 22, 2010

I LIKE TO MOVE IT
First the Broadcom takeover, now HPE snatches up Juniper?

It's time for a career change. Every company/product stack that I enjoy is apparently on a conveyor belt within a slaughterhouse.

namlosh
Feb 11, 2014

I name this haircut "The Sad Rhino".

Kibner posted:

I'm sure this is nothing new to you guys, but I found this video on the subject of KVM and figured I would link it here since it is by someone I know has been talked about positively before in these forums:

https://www.youtube.com/watch?v=BgZHbCDFODk

Thanks for this… I actually had the exact same question you asked earlier, so this helps.

Anyone know if there’s a way to have a windows vm with dual monitor support? I figure maybe it’s possible using GPU passthrough but is that the only way?
e: should have specified that the host would have two monitors… and I’d like to use the vm using both of them. Sort of a desktop replacement thing for work

namlosh fucked around with this message at 15:27 on Jan 10, 2024

Watermelon Daiquiri
Jul 10, 2010
I TRIED TO BAIT THE TXPOL THREAD WITH THE WORLD'S WORST POSSIBLE TAKE AND ALL I GOT WAS THIS STUPID AVATAR.

namlosh posted:

Thanks for this… I actually had the exact same question you asked earlier, so this helps.

Anyone know if there’s a way to have a windows vm with dual monitor support? I figure maybe it’s possible using GPU passthrough but is that the only way?
e: should have specified that the host would have two monitors… and I’d like to use the vm using both of them. Sort of a desktop replacement thing for work

i was able to do this via pci-e passthrough

Saukkis
May 16, 2003

Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.

namlosh posted:

Anyone know if there’s a way to have a windows vm with dual monitor support? I figure maybe it’s possible using GPU passthrough but is that the only way?
e: should have specified that the host would have two monitors… and I’d like to use the vm using both of them. Sort of a desktop replacement thing for work

I used to do that with VirtualBox running on Linux. It was just two separate windows I could position where I want. Should also be possible with KVM and Virt-manager, but I didn't get it to work last time I tried.

namlosh
Feb 11, 2014

I name this haircut "The Sad Rhino".

Watermelon Daiquiri posted:

i was able to do this via pci-e passthrough

Saukkis posted:

I used to do that with VirtualBox running on Linux. It was just two separate windows I could position where I want. Should also be possible with KVM and Virt-manager, but I didn't get it to work last time I tried.

Thanks for the replies… while I’ve never tried it, I’ve always heard that pic-e pass through was a pain to set up correctly. If I can’t just get it to work via kvm and virt-manager, any tips on how to go about it successfully are appreciated.

Watermelon Daiquiri
Jul 10, 2010
I TRIED TO BAIT THE TXPOL THREAD WITH THE WORLD'S WORST POSSIBLE TAKE AND ALL I GOT WAS THIS STUPID AVATAR.
oh it is a pain. i got it working ish on ubuntu, but i run into an issue where the entire computer freezes after i shut down the vm. It has something to do with what all i'm sending into the kvm but since i need the usb3 and hdmi sound i put up with it. You need to make sure you're enabling all the correct options, blacklisting the correct modules, and passing the right things through. I only got it working because my workstation seems to be in the ideal setup for it with the dedicated gpu taking over the hdmi port and one of the usb3 ports, and it being in it's own IOMMU group outside of the integrated graphics.

Kaddish
Feb 7, 2002
I have no practical knowledge or experience with it but why haven't I seen Nutanix in the VMware replacement conversation very much?

fresh_cheese
Jul 2, 2014

MY KPI IS HOW MANY VP NUTS I SUCK IN A FISCAL YEAR AND MY LAST THREE OFFICE CHAIRS COMMITTED SUICIDE

Kaddish posted:

I have no practical knowledge or experience with it but why haven't I seen Nutanix in the VMware replacement conversation very much?

I guess probably just general awareness, and then looking at it they have their own hypervisor but 5 minutes of googling doesnt turn up any description of what technology its based on. So thats a “hmmmmmmm….”

Then once you put it in front of the technical team they give you a list of reasons why it wont work that all come after unspoken reason 0: its not vmware so i dont want to do it.

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.

fresh_cheese posted:

I guess probably just general awareness, and then looking at it they have their own hypervisor but 5 minutes of googling doesnt turn up any description of what technology its based on. So thats a “hmmmmmmm….”

Then once you put it in front of the technical team they give you a list of reasons why it wont work that all come after unspoken reason 0: its not vmware so i dont want to do it.

My understanding is that Nutanix is just a much less trusted hypervisor at the core than VMWare, Hyper-V, or KVM/Qemu, which are all rock solid and proven. The various management interfaces for KVM might all suck, but KVM itself will never let you down. Nutanix was designed for doing hyper-converged VDI and just wasn't designed to be an all purpose virtualization solution. VMWare is not the only option, I bet we're going to see plenty of shops move to Hyper-V.

Look at the OP of the Homelab thread for a firsthand account into why Nutanix failed to ever get a foothold in the hobbyist market, which matters because people will try things at home before bringing them up at work:

H2SO4 posted:

In case anybody is starting out on the homelab journey and is thinking about going the Nutanix CE route, let me offer a couple words of widsom.

DON'T loving DO IT.

I originally went down the Nutanix CE route because I found a mislabled Supermicro x9 based 4-node 2U server on eBay and got it for mad cheap. It was all JBOD storage though, no RAID, so I figured it'd be a fun testbed for hyperconverged stuff like VSAN and Nutanix CE.

Nutanix CE is an absolute house of cards, you have zero access to any knowledge base or documentation unless you've also got a Nutanix account with active support through work/etc. The simplest of functions, such as replacing a failed disk drive or adding a new one, is completely undocumented and abstracted away from the interface leaving you to have to troll through god knows how many blogs and random forum posts with broken image links and missing formatting to attempt to figure out what esoteric CLI fuckery you need to do in order to make it happen.

Also, upgrades are compulsory. If they release a new version of the hypervisor/platform, the next time you login to your cluster you are forced to apply it right away. No access to your cluster/VMs/etc until the process completes, no option to defer. But that's not such a bad thing, their whole gimmick is one-click painless upgrades, right?

Wrong. Every single time I've attempted to upgrade my cluster it's resulted in needing some form of recovery. The best case scenario was a node gets hung at upgrade, evicts itself from the metadata ring, then after a manual reboot it comes back up and you can manually enable the metadata store again. This latest version caused a total cluster-wide data loss event, as when the first node updated it immediately started evicting every single disk with no warning and no loving reason. Something its own configuration should have prevented considering it knows how many drive failures/node failures/etc it can tolerate. If it had just evicted the node itself and halted the process it would have been fine, but it's not smart enough to do that.

There is at best a total lack of QA on any CE releases. For instance, the previous version's ISO installer just plain didn't work, and they were too lazy to fix it so they just removed the ISO. This led to people just using the USB image to deploy their clusters, but the USB image doesn't do any kind of repartitioning of the install drives which means that slowly but surely nodes would run out of space on the system drive making things get real fucky. If you figured out what was going on you could do some fdisk magic, write a new partition table, and extend the existing partitions but that's only if you whispered the right incantations into google and found your issue on page 32 of the results.

With the latest crash and burn I finally just said "gently caress it" and bought four 1U boxes with boring old RAID controllers, transferred the guts over, and swapped back to good old ESX and I wish I cut bait sooner. Just doing that effectively gave me back 128GB of memory that was previously wasted on the Nutanix controller VMs.

I know I'm salty, but I argue I have good reason to be considering all the headaches they caused in my lab, which by extension gave me a glimpse into how flimsy their house of cards really is. I'm sure their retail product is great. I'm sure their support is great. But based on this experience over four generations of CE, I'd actively block them from being considered for any projects at work.

Zorak of Michigan
Jun 10, 2006

Twerk from Home posted:

My understanding is that Nutanix is just a much less trusted hypervisor at the core than VMWare, Hyper-V, or KVM/Qemu, which are all rock solid and proven. The various management interfaces for KVM might all suck, but KVM itself will never let you down. Nutanix was designed for doing hyper-converged VDI and just wasn't designed to be an all purpose virtualization solution. VMWare is not the only option, I bet we're going to see plenty of shops move to Hyper-V.

Look at the OP of the Homelab thread for a firsthand account into why Nutanix failed to ever get a foothold in the hobbyist market, which matters because people will try things at home before bringing them up at work:

According to https://portal.nutanix.com/page/documents/solutions/details?targetId=TN-2038-AHV:nutanix-ahv-components.html , AHV is based on KVM/Qemu.

My problem with it as a new hypervisor is that we have a huge investment in non-Nutanix hardware and storage. You start asking them about how to use your existing blade servers and vvol storage with AHV, they look for diplomatic ways to say, "not our circus, not our monkeys."

Zorak of Michigan fucked around with this message at 17:39 on Jan 28, 2024

Potato Salad
Oct 23, 2014

nobody cares


I just don't see a big migration to Hyper-V without a Microsoft vCenter. I absolutely see Microsoft squandering the opportunity to become the virtualization platform of small/medium American businesses by continuing to confound what should be simple interfaces and tools with cloud integrations. Already, 2/3 of what's in Windows Admin Center is functionally nagware to pay for things that SMBs probably aren't ever going to pay for, and it confuses the everliving hell out of the hordes and hordes of general practice IT workers who burn the incense that keeps AA_april2013-gross_FINAL(1)_FINAL DONT REMOVE(1)(1).xls running.

Potato Salad
Oct 23, 2014

nobody cares


I'm going to Azurestack HCI but that's because we have the budget for genuine engineers. Half the guys who do IT at my local card shop are just happy to churn through five or six tickets a day and drive to a site to get a laptop deployed without hitting traffic. there's billions upon billions of dollars in the market that serves those dudes, and that's why I continue to insist that, after KVM matured, vmware's core product was actually vCenter.

fresh_cheese
Jul 2, 2014

MY KPI IS HOW MANY VP NUTS I SUCK IN A FISCAL YEAR AND MY LAST THREE OFFICE CHAIRS COMMITTED SUICIDE
OVirt is still there, has it started withering yet since redhat killed RHV?

Internet Explorer
Jun 1, 2005





Finally, XenServer's time to shine!

fresh_cheese
Jul 2, 2014

MY KPI IS HOW MANY VP NUTS I SUCK IN A FISCAL YEAR AND MY LAST THREE OFFICE CHAIRS COMMITTED SUICIDE

Internet Explorer posted:

Finally, XenServer's time to shine!

:frogout:

Thanks Ants
May 21, 2004

#essereFerrari


Kaddish posted:

I have no practical knowledge or experience with it but why haven't I seen Nutanix in the VMware replacement conversation very much?
I can see why people getting hosed by an acquisition of VMware by complete vultures are reluctant to include another single-vendor solution in their shortlist for a replacement

Thanks Ants fucked around with this message at 17:55 on Jan 28, 2024

Adbot
ADBOT LOVES YOU

Potato Salad
Oct 23, 2014

nobody cares



:getin:

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply