Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer

Wicaeed posted:

Guess I have to write my own, or something *sigh*
This should be like 10 lines in powercli.

Adbot
ADBOT LOVES YOU

Goonerousity
Sep 25, 2017

aww yeah
Any tips on running virt-manager and bhyve with gpu passthrough on freebsd 12? I plan on playing wow in a vm


I have an Asus prime z270 and an i7

evol262
Nov 30, 2010
#!/usr/bin/perl
Just use WINE. Wow has been fine with that for years.

If you want passthrough, you'll need to edit the libvirt XML. I'm not sure how many of the options bhyve actually supports, so hopefully you have an AMD card.

Otherwise, with a UEFI firmware on the card, it's plain PCI passthrough, so follow those docs. If it's an old VGA (legacy BIOS) card, sorry, bhyve won't do it

Goonerousity
Sep 25, 2017

aww yeah

evol262 posted:

Just use WINE. Wow has been fine with that for years.

If you want passthrough, you'll need to edit the libvirt XML. I'm not sure how many of the options bhyve actually supports, so hopefully you have an AMD card.

Otherwise, with a UEFI firmware on the card, it's plain PCI passthrough, so follow those docs. If it's an old VGA (legacy BIOS) card, sorry, bhyve won't do it

Wine breaks on bfa updates, I know this because I’m a virgin and I smell bad

Don’t want to sound mean but do you have experience getting gpu passthrough working with windows 10 guests on hopefully a freebsd 12-current install? I’d really like a 1920x1080 maxes out wow window on my 4K desktop

Does anyone have experience with xen and gpu passthrough?

And help with getting virt-manager/bhyve working with win10 and gpu passthrough would be awesome

evol262
Nov 30, 2010
#!/usr/bin/perl

Goonerousity posted:

Wine breaks on bfa updates, I know this because I’m a virgin and I smell bad

Don’t want to sound mean but do you have experience getting gpu passthrough working with windows 10 guests on hopefully a freebsd 12-current install? I’d really like a 1920x1080 maxes out wow window on my 4K desktop

Does anyone have experience with xen and gpu passthrough?

And help with getting virt-manager/bhyve working with win10 and gpu passthrough would be awesome

I'm a KVM/RHV engineering lead, and I've done it repeatedly on Linux. libvirt is libvirt, and virt-manager is virt-manager. virt-manager doesn't expose the knobs you need to twiddle for nVidia's drivers to work on passthrough (mostly hiding that it's virtualized). You need to edit the libvirt XML. Bhyve has good support for most of the libvirt options.

I checked the Googles before your post to verify that UEFI passthrough works on bhyve. "Bhyve PCI passthrough" will lead you to the relevant docs. You'll still need to ensure that the card is on its own iommu group, and that your motherboard supports vt-d. Once that's done, you can basically jump on any of the Linux passthrough tutorials (skipping the parts about dracut and vfio, because the PCI passthrough docs for FBSD show you the right way to do it), because libvirt is libvirt.

You're not gonna get WoW in a 1080 window on your desktop with passthrough. You need a second card, and a second monitor (or toggle input). Or use some Steam streaming analogue for WoW, if it exists.

Frankly, if this is what you want, you'd want virtio-gpu+virgl, which probably isn't supported on bhyve, and is still pretty experimental on Linux

Potato Salad
Oct 23, 2014

nobody cares


evol262 posted:

I'm a KVM/RHV engineering lead, and I've done it repeatedly on Linux. libvirt is libvirt, and virt-manager is virt-manager. virt-manager doesn't expose the knobs you need to twiddle for nVidia's drivers to work on passthrough (mostly hiding that it's virtualized). You need to edit the libvirt XML. Bhyve has good support for most of the libvirt options.

I checked the Googles before your post to verify that UEFI passthrough works on bhyve. "Bhyve PCI passthrough" will lead you to the relevant docs. You'll still need to ensure that the card is on its own iommu group, and that your motherboard supports vt-d. Once that's done, you can basically jump on any of the Linux passthrough tutorials (skipping the parts about dracut and vfio, because the PCI passthrough docs for FBSD show you the right way to do it), because libvirt is libvirt.

You're not gonna get WoW in a 1080 window on your desktop with passthrough. You need a second card, and a second monitor (or toggle input). Or use some Steam streaming analogue for WoW, if it exists.

Frankly, if this is what you want, you'd want virtio-gpu+virgl, which probably isn't supported on bhyve, and is still pretty experimental on Linux

I'm just gonna quote this for experimentation during some upcoming travel.

Goonerousity
Sep 25, 2017

aww yeah

evol262 posted:

I'm a KVM/RHV engineering lead, and I've done it repeatedly on Linux. libvirt is libvirt, and virt-manager is virt-manager. virt-manager doesn't expose the knobs you need to twiddle for nVidia's drivers to work on passthrough (mostly hiding that it's virtualized). You need to edit the libvirt XML. Bhyve has good support for most of the libvirt options.

I checked the Googles before your post to verify that UEFI passthrough works on bhyve. "Bhyve PCI passthrough" will lead you to the relevant docs. You'll still need to ensure that the card is on its own iommu group, and that your motherboard supports vt-d. Once that's done, you can basically jump on any of the Linux passthrough tutorials (skipping the parts about dracut and vfio, because the PCI passthrough docs for FBSD show you the right way to do it), because libvirt is libvirt.

You're not gonna get WoW in a 1080 window on your desktop with passthrough. You need a second card, and a second monitor (or toggle input). Or use some Steam streaming analogue for WoW, if it exists.

Frankly, if this is what you want, you'd want virtio-gpu+virgl, which probably isn't supported on bhyve, and is still pretty experimental on Linux

This is exactly what I needed, thank you. I’ll stick to windows 10 and full screen virtual machines when I need to write c

Also is there a free version of VMware’s virtualization software? I remember reading through google theres like a VMware workstation but got devestatingly lost on VMware’s website

Volguus
Mar 3, 2009

Goonerousity posted:

This is exactly what I needed, thank you. I’ll stick to windows 10 and full screen virtual machines when I need to write c

Also is there a free version of VMware’s virtualization software? I remember reading through google theres like a VMware workstation but got devestatingly lost on VMware’s website

I used to play wow on linux (most recently last month with the new expansion, now i stopped again) and it's perfectly fine with wine. By perfectly fine I mean 30FPS in Orgrimmar on Mal'Ganis. FreeBSD performance shouldn't be any different from the linux one.
Now I'm playing Diablo, and it works as it should. Nothing to complain about here. What im trying to say is that you may not need the passthrough (even though it does provide the best FPS) for normal gameplay. If you're one of those streaming on twitch ... then stick with windows, yes.

Docjowles
Apr 9, 2009

Goonerousity posted:

This is exactly what I needed, thank you. I’ll stick to windows 10 and full screen virtual machines when I need to write c

Also is there a free version of VMware’s virtualization software? I remember reading through google theres like a VMware workstation but got devestatingly lost on VMware’s website

I haven’t looked at desktop virt in a long time. But there used to be VMware Player as an extremely bare bones version. Searching around now, it still exists but has been rebranded under Workstation. There appears to be a free version still if you try hard enough to avoid all the Pro upsells on their site.

If you are talking about the bare hypervisor, a stand-alone ESXi instance is still free as far as I know.

SlowBloke
Aug 14, 2017

Goonerousity posted:

This is exactly what I needed, thank you. I’ll stick to windows 10 and full screen virtual machines when I need to write c

Also is there a free version of VMware’s virtualization software? I remember reading through google theres like a VMware workstation but got devestatingly lost on VMware’s website

https://www.vmware.com/products/workstation-player/workstation-player-evaluation.html

This is the free type-2 software(run it on a Windows or linux computer)

https://www.vmware.com/products/vsphere-hypervisor.html

This is the free type-1 software(run it as the os)

Potato Salad
Oct 23, 2014

nobody cares


Vmug Advantage is $200/year and you get to use basically all of VMware's everything.

KVM is freeeeeeeeeeeeeeeeeee

evol262
Nov 30, 2010
#!/usr/bin/perl
Not that I don't love KVM, but there's an awful lot of tooling missing, even in oVirt/RHV.

Off the top of my head:
Imports of VMs from VMware which contain snapshots
3d accelerated guests (minus virgl, which is experimental)
3rd party backup ecosystem
Some NUMA affinity bits
Sane UEFI support just stabilized this year
The hyperconverged ecosystem is very new, and there's basically zero commercial support for running it with Ceph
Lots of IPv6 stuff
...

If all you want is a couple of VMs, sure. And most of these can be worked around. But it's not exactly ready as a drop-in VMware replacement out of the box

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
Does anyone know of documentation that lists the technical differences between VT-X and SVM, which does what differently and possibly better than the other?

evol262
Nov 30, 2010
#!/usr/bin/perl
You're really looking at processor generations here. SVM is simpler to implement technically (which is why, generally, support for SVM lands first in new hypervisors, and nested virt is usually AMD first)

That said, VMexit, VMenter, etc, are basically the same. And even though the instruction set is identical, silicon/microcode-level updates can make this dramatically faster. The overhead on VT-x on Sky Lake v Sandy Bridge on an average workload is about 40% lower.

It's the absolute last thing you should be thinking about unless triply-nested virt speed is important to you (use SVM). Otherwise, compare workloads. Density, instructions per clock, memory available, etc

in a well actually
Jan 26, 2011

dude, you gotta end it on the rhyme

Wicaeed posted:

Why does VMware make it so difficult to unmount a datastore & detach a LUN across multiple hosts?

Unmounting a datastore they've got down as bulk unmounting is fairly easy, but detaching the LUN is still a tedious task of going to each ESX host, finding the LUN and manually detaching it.

I've only found 1 or 2 scripts that can do the same thing, but they all jump right in and try to do things like unmount all datastores on a host, or they only do it for clusters of ESX hosts.

Guess I have to write my own, or something *sigh*

It is a feature in whatever version of esxi you’re not licensed for.

BallerBallerDillz
Jun 11, 2009

Cock, Rules, Everything, Around, Me
Scratchmo

evol262 posted:

Not that I don't love KVM, but there's an awful lot of tooling missing, even in oVirt/RHV.

Off the top of my head:
Imports of VMs from VMware which contain snapshots
3d accelerated guests (minus virgl, which is experimental)
3rd party backup ecosystem
Some NUMA affinity bits
Sane UEFI support just stabilized this year
The hyperconverged ecosystem is very new, and there's basically zero commercial support for running it with Ceph
Lots of IPv6 stuff
...

If all you want is a couple of VMs, sure. And most of these can be worked around. But it's not exactly ready as a drop-in VMware replacement out of the box

Just curious - does VMWare support importing hyper-v or KVM VMs with snapshots? A cursory Google makes me think it doesn't. Seems like that's a limitation of any virilization platform.

I always wanted a chance to play with oVirt/RHEV in a real environment. I messed around with it in my home lab for a while but it wasn't really the same. These days all my work is in the cloud so I don't even get to work on VMWare anymore. It's probably better for my professional development but I do miss tinkering with VMs. Now the only ones I set up are for local testing with Vagrant - another place where I like using libvirt but find that support for it lags behind Virtual box.

E: see the typo but I like it and I'm leaving it. :colbert:

evol262
Nov 30, 2010
#!/usr/bin/perl
No idea if VMware supports it or not, but we do (virt-v2v is good at this). However, IMO, this is worse than not having support at all. We'll actually import VMs with snapshots, but they're not coalesced. So a VM with the first snapshot taken in 2014 will be "successfully" imported with data from 4 years ago.

I mentioned this off the top of my head since a lot of VMware shops (especially public sector and healthcare) are moving off VMware to RHV recently. The move to :yaycloud: and k8s means that traditional virt is becoming a core business entity, and shops would rather save 30%/yr than have functionality they've never even used.

oVirt, for a lab, is drop-in replacement for VMware (assuming you have shared storage somewhere), at least

As always, there's no such thing as the cloud, just somebody else's computer. A surprising amount of public cloud hosting (especially k8s hosting) is backed by RHV, and there are 10+ major deployments of oVirt for public infra (mostly European universities, but some cloud -- details crop up on the mailing lists every so often)

Docjowles
Apr 9, 2009

BallerBallerDillz posted:

virilization platform.

E: see the typo but I like it and I'm leaving it. :colbert:

ok, this was an extremely good avatar/post combo :golfclap:

Sad Panda
Sep 22, 2004

I'm a Sad Panda.
Is there a particular idiots guide to ESXi that people recommend? Maybe something to tell me which of the 1000 options are useful to toggle to something other than the default?

For some background, I own a Mac and got a new computer with ESXI that has three Win 10 VMs on it. The immediate questions...

1 Is there a better way of editing the Datastore? I made my first VM and to create the 2nd + 3rd used what looks like Baby's First File Manager (datastore -> datastore browser) which doesn't even include options to rename. Am I supposed to just be using the CLI?

2. Secondly, being on a Mac it seems I'm meant to use VMWare Remote Console is there a way to make it take in all the commands? I ask because at the moment if I press the keyboard shortcut to close a tab, it often closes VMWare Remote Console, and that's going to be very frustrating.

3. How do I make the web client default to opening in VMWare Remote Console? If I double click it likes to open them in the browser window itself which I don't want.

SlowBloke
Aug 14, 2017

Sad Panda posted:

Is there a particular idiots guide to ESXi that people recommend? Maybe something to tell me which of the 1000 options are useful to toggle to something other than the default?

For some background, I own a Mac and got a new computer with ESXI that has three Win 10 VMs on it. The immediate questions...

1 Is there a better way of editing the Datastore? I made my first VM and to create the 2nd + 3rd used what looks like Baby's First File Manager (datastore -> datastore browser) which doesn't even include options to rename. Am I supposed to just be using the CLI?

2. Secondly, being on a Mac it seems I'm meant to use VMWare Remote Console is there a way to make it take in all the commands? I ask because at the moment if I press the keyboard shortcut to close a tab, it often closes VMWare Remote Console, and that's going to be very frustrating.

3. How do I make the web client default to opening in VMWare Remote Console? If I double click it likes to open them in the browser window itself which I don't want.

1. You are not supposed to handle the vm using the datastore browser but rather the host pane, datastore browser is to remove data or import isos in your case

2. Keyboard macros on the remote are Win centric sadly, you just need to use the remote console to get into the vm and enable remote desktop or vnc. Using remote console for day to day interactive usage is far from optimal.

3. On the vSphere Web Client virtual machine Summary page, click the gear icon in the lower right corner of the console thumbnail. -> Select Change Default Console. -> Select VMware Remote Console and click OK.

Sad Panda
Sep 22, 2004

I'm a Sad Panda.

SlowBloke posted:

1. You are not supposed to handle the vm using the datastore browser but rather the host pane, datastore browser is to remove data or import isos in your case

2. Keyboard macros on the remote are Win centric sadly, you just need to use the remote console to get into the vm and enable remote desktop or vnc. Using remote console for day to day interactive usage is far from optimal.

3. On the vSphere Web Client virtual machine Summary page, click the gear icon in the lower right corner of the console thumbnail. -> Select Change Default Console. -> Select VMware Remote Console and click OK.

OK, I guess I should have cloned them in a different way. The friend who introduced me to ESXi said that the easiest way to do it is simply to load up datastore browser copy + paste the folder each time you want a clone. I did that, change the VM name in the settings and then ensured they had a different MAC.

What advantage does Remote Desktop/VNC have over the VMWare Remote Console? I'm so new to this stuff that it's marginally overwhelming. My only prior remoting experience is that I've got RealVNC installed on my Pi so I can access that from the network or via the Cloud.

SlowBloke
Aug 14, 2017

Sad Panda posted:

OK, I guess I should have cloned them in a different way. The friend who introduced me to ESXi said that the easiest way to do it is simply to load up datastore browser copy + paste the folder each time you want a clone. I did that, change the VM name in the settings and then ensured they had a different MAC.

What advantage does Remote Desktop/VNC have over the VMWare Remote Console? I'm so new to this stuff that it's marginally overwhelming. My only prior remoting experience is that I've got RealVNC installed on my Pi so I can access that from the network or via the Cloud.

Doing a "ghetto template" like you did was a sure fire way to get the same sid on Windows machines(very bad on old windows, no longer a thing). Windows RDP don't have the same issues VMware remote console has with keyboards for one, you should treat VMRC as a maintainance tool instead of a everyday interface. If you want to learn the ins and outs of vSphere I'd suggest you source a copy of Mastering VMware vSphere 6 by Nick Marshall with supervision from Scott Lowe(Mastering VMware vSphere 6.5 is by another group, I have no idea if it's good or not) which explains pretty much the basics of VMware vSphere and ESXi.

SlowBloke fucked around with this message at 21:46 on Sep 24, 2018

BangersInMyKnickers
Nov 3, 2004

I have a thing for courageous dongles

Enabling hot-add (which disabled vNUMA exposure) only matters for VMs that will span beyond a node right? Our infrastructure people are turning it off for 2 vCPU guests on hardware with 12cores on a socket (2 nodes with 6c each for cluster on die topology too I guess but that performance penalty is small) and I'm pretty sure it's a pointless and lovely thing to do on everything but the largest VMs we run.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

BangersInMyKnickers posted:

Enabling hot-add (which disabled vNUMA exposure) only matters for VMs that will span beyond a node right? Our infrastructure people are turning it off for 2 vCPU guests on hardware with 12cores on a socket (2 nodes with 6c each for cluster on die topology too I guess but that performance penalty is small) and I'm pretty sure it's a pointless and lovely thing to do on everything but the largest VMs we run.
Probably, yeah. I thought I remembered something about CPU hot-add causing a slight increase in baseline CPU utilization on Windows platforms a few years ago, but I don't know if this is still the case. I might have been making it up all along.

vNUMA isn't enabled on a VM until it has 9 or more vCPUs assigned to it at the time the virtual machine is powered on (this number is higher if you have more than 8 cores per socket; for 12-core nodes, you would need 13 vCPUs). So for 2 vCPU VMs, it literally makes no difference—there's no vNUMA anyway.

Vulture Culture fucked around with this message at 04:37 on Oct 8, 2018

BangersInMyKnickers
Nov 3, 2004

I have a thing for courageous dongles

Even better. Thanks.

evil_bunnY
Apr 2, 2003

http://frankdenneman.nl/2017/04/14/impact-cpu-hot-add-numa-scheduling/

This is a deece write-up on hot-add/vNUMA

evil_bunnY
Apr 2, 2003

are there any cards besides x710's that are not well supported despite claims to the contrary?

TheFace
Oct 4, 2004

Fuck anyone that doesn't wanna be this beautiful
Anyone jump through the hoops to get vSphere clear of CVE-2018-3646? https://kb.vmware.com/s/article/55806

Being is basically turns off Hyperthreading I'm really concerned with the potential performance impact. Wondering what any of you might have seen.

DevNull
Apr 4, 2007

And sometimes is seen a strange spot in the sky
A human being that was given to fly

Use the tool mentioned at the bottom of that kb. https://kb.vmware.com/s/article/56931

If your CPUs are a high usage, it might cause problems. If they are at low usage, you should be ok.

BangersInMyKnickers
Nov 3, 2004

I have a thing for courageous dongles

Factor in worst-case scenarios for your scaling. Either host failures in the cluster (especially important for small clusters) or HA failover to backup sites if you're active-active. HT helps you squeeze more guest cycles in to the host CPU, so if you're not under high CPU load right now its probably isn't doing much for you because the resource scheduler is going to throw the guest CPU load on to a core that has more availability.

If you are under moderate/high load in the cluster right now (I'd say consistently running 70%+ during peak), it will be important to set up resource pools with CPU share assignments to help de-prioritize your non-critical VMs during load contention.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.
nm

figured it out

Agrikk fucked around with this message at 04:16 on Oct 16, 2018

Mr. Crow
May 22, 2008

Snap City mayor for life
Is "Linux gaming" at near native really as simple as running a windows VM and doing GPU passthrough? My buddy says that's all he's doing and gets like 97% performance of native, I have a hard time believing that doesn't run into weird driver issues or something?

anthonypants
May 6, 2007

by Nyc_Tattoo
Dinosaur Gum

Mr. Crow posted:

Is "Linux gaming" at near native really as simple as running a windows VM and doing GPU passthrough? My buddy says that's all he's doing and gets like 97% performance of native, I have a hard time believing that doesn't run into weird driver issues or something?
Isn't this still something that heavily depends on what specific game you're trying to run?

Mr. Crow
May 22, 2008

Snap City mayor for life
I dunno that's my question and why I'm skeptical.

Less Fat Luke
May 23, 2003

Exciting Lemon

Mr. Crow posted:

Is "Linux gaming" at near native really as simple as running a windows VM and doing GPU passthrough? My buddy says that's all he's doing and gets like 97% performance of native, I have a hard time believing that doesn't run into weird driver issues or something?

It is really fantastic when you have the right hardware that supports it (which isn't rare); I ended up having to stop using VFIO cause it trips up a lot of anti-cheat systems.

Mr Shiny Pants
Nov 12, 2012

Mr. Crow posted:

Is "Linux gaming" at near native really as simple as running a windows VM and doing GPU passthrough? My buddy says that's all he's doing and gets like 97% performance of native, I have a hard time believing that doesn't run into weird driver issues or something?

Works a treat, got a 1080GTX passed though on my ThreadRipper and combined with Looking Glass and a SteamLink it is pretty awesome.

Just be sure to put KVM = hidden when running an NVidia card, they don't like you passing through consumer cards.

Mr. Crow
May 22, 2008

Snap City mayor for life
I was looking at Looking Glass too, how much do you notice the input lag? Does it make first person shooters difficult or is it not noticeable?

Mr Shiny Pants
Nov 12, 2012

Mr. Crow posted:

I was looking at Looking Glass too, how much do you notice the input lag? Does it make first person shooters difficult or is it not noticeable?

You should hookup a second mouse for sure, because you have some weird geometry issues otherwise.
A second keyboard is also handy, I just passed through a complete USB controller from my motherboard to minimize latency and have a second KB/M on it.

I don't know, I haven't tested FPS with a KB/M. Playing with a controller on the couch is fine though.

Eletriarnation
Apr 6, 2005

People don't appreciate the substance of things...
objects in space.


Oven Wrangler

Mr. Crow posted:

Is "Linux gaming" at near native really as simple as running a windows VM and doing GPU passthrough? My buddy says that's all he's doing and gets like 97% performance of native, I have a hard time believing that doesn't run into weird driver issues or something?

You don't get driver issues because Linux does not interact with the card being used. Windows sees it just as if it were a physical machine with a physical GPU.

I had the best success setting it up on Proxmox, but with the RX 460 I used as a test card I did have a strange issue where if I shut down the VM after starting it then trying to start it again before rebooting the host caused a kernel panic. It booted back up in under a minute and performance inside the VM seemed fine though.

Adbot
ADBOT LOVES YOU

Sad Panda
Sep 22, 2004

I'm a Sad Panda.
edit - Rubber Duck Debugging.

Typing out got me to a solution. I now have my graphics card passing through.

Sad Panda fucked around with this message at 18:40 on Oct 17, 2018

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply