Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
SlowBloke
Aug 14, 2017

Subjunctive posted:

I’ve got a small Ubuntu home server running Home Assistant and a Unifi Controller under a creaky qemu/kvm setup, plus some other under-maintained services. I also have a Steam Deck which is shockingly compatible with the things I’ve been playing. I have virtualization questions about both arenas!

For the home server, I think I’d like to get rid of the mini tower and move to a NUCish form factor, and beef it up so I can play with some more modern homelab/clustering things. My light reading has led me to a Simply NUC Ruby r8 on which I will stick proxmox and then figure out how I want containers and VMs to interplay. And then stub my face on k3s, probably. Is this a sane path to pursue?

For gaming, the time for my Zen 4/Lovelace upgrade is coming and I’m seriously thinking of giving Linux gaming a shot for the first time since “Civ: Call to Power”. If it doesn’t work out for everything, I will probably want to do GPU+USB passthrough to Windows 11 or similar. Can I plausibly do that with a single NVIDIA GPU? I’ve heard tell, but I don’t know how reliable it is.

What’s the state of the art for doing things like clipboard in/out of VMs? I’ve only used the VMware stuff for that, and I’d rather not entangle myself with their stack if I can avoid it.

Thanks for any guidance you can provide!

Topic1 The processor is a getting a bit long in the tooth but should suffice for your use case.
Topic2 I would suggest going baremetal win11 plus WSL2 rather than doing weird stuff with PCIe passthrough. WSL2 has CUDA/OPENCL access now along with GUI apps so you will not miss anything.

Adbot
ADBOT LOVES YOU

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

SlowBloke posted:

Topic2 I would suggest going baremetal win11 plus WSL2 rather than doing weird stuff with PCIe passthrough. WSL2 has CUDA/OPENCL access now along with GUI apps so you will not miss anything.

Can I run stuff like k3s and systemd bits and so forth on Arch with WSL2, ideally in a way that doesn’t require reading a handful of “how to X on WSL2” gists every time I want to do something new? I’m hoping to keep the Linux environments as similar as reasonable between desktop, homelab, and Steam a deck because I’m a bear of little brain.

freeasinbeer
Mar 26, 2015

by Fluffdaddy
Any games with anticheat are likely to freak out about virtualization, so basically no recent shooters.

Which really sucks.

As for the NUC solution; I uhh have like 6 of them and then another 6-8 raspberry pis, and I put 99% of my apps on my little x86 router using docker compose. There are a number of low power intel x86 nuc like or smaller things you can get that have 4-6 2.5gbe or even one with 10g sfp+ ports.

I like K8s/k3s and use it all day professionally, but docker compose is just a bit easier for a single node “thing”.

in a well actually
Jan 26, 2011

dude, you gotta end it on the rhyme

Subjunctive posted:

I’ve got a small Ubuntu home server running Home Assistant and a Unifi Controller under a creaky qemu/kvm setup, plus some other under-maintained services. I also have a Steam Deck which is shockingly compatible with the things I’ve been playing. I have virtualization questions about both arenas!

For the home server, I think I’d like to get rid of the mini tower and move to a NUCish form factor, and beef it up so I can play with some more modern homelab/clustering things. My light reading has led me to a Simply NUC Ruby r8 on which I will stick proxmox and then figure out how I want containers and VMs to interplay. And then stub my face on k3s, probably. Is this a sane path to pursue?

For gaming, the time for my Zen 4/Lovelace upgrade is coming and I’m seriously thinking of giving Linux gaming a shot for the first time since “Civ: Call to Power”. If it doesn’t work out for everything, I will probably want to do GPU+USB passthrough to Windows 11 or similar. Can I plausibly do that with a single NVIDIA GPU? I’ve heard tell, but I don’t know how reliable it is.

What’s the state of the art for doing things like clipboard in/out of VMs? I’ve only used the VMware stuff for that, and I’d rather not entangle myself with their stack if I can avoid it.

Thanks for any guidance you can provide!

Don’t buy from SimplyNUC; I got on their mailing list a decade ago and they’ve been sending me spam constantly since, and none or their unsubscribe links work. They also change domains and senders around so it’s annoying to block.

gently caress those guys.

SlowBloke
Aug 14, 2017

Subjunctive posted:

Can I run stuff like k3s and systemd bits and so forth on Arch with WSL2, ideally in a way that doesn’t require reading a handful of “how to X on WSL2” gists every time I want to do something new? I’m hoping to keep the Linux environments as similar as reasonable between desktop, homelab, and Steam a deck because I’m a bear of little brain.

WSL2 on supported distros is cli until you start apps, also the Wayland part is done during installing. Arch require heavy tinker to run in that scenario, we use Ubuntu on our fleet and it's pretty much painless.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

in a well actually posted:

Don’t buy from SimplyNUC; I got on their mailing list a decade ago and they’ve been sending me spam constantly since, and none or their unsubscribe links work. They also change domains and senders around so it’s annoying to block.

gently caress those guys.

Oh, I was going to use your email address anyway.

Better players in that space?

(Thanks for the tip. If I end up buying from them after all then I’ll use a throwaway.)

freeasinbeer posted:

Any games with anticheat are likely to freak out about virtualization, so basically no recent shooters.

Which really sucks.

As for the NUC solution; I uhh have like 6 of them and then another 6-8 raspberry pis, and I put 99% of my apps on my little x86 router using docker compose. There are a number of low power intel x86 nuc like or smaller things you can get that have 4-6 2.5gbe or even one with 10g sfp+ ports.

I like K8s/k3s and use it all day professionally, but docker compose is just a bit easier for a single node “thing”.

I’m too slow for shooters anyway.

I want to play with k[83]s to better understand the poo poo people keep talking about at work, but I have generally enjoyed docker-compose in the past.

Do you have a recommendation for specific “low power intel x86 nuc like or smaller things”, by chance?

SlowBloke posted:

WSL2 on supported distros is cli until you start apps, also the Wayland part is done during installing. Arch require heavy tinker to run in that scenario, we use Ubuntu on our fleet and it's pretty much painless.

Hmm, that’s too bad. I really do want arch I think. (I have Ubuntu running on my current desktop under WSL2, mostly to avoid having to learn Powershell.)

SlowBloke
Aug 14, 2017

Subjunctive posted:

Do you have a recommendation for specific “low power intel x86 nuc like or smaller things”, by chance?

MSI, asrock and gigabyte have plenty of picks for ryzen 1l or less. Simplynuc seems to be using a white label asrock industrial 4X4 BOX-4800U in the sku you wanted to buy.

Subjunctive posted:

Hmm, that’s too bad. I really do want arch I think. (I have Ubuntu running on my current desktop under WSL2, mostly to avoid having to learn Powershell.)
I would say learning powershell to be a more useful usage of time than to keep running Arch but you do you :)

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

I’m sort of itching to try Proxmox vGPU stuff on my desktop, because 3950X+3090 should be able to handle more than one game at a time, but I might wait a few months until this machine isn’t my daily driver any more.

SlowBloke
Aug 14, 2017
New vSphere is out, it's all tanzu and DPU with a handful of QoL improvements. Nothing to justify jumping on it.

BlankSystemDaemon
Mar 13, 2009



SlowBloke posted:

New vSphere is out, it's all tanzu and DPU with a handful of QoL improvements. Nothing to justify jumping on it.
There's also something to be said for not supporting Broadcom.

Actuarial Fables
Jul 29, 2014

Taco Defender

Subjunctive posted:

I’m sort of itching to try Proxmox vGPU stuff on my desktop, because 3950X+3090 should be able to handle more than one game at a time, but I might wait a few months until this machine isn’t my daily driver any more.

Does that work with consumer cards now? I've got a 3700X and a 1080 in one of my hosts, could be fun to get that configured.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

Actuarial Fables posted:

Does that work with consumer cards now? I've got a 3700X and a 1080 in one of my hosts, could be fun to get that configured.

I’m told it works up to Turing, yeah.

Potato Salad
Oct 23, 2014

nobody cares


DPU -- placing security decisions right on the network and remote memory/storage card -- is excellent and an extension of the sorts of functionality previously only available to hyperscalers

Actuarial Fables
Jul 29, 2014

Taco Defender

Subjunctive posted:

I’m told it works up to Turing, yeah.

Thanks for bringing this up. I was able to get my GTX1080 set up as a vgpu-capable device and my gaming VM now has a nice """Quadro P5000""" installed.

Mr Shiny Pants
Nov 12, 2012

Actuarial Fables posted:

Thanks for bringing this up. I was able to get my GTX1080 set up as a vgpu-capable device and my gaming VM now has a nice """Quadro P5000""" installed.

What did you use? Is this different from the patched vGPU stuff?

Actuarial Fables
Jul 29, 2014

Taco Defender

Mr Shiny Pants posted:

What did you use? Is this different from the patched vGPU stuff?

https://gitlab.com/polloloco/vgpu-proxmox/-/tree/master
https://www.michaelstinkerings.org/using-vgpu-unlock-with-proxmox-7/

Patching the driver to allow consumer cards to be a vGPU-capable device is the host portion. Spoofing the PCI device to present a workstation card to the VM instead of a GRID vGPU is for the guest VM, otherwise performance would degrade as the vGPU gets more and more angry that it can't reach a licensing server.

The guides have you set the Vendor and Device ID's in the profile_override.toml file, but I had to set them in the VM config file. If using the GUI, there's fields when adding a PCI device to set those variables.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

A friend of mine (he worked on Space/TimeWarp and general GPU insanity at Oculus when I was there) brought his company out of stealth today: Juice, which does IP-based transparent remoting of GPU resources at high speed.

https://www.juicelabs.co/

Binaries available for Windows, Linux (Ubuntu), and Mac; works inside VMs; no client program modifications required. I’ve seen some demo videos of it before and it is pretty friggin’ nuts. Could really change the game for GPU-passthrough sorts of applications. I’m travelling right now so I haven’t installed it yet myself.

Pile Of Garbage
May 28, 2007



Subjunctive posted:

A friend of mine (he worked on Space/TimeWarp and general GPU insanity at Oculus when I was there) brought his company out of stealth today: Juice, which does IP-based transparent remoting of GPU resources at high speed.

https://www.juicelabs.co/

Binaries available for Windows, Linux (Ubuntu), and Mac; works inside VMs; no client program modifications required. I’ve seen some demo videos of it before and it is pretty friggin’ nuts. Could really change the game for GPU-passthrough sorts of applications. I’m travelling right now so I haven’t installed it yet myself.

Not really sure how this would work for any non-async workloads. Might be good for transcode offload from one's Plex server?

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

Pile Of Garbage posted:

Not really sure how this would work for any non-async workloads. Might be good for transcode offload from one's Plex server?

It apparently works fine for gaming, with ~150Mbit network! I’m travelling right now but I’m going to see if I can use it to virtualize my 4090 to serve a few Minecraft clients on underpowered machines.

The engineers are actively answering questions in their discord, fwiw.

Pile Of Garbage
May 28, 2007



Subjunctive posted:

It apparently works fine for gaming, with ~150Mbit network! I’m travelling right now but I’m going to see if I can use it to virtualize my 4090 to serve a few Minecraft clients on underpowered machines.

The engineers are actively answering questions in their discord, fwiw.

Just noticed you're selling this in other threads, I don't think anything you say re the tech can be taken as neutral. Further still this sounds cooked I'd advise caution.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

Pile Of Garbage posted:

Just noticed you're selling this in other threads, I don't think anything you say re the tech can be taken as neutral. Further still this sounds cooked I'd advise caution.

I am not neutral, I think the tech is very cool from what I’ve seen of it in demos. I don’t know what you mean by “cooked”, I admit. I’ll have personal experience with it when I get home this weekend.

By “selling this in other threads” do you mean “also posted about it in one other thread, the one about GPUs that often discusses cloud gaming”?

E: To be more explicit, I stand to gain exactly nothing if this is successful commercially, and lose exactly nothing if it fails. I just think it’s exciting tech, and usefully to me personally.

Subjunctive fucked around with this message at 05:09 on Nov 12, 2022

Mr. Crow
May 22, 2008

Snap City mayor for life
It sounds neat I guess and if its successful it could be a game changer but there was no actual content on their website with like benchmarks or anything just kind of vague implementation details in the whitepaper so not too interested until there is some real world data. Lol if you think I'm going to discord for it.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

Yeah, that’s fair. There’s going to be some stuff at SuperComputing (the conference) soon I think but that’ll probably be non-gaming stuff.

As of a couple of hours ago there was a public server instance up and running but I’m on a train and don’t want to gently caress my data plan for the month by trying it out. I’ll give it a try locally if nothing else when I get a chance to kick the kids off the computers this weekend.



Not sure if it’s still up. I asked about benchmarks but I think they’re travelling for the conference so we’ll see.

Methanar
Sep 26, 2013

by the sex ghost
What are the odds Nvidia disagrees with the business model and deliberately breaks it?

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

Probably depends on the devices being used in terms of data centre deployment (IIRC the consumer cards aren’t licensed for use in DCs or something) but they already support vGPU for the industrial cards, right?

I’m not sure what they could do to break it, I admit, without also breaking NVIDIA and Valve’s own remoting, but I haven’t thought about it very much.

Thanks Ants
May 21, 2004

#essereFerrari


They break it in the same way Microsoft broke virtualising the Windows client OS in VDI deployments - never sell a suitable license to permit deploying in that way, and launch your own service in Azure.

Ihmemies
Oct 6, 2012

I have proxmox and a couple of servers running on it. I can manually make VM backup images of the servers. Where I should store them? Can I automate this stuff somehow? How to actually backup the proxmox hypervisor so I don’t have to setup everything again if something breaks? Nas backups are hard :v:

Edit: also I need a new ip address for some docker containers.

I currently have proxmox > debian server > docker + portainer. I manage containers with portainer. Should I make a new virtual network card in proxmox for the debian server? Or something else? I want containers 1-6 use .16 ip and 7-12 .20 ip address for example.

Edit: apparently make a 2nd virtual nic for the debian server in proxmox. Assign static ip from dhcp server. This works so far, now the deb server has 2 nic's and 2 ip's.
Then do ??? and remote ssh with vscode to the docker server and ??? This will take time with my skills.

Edit2: seems it would be best to roll a new VM with docker and install remote dev extensions to vscode. Then I can ssh to the dev vm and vscode will remotely issue commands etc. And If I gently caress up I can nuke the whole thing.

Ihmemies fucked around with this message at 23:12 on Nov 19, 2022

Moey
Oct 22, 2010

I LIKE TO MOVE IT
I know this has been a long time in the coming, but VMware is finally kicking Teradici (now owned by HP) to the curb.

I saw the writing on the walls as I watched Blast come together. Teradici must have as well, since those fucks moved their firmware and management from free, to monthly subscription. Let em rot.

https://blogs.vmware.com/euc/2023/03/announcing-end-of-support-for-pcoip-in-vmware-horizon.html

DevNull
Apr 4, 2007

And sometimes is seen a strange spot in the sky
A human being that was given to fly

Moey posted:

I know this has been a long time in the coming, but VMware is finally kicking Teradici (now owned by HP) to the curb.

I saw the writing on the walls as I watched Blast come together. Teradici must have as well, since those fucks moved their firmware and management from free, to monthly subscription. Let em rot.

https://blogs.vmware.com/euc/2023/03/announcing-end-of-support-for-pcoip-in-vmware-horizon.html

It was decided a long time ago when Teradici decided to jack up their licensing prices for VMware. I worked on Blast 2.0 back before the decision was made. We tried to convince View/Horizon on the idea that we could do it much cheaper and much better in house. They all raved about how great PCoIP was. Even though we worked with any guest and performed better. A year or so later, Teradici supposedly said they wanted a ton more money. More money as in 10x what they were getting. Leadership had a change of heart with Blast after that.

ihafarm
Aug 12, 2004

DevNull posted:

It was decided a long time ago when Teradici decided to jack up their licensing prices for VMware. I worked on Blast 2.0 back before the decision was made. We tried to convince View/Horizon on the idea that we could do it much cheaper and much better in house. They all raved about how great PCoIP was. Even though we worked with any guest and performed better. A year or so later, Teradici supposedly said they wanted a ton more money. More money as in 10x what they were getting. Leadership had a change of heart with Blast after that.

Who developed Blast/Project Octopus(?) originally? I feel like there was a blogger oh so many moons ago that talked about it vis a vis a new on-demand lab system for vmworld(or whatever the gently caress it’s called). Nick somebody. But my memory is only as good as the lifetime of google reader.

I do certainly remember being so impressed by local USB device/multi-monitor support in PCoIP thin clients.

DevNull
Apr 4, 2007

And sometimes is seen a strange spot in the sky
A human being that was given to fly

ihafarm posted:

Who developed Blast/Project Octopus(?) originally? I feel like there was a blogger oh so many moons ago that talked about it vis a vis a new on-demand lab system for vmworld(or whatever the gently caress it’s called). Nick somebody. But my memory is only as good as the lifetime of google reader.

I do certainly remember being so impressed by local USB device/multi-monitor support in PCoIP thin clients.

It has been so long I had to look it up. I think his name was Jonathan. He created AppBlast, which ran on the machine and scraped contents of the app window and remoted it to your browser. The guy I worked with, Keith, basically took the compression protocols and shoved it into the remoteMKS, which is what you use to connect to your ESX host. It obviously has the contents of the whole screen. The remoteMKS was also what was handling your mouse, keyboard, and USB for View. We worked on adding more encodings to the remoteMKS screen remoting, which is just VNC by the way. If you noticed your remote screen connections get a lot faster on ESX during this time, it was because we were trying to unify it all on that improved protocol. PCoIP ran in the guest, because their app only worked on Windows. View was built to deal with that. When things moved to Blast, they kept that architecture. I'm not sure if they kept it that way. We tried to convince them to switch to using the host frame buffer because it would work with any guest, but they always has excuses not to. PCoIP was just the streaming of screen data, and most everything else was the MKS. It was foolish for VMware to hype it up so much.

PBCrunch
Jun 17, 2002

Lawrence Phillips Always #1 to Me
I've tried Googling for this but I haven't been able to put together the right combination of keywords to get to an answer.

I have a physical box with two KVM VMs running on it. One VM runs Zoneminder on Ubuntu 22.04 and the other one runs Home Assistant OS. When the host reboots the two VMs start automatically, which is great, but I want the Zoneminder VM to start and be "all the way running" before the Home Assistant VM kicks off. How would I go about doing this?

Delaying the Home Assistant VM boot by 30 seconds maybe? Seems kludgey. Keeping the VM from booting until the Zoneminder instance is responding its assigned port?

Crime on a Dime
Nov 28, 2006
What's kludgey?

PBCrunch
Jun 17, 2002

Lawrence Phillips Always #1 to Me

Crime on a Dime posted:

What's kludgey?
Kludgey in that the Zoneminder instance might not be ready after 30 seconds. Or might regularly be ready in much less time. If I'm waiting for a service in the VM to be ready to go, isn't it better to have the "dependent" VM wait for that service to be available?

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



PBCrunch posted:

Kludgey in that the Zoneminder instance might not be ready after 30 seconds. Or might regularly be ready in much less time. If I'm waiting for a service in the VM to be ready to go, isn't it better to have the "dependent" VM wait for that service to be available?

Proxmox has the ability to set the startup order for the guests. I presume there is some underlying terminal setting that can be toggled and this is not gui specific so perhaps you can look into how it does that?



I spent the time I'm willing to put into this and this was all I was able to find.

https://pve.proxmox.com/pve-docs/chapter-qm.html#qm_startup_and_shutdown

Saukkis
May 16, 2003

Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.

PBCrunch posted:

Keeping the VM from booting until the Zoneminder instance is responding its assigned port?

That's is probably the most exact method, use a ExecStartPre script that waits until the Zoneminder is responsive. Best to also to increase the timeout for the Home Assistant service.

Shaocaholica
Oct 29, 2002

Fig. 5E
How does VM network speed work for VMs running on the same host? Are connections capped to virtual standards like 1G, 10G? Can you have as fast as possible between VMs running on the same host? How fast can that be?

SlowBloke
Aug 14, 2017

Shaocaholica posted:

How does VM network speed work for VMs running on the same host? Are connections capped to virtual standards like 1G, 10G? Can you have as fast as possible between VMs running on the same host? How fast can that be?

VMXNET3 and PVRDMA are the fastest ones you can use.

SamDabbers
May 26, 2003



Semi-related: is there any benefit for east-west traffic between VMs on the same box to use SR-IOV and let the NIC switch the packets instead of the CPU? Seems like there'd be less CPU load at the expense of a round trip over PCIe, and would also be dependent on the internal switching capacity of the particular NIC.

Has anybody tried this?

Adbot
ADBOT LOVES YOU

SlowBloke
Aug 14, 2017

SamDabbers posted:

Semi-related: is there any benefit for east-west traffic between VMs on the same box to use SR-IOV and let the NIC switch the packets instead of the CPU? Seems like there'd be less CPU load at the expense of a round trip over PCIe, and would also be dependent on the internal switching capacity of the particular NIC.

Has anybody tried this?


SR-IOV has still a cpu overhead, higher than the integrated vswitches pass thru channels. There are advantages for traffic outside the box, same box it's not convenient to use it.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply