|
Watermelon Daiquiri posted:yeah, ive seen this on the internet and a related old old qemu bug but unless it's a regression it's not it. Instead what I think it is is it has something to do with binding/rebinding some audio source for snd hda intel after the vm shuts down. I posted in the other thread, but it would help if you could think about each independent task individually, and then use any of the many job schedulers that are designed to fit a lot of smaller tasks onto bigger nodes efficiently. Or just use AWS Fargate, which gives you fine grained options over how much vCPU / RAM each individual task gets. You're going to get better, more cost efficient job throughput with a job scheduler, though.
|
#
¿
Dec 19, 2023 04:11
|
|
|
|
| # ¿ Apr 27, 2024 11:05 |
|
|
Nitrousoxide posted:Yeah, Docker Swarm or Kubernetes (K3s or K8s) would be your main starting point for a widely used on-prem or cloud scheduler. Unless you use some companies special sauce scheduler. Whats your 2c on kubernetes distributions that one might run on their own, physical hardware without wanting to pay VMWare licenses? Is K3s the winner for that? Canonical is out there marketing Mikrok8s wherever they can: https://microk8s.io/compare . Including in Ubuntu's default MOTD for a few years now.
|
|
#
¿
Dec 19, 2023 04:30
|
|
|
Yeah, it really sounds like you should look at AWS Fargate: https://aws.amazon.com/fargate You can use it via AWS Batch or just directly, but it lets you size the compute to each task and only pay on demand without having to actually deal with a VM yourself. Go ahead and launch 21 right-sized instances at once, that way when 1 task finishes you can stop paying for the resources it's using rather than having unused cycles on a bigger instance. Also, you get to call it "Fartgate". I take it that each of these things is single threaded, but wants a lot of RAM?
|
|
#
¿
Dec 19, 2023 05:36
|
|
|
Watermelon Daiquiri posted:nope, only like 500mb total, it's just a shittily optimized fd mesh but that fargate looks interesting. I'm just an EE who's been roped into trying to improve the run time of the models we have since I'm a computer nerd and I have so much other stuff to do, so I'll likely just say that getting everything set up will be a massive project. Aw jeez. Have you tried running it on your laptops with GNU parallel or something? You don't need cloud for this, especially if the point of this is to get faster turnaround time for a batch. A $600 Dell desktop will have 24 threads and 32GB of RAM, letting you easily run 21 of these simultaneously. Hell, if your laptops are somewhat modern they probably have more than 12 threads so they can chew through them much faster than running sequentially. If you don't want to deal with the full complexity of AWS and wanted a cheap hourly rental machine that you can log into and do what you need to, then destroy and stop paying for, that workload also would fit well onto a single Virtual Private Server from any of the much cheaper vendors: https://www.vultr.com/pricing/#cloud-compute/ Vultr has a 32vCPU / 64GB RAM CPU-optimized option for 95 cents per hour.
|
|
#
¿
Dec 19, 2023 16:44
|
|
|
ESXi is clearly hosed under Broadcom, if IBM hadn't simultaneously been wrecking Red Hat I'd actually be betting on Red Hat Virtualization/ oVirt picking up some market share. KVM itself is completely rock solid and has been for ages. There's not going to be any real replacement for how big ESXi was because operating an on-prem virtualization farm will become lostech and everyone will just pay AWS more money. Edit: if anything in the longer term I'd bet that the groups that continue to run on-premises will move to a bare-metal kubernetes solution without any hypervisor. You can run a KVM guest as a kubernetes pod if you need actual VM isolation. Twerk from Home fucked around with this message at 23:40 on Jan 5, 2024 |
|
#
¿
Jan 5, 2024 23:29
|
|
|
I have been skimming the Proxmox support forums, it's a mix of hobbyists that have no idea what they're doing running into predictable problems with their stupid home setups, and pros with what sounds like a reasonable, modern environment experiencing a bunch of problems with HA. Here's an example of the latter: https://forum.proxmox.com/threads/unexpected-fencing.136345/ quote:I have a 28 node PVE cluster running PVE 7.4-16. All nodes are Dell R640/R650 servers with 1.5 or 2TB of RAM and Intel Xeon Gold CPU's. They all have 2 x 1GB NIC's and 4 x 25 GB NIC's. HA sounds completely busted for them. Apparently that's around the effective max size of a proxmox cluster right now before it collapses under its own weight, judging by responses.
|
|
#
¿
Jan 5, 2024 23:49
|
|
|
fresh_cheese posted:I guess probably just general awareness, and then looking at it they have their own hypervisor but 5 minutes of googling doesnt turn up any description of what technology its based on. So thats a “hmmmmmmm….” My understanding is that Nutanix is just a much less trusted hypervisor at the core than VMWare, Hyper-V, or KVM/Qemu, which are all rock solid and proven. The various management interfaces for KVM might all suck, but KVM itself will never let you down. Nutanix was designed for doing hyper-converged VDI and just wasn't designed to be an all purpose virtualization solution. VMWare is not the only option, I bet we're going to see plenty of shops move to Hyper-V. Look at the OP of the Homelab thread for a firsthand account into why Nutanix failed to ever get a foothold in the hobbyist market, which matters because people will try things at home before bringing them up at work: H2SO4 posted:In case anybody is starting out on the homelab journey and is thinking about going the Nutanix CE route, let me offer a couple words of widsom.
|
|
#
¿
Jan 28, 2024 16:39
|
|
|
RVWinkle posted:I think you're looking at it backwards. You have more flexibility if you run docker in a vm because you can allocate resources and manage scaling. If you install docker on the base hypervisor then it will just use as much resources as it wants. I'll respond and say that CPU and memory limits with cgroups via docker is incredibly easy and has been mature for ages, otherwise it'd be pretty useless to try and schedule low latency heterogenous services on clusters. Security I'll give you, but the point of linux containers is that any modern container runtime can run them with no fuss, I don't see what portability advantages a VM gives you. I also wish that Proxmox had first class Docker or preferably Podman instead of LXC. LXC containers seem to combine some of the bad parts of both VMs and OCI containers together.
|
|
#
¿
Feb 1, 2024 03:52
|
|
|
HalloKitty posted:At least not where I live; all of the new guys learn ESXi and vCenter as part of their education, and can apply it immediately in the workplace. That's going to be irrelevant in future, as all the customers in the market we serve won't be in Broadcom's exclusive club. So the future really does just look like more people using Hyper-V but everything is worse?
|
|
#
¿
Feb 13, 2024 15:57
|
|
|
Dancing Peasant posted:My group is working to get off of VMWare (for reasons stated already). And while there has been discussion on ProxMox, management and some engineers are leaning towards OpenShift/OpenStack as another solution. I'm repeating secondhand (or worse) information and general community chatter that may not be up to date or reflect reality, but OpenStack has a reputation for being a pain box. It's difficult to deploy well, the integration of different components of it sometimes feel like they're not even part of the same over-arching product vision, and everyone's OpenStack ends up being a unique beast, making ongoing operations of it a pain. A good number of operators feel trapped on it, and I would bet that for a few years now it has not been a common choice for groups setting up a new environment. OpenShift seems much healthier by comparison, but that's k8s and not a complete replacement for clustered virtualization. Twerk from Home fucked around with this message at 19:13 on Mar 4, 2024 |
|
#
¿
Mar 4, 2024 18:52
|
|
|
tokin opposition posted:tell me to piss off if this isn't the right place for this, but if I wanted to get a job that does virtualization*, what kind of skills or homelab projects would be good for putting on a resume or talking about in an interview? Tokin I don't know your exact skillset or where specifically you are wanting to go, but here's what I'd learn if I were in helldesk, Good At Computer, and wanted to get a better job ASAP:
If you want to learn more or do it better, I'd also think about moving the database into a separate managed RDS instance like db.t4g.micro, in any real organization you'd be doing infrastructure as code so instead of creating resources in the AWS console you'd be using something like the CDK or Pulumi if you are lucky or Terraform if you are not. You could also move VMs that are doing something more complicated than Wordpress, but I don't know what languages or techs you're comfortable with. You could build a tiny thing in Django or Laravel or Spring Boot that you then move into AWS, ideally with more outside parts like Redis for caching or a message queue. If you really want to focus on on-premises infrastructure I'd replace the second half of the list with moving things into containers, then kubernetes pods. Keep the database on a separate VM outside of k8s, and if you've got 2 PCs to do this on have ESXi on one and move applications into k8s on VMs on Proxmox on the other one or something. Edit: in the longer term I don't think Proxmox is going to be a winner. I have no idea what will, but k8s or something substantially similar to it sure is around to stay. Also, I completely omitted all the networking/firewall/IAM stuff that would be the most complex parts of all this, figure out how to limit access to just your home IP and just your user through some identity platform. AWS Cognito maybe? And if you talk about this in an interview say "lift and shift multi-tier application" not "WordPress migration". Twerk from Home fucked around with this message at 15:05 on Mar 15, 2024 |
|
#
¿
Mar 15, 2024 14:52
|
|
|
|
| # ¿ Apr 27, 2024 11:05 |
|
|
in a well actually posted:Tailscale. For it to Just Work and be something that you'd use? Sure. To learn skills that you're hoping make you immediately employable? AWS Cognito https://aws.amazon.com/cognito/ and OIDC federation with an outside identity provider.
|
|
#
¿
Mar 15, 2024 17:55
|
|