|
Next week I'm planning on re-doing my home server that is stuffed full of 24 drives in ZFS pools. Here's what I'm thinking of doing, tell me if I'm dumb or not. 1. Ubuntu Server that doesn't do anything but run KVM. 2. A file server KVM-managed VM managing all my pools. 2. Another VM running Postgres and MySQL which I need for development and for some apps I run on the server. 3. Another VM that manages a half dozen docker-ified apps. I'm not sure if its worth it, but it feels like a nice idea to not run anything on the host OS but KVM and then delegate everything I use to VMs. Maybe its a better idea to just run docker on the host OS?
|
#
?
Jul 7, 2017 19:33
|
|
|
|
| # ? Jun 11, 2024 02:59 |
|
|
On a smaller scale than you're doing (well, as far as storage capacity is concerned) I have 1 host with 3 guests hanging off of it. The host is CentOS 7 and the guests are currently 2xCentOS and 1xFedora server. My storage is a mirror ZFS pair that's attached to the host and shared out to the rest via samba and NFS. The host OS (and subsequent qcow2 storage for guests) is all on a Samsung 850 Evo. So all my system drives are on the SSD and all of my data is on the mirror ZFS pair of spinny's. I used the same ethos as you regarding the host: it's there to run ZFS and KVM for the guests and do nothing else. I have spare room on the SSD to fire up another VM if I want to add another system. I can use a qcow2 from as little as 10GB to whatever size and provision some data storage via NFS/samba from my ZFS pool. I've also got flexibility with the ZFS pool, in that I can create another dataset on it as long as it's not full and use that if I need to further segregate my data. It's working OK so far. The main thing I'm doing differently from what you propose is that I have my ZFS storage attached to my host, not one of the VM's.
|
|
#
?
Jul 7, 2017 19:49
|
|
|
Thermopyle posted:1. Ubuntu Server that doesn't do anything but run KVM. Host OS as a minimal hypervisor is not a bad idea at all. Are your service instances also going to be running a flavor of Linux? An alternative could be to use LXD on the host instead of full-fat KVM and run your services within those. You can even run Docker inside an LXD instance. I've been running a similar setup with FreeBSD jails for years. My host OS has no services running except NTP and SSH, and everything else has its own jail, with appropriate portions of the filesystem bind/nullfs mounted in the right places, and network segmentation between jails is enforced by the host firewall. Performance is excellent, and I don't have to micromanage static resource assignments for each container, though I have the option. SamDabbers fucked around with this message at 20:02 on Jul 7, 2017 |
|
#
?
Jul 7, 2017 19:51
|
|
|
Further to the VM gaming related chat, how are people activating Windows on their virtual gaming platforma? Particularly 10, and particularly where you can pay for a license but be able to use it aver and over in case you need to tear down the VM and start again. Retail license?
|
|
#
?
Jul 7, 2017 20:43
|
|
|
SamDabbers posted:Host OS as a minimal hypervisor is not a bad idea at all. Are your service instances also going to be running a flavor of Linux? An alternative could be to use LXD on the host instead of full-fat KVM and run your services within those. You can even run Docker inside an LXD instance. Hmm, all guests are Linux except for one which is Windows XP(!). I guess I could do LXD for everything except the XP guest which I could leave on KVM.
|
|
#
?
Jul 7, 2017 20:53
|
|
|
Thermopyle posted:Hmm, all guests are Linux except for one which is Windows XP(!). I guess I could do LXD for everything except the XP guest which I could leave on KVM. You can certainly run both LXD and KVM on the same host. LXD just has less overhead and more flexibility for Linux instances than running a separate kernel in KVM. Out of curiousity, what do you need an XP guest for in 2017? apropos man posted:Further to the VM gaming related chat, how are people activating Windows on their virtual gaming platforma? Particularly 10, and particularly where you can pay for a license but be able to use it aver and over in case you need to tear down the VM and start again. Retail license? Windows 10 seems to tie activation to the system UUID, which is configurable. Just make it the same value on your new VM. The QEMU command line option is -uuid, and if you use a libvirt-based management tool, you can use virsh to change the VM's uuid in XML. code:SamDabbers fucked around with this message at 21:50 on Jul 7, 2017 |
|
#
?
Jul 7, 2017 21:38
|
|
|
SamDabbers posted:You can certainly run both LXD and KVM on the same host. Out of curiousity, what do you need an XP guest for in 2017? maybe you did it that way to more easily Show Your Work but you don't have to undefine/redefine. just 'virsh edit' that thing.
|
|
#
?
Jul 7, 2017 21:43
|
|
Associate Christ
|
other people posted:maybe you did it that way to more easily Show Your Work but you don't have to undefine/redefine. just 'virsh edit' that thing. Actually, I had to undefine/redefine in order to change the UUID in particular. I got an error when I tried to change it using virsh edit. code:
|
|
#
?
Jul 7, 2017 21:52
|
|
|
SamDabbers posted:You can certainly run both LXD and KVM on the same host. LXD just has less overhead and more flexibility for Linux instances than running a separate kernel in KVM. Out of curiousity, what do you need an XP guest for in 2017? A custom 16-bit line of business application developed in 1993 that my wife has to use for her work. The latest version of windows I can get it to run on in is XP.
|
|
#
?
Jul 7, 2017 22:01
|
|
|
SamDabbers posted:Actually, I had to undefine/redefine in order to change the UUID in particular. I got an error when I tried to change it using virsh edit. ah well see maybe i am an idiot then my phone doesn't have virsh so i can't test this right now edit: the man pages and libvirt.org make no mention of not being able to change a domain's uuid but maybe there is some limitation of the xml validator for this.... other people fucked around with this message at 22:13 on Jul 7, 2017 |
|
#
?
Jul 7, 2017 22:01
|
|
|
apropos man posted:Further to the VM gaming related chat, how are people activating Windows on their virtual gaming platforma? Particularly 10, and particularly where you can pay for a license but be able to use it aver and over in case you need to tear down the VM and start again. Retail license? You don't need to activate Windows 10 to install it, by the way. The only difference is that you can't use any personalization options and a persistent watermark reminding you to activate. I'm not using the VM passthrough method (yet?) but since last Christmas I've been dual booting with a Windows 10 ISO I grabbed from microsoft.com. I'm still getting updates, which is all I care about for a system I'm only using for games.
|
|
#
?
Jul 7, 2017 22:41
|
|
|
Thermopyle posted:A custom 16-bit line of business application developed in 1993 that my wife has to use for her work. The latest version of windows I can get it to run on in is XP. I wonder if it would run in dosbox, possibly inside win3.1.
|
|
#
?
Jul 7, 2017 22:46
|
|
|
SamDabbers posted:Actually, I had to undefine/redefine in order to change the UUID in particular. I got an error when I tried to change it using virsh edit. Yeah. That XML check thing was annoying the poo poo out of me the other day in Ubuntu until I managed to somehow get 'virsh edit' to ignore the template. A trick I then failed to replicate when I tore everything down and switched to Manjaro. The annoying thing is that sometimes the ignore key doesn't work. If you're going to give me the option gently caress this up, at least be consistent about it. I'll try activating Windows and then keep the UUID safe for when I inevitably reinstall something.
|
|
#
?
Jul 7, 2017 23:52
|
|
|
SamDabbers posted:Windows 10 seems to tie activation to the system UUID, which is configurable. Just make it the same value on your new VM. The QEMU command line option is -uuid, and if you use a libvirt-based management tool, you can use virsh to change the VM's uuid in XML. Note that in the case of OEM systems, this may actually be tied in an annoying efivar instead of a uuid, but you can also extract that. Thermopyle posted:Next week I'm planning on re-doing my home server that is stuffed full of 24 drives in ZFS pools. Here's what I'm thinking of doing, tell me if I'm dumb or not. If it were me, I'd manage the file storage on the host, and shove postgres/mysql and the docker containers into a coreos host with persistent storage using kubernetes to manage it all. I'd avoid LXD, not because it's bad, but because it's a Canonical project which is inevitably going to become less and less supported until it dies.
|
|
#
?
Jul 8, 2017 00:08
|
|
|
evol262 posted:If it were me, I'd manage the file storage on the host, and shove postgres/mysql and the docker containers into a coreos host with persistent storage using kubernetes to manage it all. I'm barely familiar with docker and the extent of my knowledge about kubernetes is from reading the past 10 minutes, and I still can't figure out what coreos is. Their home page is remarkably unhelpful! So, I'm guessing what youre saying here is that i leave VMs out of it and kubernetes runs my containers. I take it kubernetes can just manage containers on on the same host it is running on? I was originally under the impression that kubernetes would push containers to other machines (or VMs), but I really dont know what I'm talking about and I'm not sure where coreos comes in.
|
|
#
?
Jul 8, 2017 03:48
|
|
|
evol262 posted:I'd avoid LXD, not because it's bad, but because it's a Canonical project which is inevitably going to become less and less supported until it dies. For a home server it's not really much of a risk though. It's really too bad Linux container folk didn't implement kernel namespaces as a security boundary like FreeBSD's jails or Illumos' zones. The flexibility of being able to make a network namespace independently of a process namespace or a file system namespace is cool, but it complicates proper isolation to the point where a separate tool like LXD is necessary to lock everything down. Abstracting all the subsystems as a unit and baking the boundary into the implementation just makes everything simpler to manage and harder to gently caress up. For example, Docker containers wouldn't be nearly so easy to exploit. Thermopyle posted:I'm barely familiar with docker and the extent of my knowledge about kubernetes is from reading the past 10 minutes, and I still can't figure out what coreos is. CoreOS aka Container Linux is just a stripped down distro for running Docker containers. He's saying run it in a KVM VM and your Docker apps in it. Kubernetes is a slick automation tool for deploying and managing apps that run in Docker containers. SamDabbers fucked around with this message at 04:16 on Jul 8, 2017 |
|
#
?
Jul 8, 2017 04:03
|
|
|
How do you get notifications on ZFS errors/warning when you use a stock Linux OS?
|
|
#
?
Jul 8, 2017 07:56
|
|
|
SamDabbers posted:It's really too bad Linux container folk didn't implement kernel namespaces as a security boundary like FreeBSD's jails or Illumos' zones. The flexibility of being able to make a network namespace independently of a process namespace or a file system namespace is cool, but it complicates proper isolation to the point where a separate tool like LXD is necessary to lock everything down. Abstracting all the subsystems as a unit and baking the boundary into the implementation just makes everything simpler to manage and harder to gently caress up. For example, Docker containers wouldn't be nearly so easy to exploit. cgroups and namespacing were kind of an afterthought. The real problem with LXD is that using hardware virt to segment works, but it's yet another methodology (which makes sense given that Canonical has very few engineers) instead of really locking down the attack surface in the kernel. Plus, Linux isn't nearly as unified as BSD or Solaris. I love it and it works, but it's 200 independent projects stuck together, not one. So Docker getting brand awareness (which is mostly what their valuation is based on) means that anything based on rkt, containerd, the OCI, or another solution barely gets a glance. Thermopyle posted:I'm barely familiar with docker and the extent of my knowledge about kubernetes is from reading the past 10 minutes, and I still can't figure out what coreos is. Their home page is remarkably unhelpful! CoreOS is basically system+docker+cloud-init and nothing else, in an A/B update model instead of packages. They have a good "kubernetes for humans" setup. Kubernetes itself is etcd, skydns, flannel, and a couple of other pieces to turn "run this dockerfile on one host and map ports" into something called "pods", which are basically discrete groups of container(s) which can spread across multiple hosts, automatically scale, run in groups (like docker-swarm, in principle), manages networking mapping and container visibility, storage mapping, etc. It's to containers as openstack, vcloud, AWS, or whatever is to VMs. It's what most of the large container hosting (Azure, GCE, etc) runs on. CoreOS's kubernetes setup is dead simple. Or Openshift on top of a CentOS VM, which can be done in a one-shot setup from ansible or in a container, and also handles Jenkins workflows for building your containers from source and deploying them, mapping visible DNS names (with a wildcard for the host)
|
|
#
?
Jul 8, 2017 12:05
|
|
|
If I want to spin iSCSI and SRP targets, what do I go for? LIO or SCST? What's better maintained? Or is there something else? --edit: I suppose LIO since it's apparently in the kernel. Various pages on the web were fuzzy about it's state. I guess that's answered. Combat Pretzel fucked around with this message at 23:39 on Jul 10, 2017 |
|
#
?
Jul 10, 2017 23:19
|
|
|
Combat Pretzel posted:If I want to spin iSCSI and SRP targets, what do I go for? LIO or SCST? What's better maintained? Or is there something else?
|
|
#
?
Jul 11, 2017 04:59
|
|
|
Furism posted:How do you get notifications on ZFS errors/warning when you use a stock Linux OS? Write a cron (or timer unit these days) that runs `zpool status -x | fgrep -v 'all pools are healthy'` set to mail on any output?
|
|
#
?
Jul 12, 2017 03:35
|
|
|
How much does a scrub put wear and tear on your drives? Is once a fortnight too much or too little for `zpool scrub pool0`, where pool0 is just a mirror pair of WD Reds?
|
|
#
?
Jul 12, 2017 05:55
|
|
|
apropos man posted:How much does a scrub put wear and tear on your drives? Scrub frequency depends a lot on your workload patterns. For example, if you do regular full backups, you don't ever need to scrub, because ZFS detects (and, when it can, automatically fixes) bitrot transparently when you read. I've never found any benefit to scrubbing more than once every few months, though. Two days sounds really extreme even for paranoid people.
|
|
#
?
Jul 12, 2017 17:44
|
|
|
There's a whole lot of well-meaning enterprise-grade or even just wrong information on the internet about zfs (e.g, the 1gb ram/tb of storage "rule of thumb"). I saw a recommendation of weekly scrubs and to be honest it's really not needed. You have SMART hard drive health warning you about physical issues with the drives, checksum parity, and regular raid disk parity, so two weeks is a really short time for something to go wrong with your setup. I do once a month and I think somewhere between that and yearly is what you should shoot for. I ran a regular mdadm array for a while before I felt linux zfs was far enough along and I never had an error that only a zfs scrub could fix.Vulture Culture posted:ZFS scrubs aren't sequential. They will put some more wear and tear on your drive armatures than a traditional sequential RAID pass, but hard drives are actually designed to read and write bytes on disk pretty frequently. They'll be fine. Agreed, but a fortnight is 2 weeks
|
|
#
?
Jul 12, 2017 18:15
|
|
|
Thanks chaps. I've added a systemd timer to scrub on a Friday, somewhere in the middle of every month:code:I've also got email reports via postfix/gmail for both the S.M.A.R.T. tests and the scrub. I think this is probably more than enough for a home file server, so I can be confident that if anything untoward starts happening on the physical level or data integrity level I'm gonna get prior notice. It won't stop a drive from suddenly going kaput, of course, but between the two of them and scheduled backups I'll be alright.
|
|
#
?
Jul 12, 2017 23:07
|
|
|
hifi posted:Agreed, but a fortnight is 2 weeks In all seriousness, I'm probably the exact kind of person that crashed that orbiter into Mars.
|
|
#
?
Jul 13, 2017 04:21
|
|
|
I just wish that they'd fix the formatting when specifying a range of dates. You're supposed to be able to use either this way: OnCalendar=Fri *-*-13,14,15,16,17,18,19 02:00:00 Or this way: OnCalendar=Fri *-*-13-19 02:00:00 But last time I tried (for about 4 hours), the short way didn't work. I can't remember if it was in Ubuntu or CentOS at the time. I've never filed a bug report before, so I just left it. I had a thorough crack at getting the shortened version working and there's definitely something wrong, either in the code or the systemd manpage.
|
|
#
?
Jul 13, 2017 09:02
|
|
|
hifi posted:Agreed, but a fortnight is 2 weeks You guys and your crazy metric system measurements of time.
|
|
#
?
Jul 13, 2017 10:41
|
|
|
apropos man posted:I just wish that they'd fix the formatting when specifying a range of dates. You're supposed to be able to use either this way: The manpage here says: Each component may also contain a range of values separated by "..".
|
|
#
?
Jul 13, 2017 13:54
|
|
|
Polygynous posted:The manpage here says: Each component may also contain a range of values separated by "..". gently caress. I posted from my mobile at work. That's what I meant. I'd been trying with the double period, as stated in the manpage. I'd just misremembered.
|
|
#
?
Jul 13, 2017 14:17
|
|
|
oh. Then I have no idea. 
|
|
#
?
Jul 13, 2017 14:37
|
|
|
Is anyone here with Linux driving a 4k display with an nvidia card? I'm very interested in the support for nearest neighbour scaling as this would make gaming on a laptop with a 4k display far more attractive to me.
|
|
#
?
Jul 16, 2017 14:02
|
|
|
KingEup posted:Is anyone here with Linux driving a 4k display with an nvidia card? Yes, though I basically don't play games, and I don't think DSR is implemented on Linux yet (any games I do play are done with a passthrough GPU anyway)
|
|
#
?
Jul 16, 2017 16:37
|
|
|
I wanted to set some firewall rules using firewalld to allow inbound SNMP traffic, but only from a /32. You can't just do sudo firewall-cmd --zone=public --add-port=161/tcp --add-source=10.200.204.200/32 --permanent, because you can't add both a source IP and a source port. So instead, I made a new zone, added the source IP and port to it, and then added the snmp service to it too. With iptables I would have to chain this into an ACCEPT table, so I started to look for how to do that, but then I noticed SNMP traffic was flowing. Am I good?
|
|
#
?
Jul 18, 2017 22:07
|
|
|
anthonypants posted:eyou can't add both a source IP and a source port Is this true? If so, why? Sounds really bad. Any firewall can do that!
|
|
#
?
Jul 18, 2017 22:26
|
|
|
Furism posted:Is this true? If so, why? Sounds really bad. Any firewall can do that! quote:$ sudo firewall-cmd --zone=public --add-port=161/tcp --add-source=1.2.3.4/32 --permanent
|
|
#
?
Jul 18, 2017 22:28
|
|
|
Oh I believe you, I just want to understand why this limitation exists when it's a basic feature in any network firewall.
|
|
#
?
Jul 18, 2017 22:33
|
|
|
quote:To allow incoming SSH connections from a specific IP address or subnet, specify the source. For example, if you want to allow the entire 15.15.15.0/24 subnet, run these commands: https://www.digitalocean.com/community/tutorials/iptables-essentials-common-firewall-rules-and-commands First Google hit.
|
|
#
?
Jul 18, 2017 22:33
|
|
|
Yeah, I know how it works in iptables. I'm asking about firewalld though.
|
|
#
?
Jul 18, 2017 22:34
|
|
|
|
| # ? Jun 11, 2024 02:59 |
|
|
anthonypants posted:Yeah, I know how it works in iptables. I'm asking about firewalld though. Oops misread as iptables
|
|
#
?
Jul 18, 2017 22:35
|
|