Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Motronic
Nov 6, 2009

Agrikk posted:

Depending on the exact model and generation of your SmartUPS, a network management card for the smart slot will run you $20 if your ups is older, $100 if newer.

I strongly recommend the management card for diagnostics, monitoring, alerting and network shutdown.

Some cards even have external probes for room humidity and temperature sensors.

Quite worth it IMO

The APC BackUPS line are consumer/end user UPS units inteded for desktop machines. You can't add a network card to them. That would be the SmartUPS line.

Adbot
ADBOT LOVES YOU

chocolateTHUNDER
Jul 19, 2008

GIVE ME ALL YOUR FREE AGENTS

ALL OF THEM
Was just messing around in my homelab for the first time in awhile, and set up Tailscale on my PfSense router, advertised my internal subnet and tested it on my iPhone.

Holy poo poo, I cannot believe how easy that was. Insane.

Wibla
Feb 16, 2011

Tailscale is magic :eng101:

Hughlander
May 11, 2005

Just becareful with it and thread favorite proxmox. I set it up on the Hypervisor like a total rube. No problems everything is great for months. Next reboot, no LXC has DNS since on boot of an LXC /etc/resolv.conf is written based on host settings, and the host was using tailscale DNS and the LXC has no tailscale installed or running.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

Motronic posted:

The APC BackUPS line are consumer/end user UPS units inteded for desktop machines. You can't add a network card to them. That would be the SmartUPS line.

Whoops! I readBackUPS as SmartUPS. My mistake.

chocolateTHUNDER
Jul 19, 2008

GIVE ME ALL YOUR FREE AGENTS

ALL OF THEM

Hughlander posted:

Just becareful with it and thread favorite proxmox. I set it up on the Hypervisor like a total rube. No problems everything is great for months. Next reboot, no LXC has DNS since on boot of an LXC /etc/resolv.conf is written based on host settings, and the host was using tailscale DNS and the LXC has no tailscale installed or running.

Sounds like a nice lesson learned!

I have it running on my pfsense router via their built in package. Honestly I don’t have a real use case for it right now…but it was something to do and pretty cool. I’m sure it’ll come in handy in the future.

Hughlander
May 11, 2005

chocolateTHUNDER posted:

Sounds like a nice lesson learned!

I have it running on my pfsense router via their built in package. Honestly I don’t have a real use case for it right now…but it was something to do and pretty cool. I’m sure it’ll come in handy in the future.

It was... Just finding it took an embarrassingly long time because the symptom was that a docker container was in a crash loop without any output. Turns out the reason why was it tried to bind to a port by DNS, and well it's always DNS.

Wibla
Feb 16, 2011

Things that route packets belong on routers/firewalls, not hypervisors :psyduck:

Hughlander
May 11, 2005

Wibla posted:

Things that route packets belong on routers/firewalls, not hypervisors :psyduck:

I disagree. VPNs are for clients and should be as close to the client as possible. If an LXC needs a VPN connection, that VPN connection should be on the LXC. The docker host in particular has 4 different VPNs for different services if you count tailscale.
PIA in Docker for transmission - If PIA isn't connected, no transmission traffic goes out.
OpenVPN inbound for when I'm offsite.
OpenVPN outbound from one of the LXCs. The LXC has 'public' traffic which is to say I have a Digital Ocean box that passes 443 over the VPN to that LXC.
Tailscale that I haven't adopted yet.
That doesn't count various other times I'll spin up a PIA VPN and use docker --network container:pianame to get around google's ratelimits. Or even the fact that my windows 11 chrome instance can use a socks5 proxy from either PIA or the OpenVPN outbound when I need to use a different IP.

I don't think that I have enough knowledge to configure a unifi router to handle the complexities there.

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



Wibla posted:

Things that route packets belong on routers/firewalls, not hypervisors :psyduck:

By definition the hypervisor has to route packets. It has to know where packets inbound/outbound to its VMs need to go. Especially if it isn't creating unique network hosts on the network for each VM as is the case when you are dealing with user level VMs in Linux.

Plus there's the added benefit of it being able to route way faster than jumping out to a LAN and back if it can be done entirely locally on the physical machine.

CommieGIR
Aug 22, 2006

The blue glow is a feature, not a bug


Pillbug
Yeah, most Hypervisors have some sort of built in router/switching. Otherwise, you can't really handle a bunch of hosts that are often on different VLANs talking to the general network.

That being said - if you are specifically trying to provide a general network service like VPN, Routing, Firewalling, you do generally put those either in dedicated hardware or within a VM in the pool.

I use PFSense to do Firewalling/Routing inside my clusters, broken up by switching and interfaces.

Wibla
Feb 16, 2011

CommieGIR posted:

Yeah, most Hypervisors have some sort of
That being said - if you are specifically trying to provide a general network service like VPN, Routing, Firewalling, you do generally put those either in dedicated hardware or within a VM in the pool.

I use PFSense to do Firewalling/Routing inside my clusters, broken up by switching and interfaces.

This is what I was trying to get at.

Warbird
May 23, 2012

America's Favorite Dumbass

I don’t see a huge advantage to Tailscale and I feel like I’m missing something. It was pretty trivial to set up a WireGuard instance hooked to a DDNS domain and have my LAN readily available via turning on the tunnel when needed. TS seems neat, but as best I can tell you’re just cutting out some of the in between steps and then it’s just a machine to machine connection. Neat and handy for sure but it seems like if you’re running umpteen services locally across containers and so on that you’re only causing more work for you self to integrate it to each host. Am I completely off base here?

Scruff McGruff
Feb 13, 2007

Jesus, kid, you're almost a detective. All you need now is a gun, a gut, and three ex-wives.

Warbird posted:

I don’t see a huge advantage to Tailscale and I feel like I’m missing something. It was pretty trivial to set up a WireGuard instance hooked to a DDNS domain and have my LAN readily available via turning on the tunnel when needed. TS seems neat, but as best I can tell you’re just cutting out some of the in between steps and then it’s just a machine to machine connection. Neat and handy for sure but it seems like if you’re running umpteen services locally across containers and so on that you’re only causing more work for you self to integrate it to each host. Am I completely off base here?

By default it's machine to machine but you can set it to act as a regular VPN by designating an exit node. Otherwise you've pretty much got it, they're essentially just making Wireguard a bit more user friendly, especially if you don't own your own domain. I use it as a backup to my Wireguard since that's hosted on my main server and if that's down I can't get into anything. My PiKVM is on TS so I can get to the bare metal of the server to try and get that back up.

Scruff McGruff fucked around with this message at 19:00 on Nov 14, 2023

Warbird
May 23, 2012

America's Favorite Dumbass

Ooh, the PiKVM thing is a good idea. I’ve been vaguely meaning to get around to getting 2fa on there so I’d have access to Proxmox to spin up redundant VPNs if my main and backup went offline (though the backup is on my router so if that’s fubar it’s all fubar) and that would be a better alternative than exposing it. Does TS use WireGuard as their means of VPN-ing?

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



Yeah. TailScale uses Wireguard as the vpn back end.

chocolateTHUNDER
Jul 19, 2008

GIVE ME ALL YOUR FREE AGENTS

ALL OF THEM
I need something like ITGlue, but for my Homelab. Any recommendations?

ITGlue for reference:

https://www.itglue.com/

Motronic
Nov 6, 2009

chocolateTHUNDER posted:

I need something like ITGlue, but for my Homelab. Any recommendations?

ITGlue for reference:

https://www.itglue.com/

I don't understand. What's wrong with homenetworkips.txt?

Kibner
Oct 21, 2008

Acguy Supremacy

chocolateTHUNDER posted:

I need something like ITGlue, but for my Homelab. Any recommendations?

ITGlue for reference:

https://www.itglue.com/

Is Writerside similar to what you are looking for? You use it to create technical documentation and it spits out a static site for you. It's in early access right now, though.

chocolateTHUNDER
Jul 19, 2008

GIVE ME ALL YOUR FREE AGENTS

ALL OF THEM

Kibner posted:

Is Writerside similar to what you are looking for? You use it to create technical documentation and it spits out a static site for you. It's in early access right now, though.

This seems cool, but not exactly what I'm looking for. I'm looking for more of an ITGlue or Hudu style thing:

https://www.youtube.com/watch?v=vn0f1yHIh3s

https://www.youtube.com/watch?v=cEOZ7T3rCaM

Maybe Netbox?

Sixfools
Aug 27, 2005

You be the Moon,
I'll be the Earth
And when we burst
Start over, oh, darling
wrong thread

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.
What distributed storage are y'all running at home with low node counts? I janitor Ceph at work and not only want to learn something different at home, but also know that Ceph is much happier when both host and disk counts are much, much higher than I want to spend at home.

What's the high-level overview of Starwind VSAN vs VMWare VSAN vs Hyper-V Storage Spaces Direct? I know that officially VMWare VSAN and Microsoft SDS need validated hardware, but I'd hope I could shove it onto home stuff by saying "i know what I"m doing" and just feeing it 1 full NVME disk each across 3 nodes. Thoughts? How are you guys distributing your storage?

Hughlander
May 11, 2005

Twerk from Home posted:

What distributed storage are y'all running at home with low node counts? I janitor Ceph at work and not only want to learn something different at home, but also know that Ceph is much happier when both host and disk counts are much, much higher than I want to spend at home.

What's the high-level overview of Starwind VSAN vs VMWare VSAN vs Hyper-V Storage Spaces Direct? I know that officially VMWare VSAN and Microsoft SDS need validated hardware, but I'd hope I could shove it onto home stuff by saying "i know what I"m doing" and just feeing it 1 full NVME disk each across 3 nodes. Thoughts? How are you guys distributing your storage?

Ceph through proxmox or just resilio in a container if it’s going to windows.

cr0y
Mar 24, 2005



So I have been kicking around an idea...

I posted earlier but I have a couple free R740s with a ton of processor and memory but no GPU. I'd really like to build a machine learning rig.

Does anyone know of a semi sane way that I can shoe horn 1 or 2.... RTX 3080s(????) into that chassis? I know the power connectors are going to be a problem, that's a separate thing.

They would like 5X my power bill so I really can't use them as a replacement for my relatively efficient home server but I want to do something cool with that metal because it's really nice gear for free.

~Coxy
Dec 9, 2003

R.I.P. Inter-OS Sass - b.2000AD d.2003AD
Do you have to keep the cards in the chassis?

Get a cheap ATX case, mount a PSU and the GPUs inside that, use risers to route the slots to the R740.

Bjork Bjowlob
Feb 23, 2006
yes that's very hot and i'll deal with it in the morning


Twerk from Home posted:

What distributed storage are y'all running at home with low node counts? I janitor Ceph at work and not only want to learn something different at home, but also know that Ceph is much happier when both host and disk counts are much, much higher than I want to spend at home.

What's the high-level overview of Starwind VSAN vs VMWare VSAN vs Hyper-V Storage Spaces Direct? I know that officially VMWare VSAN and Microsoft SDS need validated hardware, but I'd hope I could shove it onto home stuff by saying "i know what I"m doing" and just feeing it 1 full NVME disk each across 3 nodes. Thoughts? How are you guys distributing your storage?

I'm using ceph at home with 3 hosts each with 5 OSDs and I haven't had any problems with it. I switched over from Gluster maybe nine months ago and haven't looked back - Gluster (and the RH derived product) are on life support anyway so it's not wise to go with it.

I don't have any experience with the options you mentioned however.

Aware
Nov 18, 2003

cr0y posted:

So I have been kicking around an idea...

I posted earlier but I have a couple free R740s with a ton of processor and memory but no GPU. I'd really like to build a machine learning rig.

Does anyone know of a semi sane way that I can shoe horn 1 or 2.... RTX 3080s(????) into that chassis? I know the power connectors are going to be a problem, that's a separate thing.

They would like 5X my power bill so I really can't use them as a replacement for my relatively efficient home server but I want to do something cool with that metal because it's really nice gear for free.

I have a 740xd and kinda wanted to do the same initially but couldn't figure out a semi same way to even attempt it.

Azhais
Feb 5, 2007
Switchblade Switcharoo

Bjork Bjowlob posted:

I'm using ceph at home with 3 hosts each with 5 OSDs and I haven't had any problems with it. I switched over from Gluster maybe nine months ago and haven't looked back - Gluster (and the RH derived product) are on life support anyway so it's not wise to go with it.

I don't have any experience with the options you mentioned however.

What sort of hardware are you on?

Bjork Bjowlob
Feb 23, 2006
yes that's very hot and i'll deal with it in the morning


Azhais posted:

What sort of hardware are you on?

It used to be three mostly identical HP DL380p servers with a mix of SAS and NVMe OSDs, but two of the servers have now been replaced with AM4 desktops with the same OSD arrangement to try and keep the power costs down. I was running the desktops as Proxmox nodes anyway, just not part of the ceph cluster, so consolidating the stack cut the power usage.

Blurb3947
Sep 30, 2022
I got Tailscale up after reading a lot about it. Had a bunch of issues trying to figure out how to use the app for my NAS off wifi but after the initial hump it is pretty cool just how easy it is.

Now all of my data is flowed through my desktop and I added my LAN subnet to also get LAN only stuff. I should maybe get a VPS or VPN so it's not my home IP being shown everywhere.

TraderStav
May 19, 2006

It feels like I was standing my entire life and I just sat down
Maybe a dumb question, but if I'm connected from my phone or other device to Tailscale, is that the same thing as activating my separately installed WireGuard connection? Meaning having private vpn to my home. I know I'm accessing my home services with Tailscale but not explicitly confident I'm concealing my traffic if on another connection.

Nitrousoxide
May 30, 2011

do not buy a oneplus phone



TraderStav posted:

Maybe a dumb question, but if I'm connected from my phone or other device to Tailscale, is that the same thing as activating my separately installed WireGuard connection? Meaning having private vpn to my home. I know I'm accessing my home services with Tailscale but not explicitly confident I'm concealing my traffic if on another connection.

You can go to https://whatismyipaddress.com/ while on your phone and away from your wifi to see if you still have the same IP. If you do it should be vpn'd the whole way.

CommieGIR
Aug 22, 2006

The blue glow is a feature, not a bug


Pillbug
I figure its time to get off the 13 year old AMD Opterons and onto something in the single digits, so trying out my Dell M420 miniblades

flappin fish
Jul 4, 2005

Nitrousoxide posted:

You can go to https://whatismyipaddress.com/ while on your phone and away from your wifi to see if you still have the same IP. If you do it should be vpn'd the whole way.

By default, Tailscale doesn't run as a full VPN, it just does direct Wireguard connections between things in the tailnet. You can set it up to tunnel all traffic through a chosen exit node, see https://tailscale.com/kb/1103/exit-nodes/.

Kibner
Oct 21, 2008

Acguy Supremacy
I just ordered 128GB of ram (https://pcpartpicker.com/product/2pYmP6/kingston-server-premier-32-gb-1-x-32-gb-ddr4-3200-cl22-memory-ksm32ed832hc) for my home server/NAS. At least ZFS will be able to use it well and I will never have to worry about running low!

BlankSystemDaemon
Mar 13, 2009




Kibner posted:

I will never have to worry about running low!
I've got bad news for you; if you've stuck with Linux and solve the "Solaris POSIX Layer is a couple of ~2100 LoC files without any integration into the VM, so any allocation above 16k pisses off the cache allocators" by setting the arc_max OID to half the systems total memory, you're still going to run into low memory issues.

On FreeBSD, the SPL is ~300 LoC shim with integration into uma(9) so you can end up with systems that have no free memory - but at least it's less of an issue:
pre:
last pid: 91172;  load averages:  0.32,  0.32,  0.32  up 27+22:01:55    18:18:22
89 processes:  1 running, 87 sleeping, 1 zombie
CPU:  0.4% user,  0.0% nice,  0.6% system,  0.0% interrupt, 98.9% idle
Mem: 412M Active, 2436M Inact, 26G Wired, 2245M Free
ARC: 24G Total, 22G MFU, 768M MRU, 432K Anon, 607M Header, 339M Other
     22G Compressed, 86G Uncompressed, 3.94:1 Ratio
Swap: 2048M Total, 2048M Free
EDIT: There's also been talk of further integration ZFS ARC into FreeBSD, by making it part of unified buffer cache - but unfortunately I don't know the status of that project.

BlankSystemDaemon fucked around with this message at 18:29 on Nov 29, 2023

Kibner
Oct 21, 2008

Acguy Supremacy
ngl, I don't think I fully get what you mean.

Kung-Fu Jesus
Dec 13, 2003

Pretty sure he's referring to the fact that on Linux, ARC size defaults to 50% of system memory because [arcane reasons] and if you try to set it higher, you're entering a very dangerous zone. And that FreeBSD doesn't have this problem.

Kibner
Oct 21, 2008

Acguy Supremacy

Kung-Fu Jesus posted:

Pretty sure he's referring to the fact that on Linux, ARC size defaults to 50% of system memory because [arcane reasons] and if you try to set it higher, you're entering a very dangerous zone. And that FreeBSD doesn't have this problem.

Ahh, gotcha. Yeah, I am fine with it using "only" half my memory. 64 GB (or near enough) still feels like a heck of a lot!

Adbot
ADBOT LOVES YOU

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb
What's the most CPU & memory you can squeeze in a 1U that consumes under 200W?

fletcher fucked around with this message at 10:37 on Nov 30, 2023

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply