|
I’ve got some pretty specific server adapter/backplane/NVMe drive questions if anyone has some knowledge. Like what does a Dell or Lenovo do to forward PERST# to the drive if the cable connection between the adapter and the backplane is mini-sas HD or slimsas which doesn’t natively have a concept of pcie fundemental reset? Is that generated by the backplane or forwarded with a separate cable? If a vendor says their backplane is NVMe do they actually mean SFF-8639 quad pcie/enterprise pcie or do they mean 2x PCIe over enhanced SAS? Or both? Or something else? If the latter, how do I tell the difference?
|
#
?
Dec 12, 2022 14:25
|
|
|
|
| # ? May 16, 2024 17:46 |
|
|
Murgos posted:I’ve got some pretty specific server adapter/backplane/NVMe drive questions if anyone has some knowledge. For the PERST# it could be a separate cable (regular sata works well for pcie clock and perst) or it could be using the reserved pins on a minisas hd. Sometimes the backplane uses an i2c to gpio and the PERST is actually generated on the backplane itself, with the host communicating on the i2c bus for that and led control etc. Also there is a bit of slop in the spec so if the PERST# is not connected on the drive side it will have a pull up and will just get released and the host doesn't have control over it. Usually it just links up fine regardless. For a nvme backplane I would assume it was 8639 connected either by minisas hd or something like oculink. There are probably some U.3 but those should be very well labelled to avoid confusion. I think if a vendor says NVMe it means the pcie lanes can get to the connector to the drive but the host side interface could be a bunch of different things and it could be x2, x4, etc. Would usually need to look more into the spec sheet to be sure.
|
|
#
?
Dec 12, 2022 23:48
|
|
|
priznat posted:For the PERST# it could be a separate cable (regular sata works well for pcie clock and perst) or it could be using the reserved pins on a minisas hd. Sometimes the backplane uses an i2c to gpio and the PERST is actually generated on the backplane itself, with the host communicating on the i2c bus for that and led control etc. Also there is a bit of slop in the spec so if the PERST# is not connected on the drive side it will have a pull up and will just get released and the host doesn't have control over it. Usually it just links up fine regardless. Thanks for this. It matches what I’ve observed but it’s hard to get detailed information. I have noticed two behaviors with the first being that in some instances PERST is pulled up and high all the time. Which is a problem for the Xilinx FPGA pcie ip because it expects to see an edge on PERST to kick off its state machine to start link training. I can probably work around this with an overboard wire from the FPGA to toggle the input after bring up if the hosts PERST is deasserted and we missed the edge. I’ve observed that if I manually force an edge everything links up okay. The second one is that the host adapter card I bought for testing doesn’t ever negate PERST. That the backplane is sometimes responsible for PERST is good for me because it means I didn’t fundamentally misunderstand the NVMe and U.2 spec. I was naively using a min-sas hd to sff-8639 cable from the adapter and no PERST rise on E5 even on a cold boot. I’ll double check the reserved pins just in case PERST is there but on a different pin than I expect. That the host side cabling is vendor specific isn’t something I anticipated.
|
|
#
?
Dec 13, 2022 02:23
|
|
|
Yeah it can be a bit of a mess sometimes especially when the solution is something internal to a fixed system and they don't care about being pcie spec compliance. Often what is presented to the end device like a drive or a card is ok but the caring in between can be a bit of a mystery if the vendor doesn't spell it out clearly (and often they don't have to). Like you have seen, sometimes things get along just fine without a proper perst deassertion and some endpoints don't even care and do their own thing. This isn't even really tested against in the compliance workshops, as long as the receivers are detected within the time of deassertion they will link up and everyone is happy.
|
|
#
?
Dec 13, 2022 03:43
|
|
|
Hope this is the right thread for this, I’m thinking about building a home media/general purpose server, especially now that we have a fiber connection with no data cap. Things I’d like this box to do, eventually: -host media we can stream locally via Plex or equivalent -host a nanny cam for the dogs, ideally with remote access on our phones -maybe handle phone photo syncing too so I can ditch iCloud? Host my own email too eventually? -host some other home automation stuff like car charging data logging, home energy use/solar production when we get around to it My hardware/software competence consists of maintaining my gaming PC-of-Theseus for the past 15 years and doing some basic terminal stuff in Linux at work (I only need to remember like a half-dozen commands to do this). There are a lot of YouTube tutorials on this kinda stuff but it’s hard to know where to start. Is this a reasonable project or a waste of time? I know a lot of these bits exist as off the shelf solutions but I want them without having Amazon or whoever hosting my data.
|
|
#
?
Dec 13, 2022 23:04
|
|
|
quote:-maybe handle phone photo syncing too so I can ditch iCloud? Host my own email too eventually? Please don't host your own email. And unless you want to janitor backups and do it right, the fee to pay for iCloud photo backup is worth every penny (assuming its in your budget).
|
|
#
?
Dec 13, 2022 23:08
|
|
|
On the photo front, maybe it’s simpler to annually remove the oldest year from iCloud and store it locally so I’m not continually increasing iCloud storage size indefinitely? Same could go for email I guess? Mostly I see that I’m approaching those storage limits and don’t like the idea of being forced into paying for more and more over time. Both of those uses are kinda throw-ins compared to the rest.
|
|
#
?
Dec 13, 2022 23:13
|
|
|
How many years back is your email going for you to be filling it up? I'd build a plan around deleting old messages rather than trying to archive it.
|
|
#
?
Dec 13, 2022 23:21
|
|
|
Been using gmail since like 2004 probably? Email is the bottom of this feature list I care about. If people here who know better say it’s dumb I’ll abandon it. Still interested in the rest though.
|
|
#
?
Dec 13, 2022 23:29
|
|
|
I tell everybody the same thing, but an Unraid box would satisfy most of your needs and allow you to expand your docker services as you venture into the self hosting world. The one great thing about your Fiber internet is you can just setup auto connecting to your server through VPN and take advantage of things like locally hosted DNS with adblocking (https://github.com/trinib/AdGuard-WireGuard-Unbound-DNScrypt) without a noticeable drop in performance or even host high bitrate 4K content for friends and family. With Unraid, you'll need to use your largest drive as prairty, but you get some data loss protection and it can completely be managed from a webgui on any device. I have some friends who treat their unraid boxes as a cloud service and others treat them more as media downloader/streamer boxes. You can do some pretty cool things with them, like setup your own private game servers for friends. Another friend setup Unraid with GPU passthru and a Windows vm so he can game in his basement, but also enjoy the other perks of Unraid. Self hosting e-mail is hard just because you run the risk of some spam filter getting mad at you and not routing stuff correctly. I'd definitely recommend self hosting photos (I heard NextCloud + Photoprism are good), but you're going to always want to keep a third copy of your data offsite somewhere - whether that be iCloud, Google Drive, Dropbox, whatever. Corb3t fucked around with this message at 23:37 on Dec 13, 2022 |
|
#
?
Dec 13, 2022 23:34
|
|
|
That sounds like the kind of thing I’m looking for. And yeah it makes sense to pay for an offsite backup of the whole thing too. Any resources you can point me towards to figure out what my hardware needs are?
|
|
#
?
Dec 13, 2022 23:39
|
|
|
bawfuls posted:That sounds like the kind of thing I’m looking for. And yeah it makes sense to pay for an offsite backup of the whole thing too. You could honestly use old PC equipment if you really wanted - you just need a flash drive to get started and install the Unraid OS and you can plug it in and boot into it. Your biggest concerns are likely video transcoding, you can't go wrong with an 11th or 12th gen Intel CPU with 6/8+ cores (onboard video for initial setup + quicksync transcoding is really nice), 32GB of RAM, and 2x platter drives (matching size, one will be used as parity, many nerds will shuck ~$200 Western Digital 14TB EasyStore Externals from Best Buy), and a fast 1x m.2 nvme gen 4 drive (1TB for ~$80, 2TB for ~$160) to be used as a cache drive - this is where your Unraid will keep appdata, docker service containers, metadata, etc. If you're interested in picking up a GPU, you could consider a Ryzen CPU - Plex's Ryzen CPU hardware transcoding isn't very good currently, which is a shame, so many users rely on a GPU for hardware transcoding. Motherboard wise, it'll be up to you whether or not your want 2.5 or 10 Gbe LAN, how many m.2 pci gen 4 slots, SATA slots, etc. A USB battery backup is nice, and you can setup Unraid to safely shutdown when your UPS's battery is depleted, and set your BIOS to restart your unraid server every morning if its been shut down. If you use macOS computers, you can even setup a Time Machine share and have your devices auto backup hourly via wifi. Constantly learning about cool new apps here Unraid's guide is good My Unraid Dashboard:  My docker container services (many are exposed through a reverse proxy and a custom domain using cloudflare):  Lots of great YouTube resources for setting up Unraid: https://www.youtube.com/watch?v=huCE4jtXOjQ https://www.youtube.com/watch?v=CcRwT7iHIcc&t=989s https://www.youtube.com/watch?v=cZTWC_z9rKs Corb3t fucked around with this message at 00:13 on Dec 14, 2022 |
|
#
?
Dec 13, 2022 23:49
|
|
|
Thanks for all that info, I’ll sift through it when I get some more time later. Last question for now, what is the value proposition of Unraid compared to free Ubuntu options that come up when I google this?
|
|
#
?
Dec 14, 2022 00:12
|
|
|
I've personally only ever messed with FreeNAS as an alternative, and felt that Unraid had a much more user friendly and more intuitive webgui and overall interface as a first time NAS user. Unraid has a really generous trial license, I used a trial key for over 6 months and had no qualms paying the $60 license after seeing how easy it was to setup and manage. I have since upgraded to their unlimited license over the years during cyber monday deals. It's also worth pointing out that you'll need to pay for Plex Pass if you want to utilize hardware transcoding, or try out alternatives like Emby (also has a premiere paid version) and Jellyfin (requires more to setup but offers things like Ryzen transcoding).
|
|
#
?
Dec 14, 2022 00:19
|
|
|
Unraid takes a looooot of computer janitoring out of the equation, from storage management, docker/VM management, shares, etc. I wouldn't not recommend diy as a learning experience but after doing that for several iterations I've settled on Unraid as a nice mix between home tinkering with docker stuff and not having to do the heavy lifting to keep my most used services running and up to date. Eg plex and all the arrs.
|
|
#
?
Dec 14, 2022 00:33
|
|
|
bawfuls posted:Same could go for email I guess? Mostly I see that I’m approaching those storage limits and don’t like the idea of being forced into paying for more and more over time. The simplest solution for that is an IMAP email client like Thunderbird. Once a year create a new local folder, move a bunch of old emails there and then make a backup of that folder. Thunderbird is a good option since it uses simple, standard MBOX format for the local folders.
|
|
#
?
Dec 14, 2022 01:08
|
|
|
As a "professional" Linux toucher I could get 90% of the way towards UnRAID on Centos but I don't want to do my day job in off hours. UnRAID is a nice balance of "I can get as dirty and deep as I want or just set it and basically forget it". My only complaint is the hard drive array is kind of slow. Make sure you have at least 1 SSD to run as a cache for your docker containers and temp storage for downloading/unpacking. Apparently ZFS is coming, but I have no idea if it's going to have the flexibility of mix and match that the current setup has.
|
|
#
?
Dec 14, 2022 01:55
|
|
|
For what it's worth, when I tried both unraid and truenas (scale, specifically) I thought truenas was more intuitive to me because unraid felt too much like it was trying to save me from myself. It's not like truenas has needed me to touch it much since I set it up either and it's been well worth the free.
|
|
#
?
Dec 14, 2022 03:15
|
|
|
@bawfuls There's loads of info here too... https://forum.level1techs.com/
|
|
#
?
Dec 14, 2022 12:35
|
|
|
I liked the idea with Unraid that hosing the array wouldn't lose *all* of my data, I could just plug a disk in and get the files that were still there. I don't care about performance, I'm running my array on something that should have been recycled by now, and none of the files are important. Unraid involved the least computer touching of the other options, and Synology was a bit pricey.
|
|
#
?
Dec 14, 2022 12:58
|
|
|
Corb3t posted:My docker container services (many are exposed through a reverse proxy and a custom domain using cloudflare): wait hold on how did you get your docker services to look like that, I want to do that
|
|
#
?
Dec 14, 2022 14:03
|
|
|
SolusLunes posted:wait hold on how did you get your docker services to look like that, I want to do that https://forums.unraid.net/topic/89702-plugin-docker-folder/ https://github.com/hernandito/unRAID-Docker-Folder-Animated-Icons---Alternate-Colors
|
|
#
?
Dec 14, 2022 14:43
|
|
|
Corb3t posted:My docker container services (many are exposed through a reverse proxy and a custom domain using cloudflare): Exposing stuff to the internets is extremely risky. Consider using a VPN if you absolutely must. From the icons in there they don't look like essential things at all.
|
|
#
?
Dec 14, 2022 17:02
|
|
|
Volguus posted:Exposing stuff to the internets is extremely risky. Consider using a VPN if you absolutely must. From the icons in there they don't look like essential things at all. I always forget when I'm posting in the self hosting thread or NAS thread, its funny how different of a response you get when discussing exposing self hosting services to the internet. You do you, but I'll continue to enjoy the convenience of being able to access the a few services (overseer, jellyfin, sabnzbd, changedetection.io) without having to constantly reconnect to my VPN. It would be a different story if I still had 1 Gbps Fiber and could VPN in to my home network on demand. Plus I really like my server widgets on my phone, and my friends love being able to make media requests with Overseerr Nginx Proxy Manager + Cloudflare has tons of mitigation for that all types of nefarious activities, you can block non-US IPs, DDoS prevention, I use a unique user/40 character unique password for every service, 2fa whenever possible, etc. It would be really nice if all these self hosting apps enabled 2fa. I even keep my ssh port open (with key-based authentication) with Deny Hosts running to block any IPs that attempt to login and fail. YouTube Guides on exposing services using Cloudflare: https://www.youtube.com/watch?v=I0lhZc25Sro https://www.youtube.com/watch?v=GarMdDTAZJo https://www.youtube.com/watch?v=cI17WMKtntA Corb3t fucked around with this message at 17:48 on Dec 14, 2022 |
|
#
?
Dec 14, 2022 17:37
|
|
|
Can someone point me towards some options for NAS focused cases? I'm looking for a compact, relatively quiet case to build a simple 4-6 bay NAS, miTX would be fine. It seems like most of the compact cases have ditched 3.5" bays, so I don't even know what's a good option anymore. Is the decade old Fractal Node 304 still the best in this space?
|
|
#
?
Dec 14, 2022 17:41
|
|
|
Twerk from Home posted:Can someone point me towards some options for NAS focused cases? I'm looking for a compact, relatively quiet case to build a simple 4-6 bay NAS, miTX would be fine. It seems like most of the compact cases have ditched 3.5" bays, so I don't even know what's a good option anymore. Is the decade old Fractal Node 304 still the best in this space? Is a Fractal Node 804 too big? It can fit 10 drives:  https://www.reddit.com/r/unRAID/comments/wpooe1/fractal_design_node_804_build/ Corb3t fucked around with this message at 17:50 on Dec 14, 2022 |
|
#
?
Dec 14, 2022 17:48
|
|
|
Twerk from Home posted:Can someone point me towards some options for NAS focused cases? I'm looking for a compact, relatively quiet case to build a simple 4-6 bay NAS, miTX would be fine. It seems like most of the compact cases have ditched 3.5" bays, so I don't even know what's a good option anymore. Is the decade old Fractal Node 304 still the best in this space? If you want ultra-compact there's the Jonesbo N1, or upcoming N2. They're very much made to be NAS cases and I think aren't great for anything but a fairly low-power CPU, because they're just so tight and small. More expensive than a Node 304, but even smaller. Only 5 drives though.
|
|
#
?
Dec 14, 2022 17:55
|
|
|
Corb3t posted:I always forget when I'm posting in the self hosting thread or NAS thread, its funny how different of a response you get when discussing exposing self hosting services to the internet. What you don't want is to expose some service with its own login method, because those are much easier for people to break than the authentication methods above, and most httpds will have had their authentication methods implemented and well-tested for decades.
|
|
#
?
Dec 14, 2022 17:57
|
|
|
Klyith posted:If you want ultra-compact there's the Jonesbo N1, or upcoming N2. They're very much made to be NAS cases and I think aren't great for anything but a fairly low-power CPU, because they're just so tight and small. Oooh, that JONSBO N1 looks great, thanks for the tip! I'm targeting a 45W TDP at most here in general, because the easiest way to get quiet is to make less heat, and modern processors are insanely efficient and perform really well with a lowered power target anyway. They're almost certainly coming too late for me, but where the hell are modern atom-based processors anyway? The Gracemont cores in Alder Lake would be amazing on their own!
|
|
#
?
Dec 14, 2022 18:03
|
|
|
Corb3t posted:I always forget when I'm posting in the self hosting thread or NAS thread, its funny how different of a response you get when discussing exposing self hosting services to the internet. The thing is ... all of these programs have bugs. All of them have vulnerabilities. Sometimes a vulnerability is discovered by a good guy and a patch is released. Sometimes (most of the time) whoever finds it, shuts up and exploits it. It's not the bugs and vulnerabilities that you know of that should be keeping you up at night, it's the ones you don't know of that are the problem. But, like you said, you do you. If that convenience is that important to you, then ... have at it. Also, they seem to be running in containers. That's just a fancy chroot, a level above just running on the host, but not by much. A VM would be quite a bit better than that at isolating a service in its own world (still with access to the network, but ... oh well).
|
|
#
?
Dec 14, 2022 18:51
|
|
|
Volguus posted:The thing is ... all of these programs have bugs. All of them have vulnerabilities. Sometimes a vulnerability is discovered by a good guy and a patch is released. Sometimes (most of the time) whoever finds it, shuts up and exploits it. Totally fair and valid, there are certainly ways I could improve the security of many of these exposed services with something like LLDAP/Authelia/Authentik, which I've considered doing. If somebody nefariously figures out a way to wipe out my media share, it would suck re-downloading everything but it wouldn't be the end of the world.
|
|
#
?
Dec 14, 2022 19:33
|
|
|
Volguus posted:The thing is ... all of these programs have bugs. All of them have vulnerabilities. Sometimes a vulnerability is discovered by a good guy and a patch is released. Sometimes (most of the time) whoever finds it, shuts up and exploits it. At a certain point, a lot of these apps are webservers running a small, lightly interactive website. 99% of the stuff that's unique to plex is on the other side of the webserver. So mostly your attack surface is apache or nginx -- some of the most inspected programs on the planet -- and the quite small chunk of web code that interacts with the backend. This is like a pretty well-defined security problem in 2022. Thus Plex has had 6 CVEs, ever, all minor. So yeah I would feel ok exposing that type of service to the wider internet, which requires both a vuln in the app or apache, plus a vuln in docker to fully exploit. Because on the day that a bad guy finds a new zero day vulnerability for those, I think my NAS box is pretty low on the list of targets. (edit: however, I don't know what 3/4ths of those services in Corb3t's pic are, and how many of them follow that model.)
|
|
#
?
Dec 14, 2022 19:51
|
|
|
I'd probably stick all the online-facing services in a VM or something. I'm pretty sure Docker does not provide any security guarantees
|
|
#
?
Dec 14, 2022 19:57
|
|
|
Corb3t posted:I always forget when I'm posting in the self hosting thread or NAS thread, its funny how different of a response you get when discussing exposing self hosting services to the internet. What sort of connection do you have now where a VPN is an issue? I have gigabit fiber now but not too long ago I had a pretty standard cable connection (albeit with no data cap) with Wireguard running on my Unraid box and don't recall any issues with just leaving it constantly enabled on my phone. Only time it's been a problem are times where my house lost power and I wasn't home and was suddenly confused about why my phone couldn't load any web pages. It's also a bonus when connected to public or BYOD networks that have content filters. But yeah, Cloudflare+Reverse Proxy are how I run Plex and Overseer. You could also enable Cloudflare's Zero Trust authentication for any applications where you don't trust the built-auth (I don't know that I'd ever expose anything that didn't have either MFA or Plex SSO). Cloudflare also has their own "Tunnels" you can set up which to my understanding is sort of like offloading the Reverse Proxy portion of the infrastructure to Cloudflare directly. But the convenience of the built-in Wireguard functionality in Unraid means that I don't think I'd ever expose anything that wasn't intended for someone other than myself to access.
|
|
#
?
Dec 14, 2022 20:10
|
|
|
The only things I expose are Plex and Overseerr (Plex SSO) via Cloudflare as well and that's for external users. Everything else is just for my access and one other user and this uses Wireguard which is also built in to unraid.
|
|
#
?
Dec 14, 2022 20:35
|
|
|
If I really want to get something unstuck from Radarr or whatever, I just VPN.
|
|
#
?
Dec 14, 2022 21:01
|
|
|
VostokProgram posted:I'd probably stick all the online-facing services in a VM or something. I'm pretty sure Docker does not provide any security guarantees I'm told by people whose opinions I trust that it is possible to lock down Linux containers (either docker, kubernetes, openvz, lxc, or something else), but it's non-trivial and is hard to get right - at least based on the experiences of the people who've done it and told me about it. Klyith is right, apache and nginx as a reverse proxy in front of everything is the way to go, even if you're not using HTTP(S) authentication like I suggested. If you're not doing anything fancy with interpreted code (like CGI, PHP, or any number of other things), the code in either of those has been battle-tested in production for decades - with nginx having a particularly good record. BlankSystemDaemon fucked around with this message at 21:17 on Dec 14, 2022 |
|
#
?
Dec 14, 2022 21:09
|
|
|
Does anyone pronounce nginx properly in their heads? Like I've definitely never heard anyone say it out loud before but I read it should be "engine-x" but that's nowhere near as satisfying as saying "en-jinx" in my head.
|
|
#
?
Dec 14, 2022 22:06
|
|
|
i've only ever heard engine x and that's how it looks in my head as well.
|
|
#
?
Dec 14, 2022 22:15
|
|
|
|
| # ? May 16, 2024 17:46 |
|
|
tuyop posted:Does anyone pronounce nginx properly in their heads? Like I've definitely never heard anyone say it out loud before but I read it should be "engine-x" but that's nowhere near as satisfying as saying "en-jinx" in my head. nuhginks
|
|
#
?
Dec 14, 2022 22:21
|
|