|
Yeah that's a weird oddity of using the baby Xeons on Supermicros even going back to some Haswell/Broadwell x10 SKUs, maybe before. They presume its a low-end vis workstation so it's always going to have a GPU. Not having to validate Intel's always-fluxuating GPU implementations helps the bottom line on those boards, too, I'm sure.
|
#
?
Jan 19, 2022 21:43
|
|
|
|
| # ? May 25, 2024 17:39 |
|
|
So I have two identical SSDs in a Linux machine and was thinking of using a union filesystem like mergerfs to combine them into one directory structure. I am going to periodically back up the contents to a drive in another machine. For my application, read speed is far more important than write speed - I will be reading far, far more than writing (I have it set up as a searchable document repository for books and scientific papers). I am also looking at other ideas. I have an HDD with the same capacity as one of the SSDs. I was also thinking I could do RAID 4 with the HDD as the parity drive. It would probably murder write speeds - limiting them with the speed of the HDD - but those are not that important to me. It would be cheaper to fix with not a lot of downtime in the event that the parity drive is the one that fails. Since the HDD doesn't contain non-parity data the read speeds should still be fast. Since the parity writes would be only to the HDD it would reduce SSD wear compared to RAID 5 with all SSDs (maybe?). I only have experience with RAID 5 and have never really thought about RAID 4 before today. Is there something I don't know that makes this a bad idea? edit: Actually for that matter, is there something that makes RAID a bad idea for SSDs? BattleMaster fucked around with this message at 08:13 on Jan 20, 2022 |
|
#
?
Jan 20, 2022 08:09
|
|
|
I think I'd rather not setup a RAID5 on QLC, if you're running (well, trying to) 4k random write workloads with any intensity. RAID5 is already pretty bad at it and depending on the stripe size either the QLC will be hit with performance degradation too, or RAID will get even worse. Other than that, I don't think there's anything inherently bad about it for NAND, especially if you don't care about the write performance that much.
|
|
#
?
Jan 20, 2022 10:18
|
|
|
Remember that RAID5 is still subject to write holes (well, most implementations - apparently RSTe doesn't have that problem somehow). May I also point out that with ZFS, you can have the equivalent of raid5 without write holes, and you get the benefit of the ARC which can make read-speeds for the most frequently and most recently accessed records as fast as several hundred GB/sec (depending on memory burst speed, since you're reading from memory)?
|
|
#
?
Jan 20, 2022 17:33
|
|
|
That Works posted:Yeah read through the guide and bought a 256 nvme and a heatsink, 8 gb of RAM and the HP S-01 just a bit ago. Oh hey, I got an S01 to be a new Plex server just a couple weeks ago and then immediately added an i5-10400 and 16GB of RAM because I couldn't resist tinkering and figured it could be a container sandbox as well. Hopefully you already knew but for anyone else looking at the S01, be aware that the optical drive bracket folds down on top of the RAM slots and only gives a few mm of clearance past standard PCB height. I wasn't thinking about this and got RAM with heatsinks which I then had to remove. Eletriarnation fucked around with this message at 17:42 on Jan 20, 2022 |
|
#
?
Jan 20, 2022 17:39
|
|
Eletriarnation posted:Oh hey, I got an S01 to be a new Plex server just a couple weeks ago and then immediately added an i5-10400 and 16GB of RAM because I couldn't resist tinkering and figured it could be a container sandbox as well. Thanks! Good to know, fortunately I got some pretty vanilla stuff without a HS. Hopefully it works OK. If not I'll just remove the optical drive entirely and cock up something to cover the open slot cover. What else you thinking about putting on it?
|
|
|
#
?
Jan 20, 2022 17:42
|
|
|
That Works posted:Thanks! Good to know, fortunately I got some pretty vanilla stuff without a HS. Hopefully it works OK. If not I'll just remove the optical drive entirely and cock up something to cover the open slot cover. For the OS I used Fedora because I didn't want to jump through too many hoops to make Plex work and I'm already familiar with RHEL derivatives. As far as other applications I just have it running a Unifi controller and MotionEye so far. Those would have been fine on the stock G5905/4GB setup, but I wanted some room to add more and play around with Vagrant/Ansible some. I was also seeing some 1080p streams take 80% of a core on the Celeron even when they claimed to use hardware transcoding. This made me question whether QuickSync was being used properly but IDK what else to do if it's showing the expected (hw) tag in the Dashboard, so I figured more cores would be a good way to ensure it wouldn't choke on 3+ streams at once.
|
|
#
?
Jan 20, 2022 18:04
|
|
|
Weird ZFS thing I'm trying to figure out: I moved a RAIDZ we'll call src into a new NAS with new drives to migrate the data to, so I made a new zpool we'll call dest and made a fresh snapshot on src called evac then rancode:e: said vdev when I meant zpool Munkeymon fucked around with this message at 18:45 on Jan 20, 2022 |
|
#
?
Jan 20, 2022 18:24
|
|
|
i think you need to run du -A -sh to get the true sizes, if the old zfs didn't have compression turned on but the new one had lz4 by default for instance. that's the only reason i can think of why there would be a difference if zfs send/receive completes without errors.
|
|
#
?
Jan 20, 2022 18:47
|
|
|
Having trouble finding the Node 804 case which I was going to move my existing 8700k machine in to take over for my existing 2500k unraid box, does anyone recommend other good NAS cases these days? Preferably something compact like the 804 (mATX) with plenty of drive space of course.
|
|
#
?
Jan 20, 2022 19:00
|
|
|
r u ready to WALK posted:i think you need to run du -A -sh to get the true sizes, if the old zfs didn't have compression turned on but the new one had lz4 by default for instance. The ls -R output is different, too, with the original having ~5000 files the new one doesn't.
|
|
#
?
Jan 20, 2022 19:39
|
|
|
no idea then, maybe transfer data with rsync instead and see if it throws error while reading certain files?
|
|
#
?
Jan 20, 2022 19:52
|
|
|
r u ready to WALK posted:i think you need to run du -A -sh to get the true sizes, if the old zfs didn't have compression turned on but the new one had lz4 by default for instance. Munkeymon posted:The ls -R output is different, too, with the original having ~5000 files the new one doesn't.
|
|
#
?
Jan 20, 2022 19:57
|
|
|
Speaking of those SP01s they are at a good price. https://www.ebay.com/itm/HP-S01-pf1...1-127632-2357-0
|
|
|
#
?
Jan 20, 2022 23:04
|
|
|
Brand new with a core i5 they are $650, I wonder if the cost difference of the used one is worth it.
|
|
#
?
Jan 20, 2022 23:49
|
|
|
Might be possible to put in a replacement CPU although HP might BIOS lock out other replacements in the Comet Lake line. I’m considering one myself and it seems slower than my i3-4130 that it would replace. Then again the S01 uses roughly 7w idle which is pretty impressive and is likely better than my NUC + Thunderbolt eGPU chassis overpriced abomination of a home NAS. Just not sure if I can get a 2.5 GbE to SFP+ connect going or if I would have to wire up the m.2 slot to support a 10 GbE NIC. Might all be a wash.
|
|
#
?
Jan 21, 2022 02:00
|
|
|
Would I be dumb to make a router / firewall out of an sp01?
|
|
|
#
?
Jan 21, 2022 02:13
|
|
|
I realize 10gbe is wayyyyyy overkill for all home environments but are people really buying into this 2.5gbps bullshit!? Enough to bother spending money on it!?
|
|
#
?
Jan 21, 2022 04:44
|
|
|
Crunchy Black posted:I realize 10gbe is wayyyyyy overkill for all home environments but are people really buying into this 2.5gbps bullshit!? Enough to bother spending money on it!? I just aggregated two 1gbit lines between the two switches I have in the house. I pulled a third cable to use later if I need it but I haven’t bottlenecked the dual gig yet.
|
|
#
?
Jan 21, 2022 05:33
|
|
|
Crunchy Black posted:I realize 10gbe is wayyyyyy overkill for all home environments but are people really buying into this 2.5gbps bullshit!? Enough to bother spending money on it!? I bought 2 $25 2.5gbe NICs to link my unraid server and my desktop together with a cat5e cable since it was far enough that fiber wouldn't work and it's lovely. The whole house? Nah not worth it.
|
|
#
?
Jan 21, 2022 06:36
|
|
|
BlankSystemDaemon posted:Are there some similarities shared between the files that didn't get moved over? After some digging, I'm wondering if the last time I moved this data, I got "clever" and made a clone instead of a snapshot. After reading the docs I can find, I'm still not sure if making a clone, writing to it and then making a snapshot on the pool would ignore/bypass the clone.
|
|
#
?
Jan 21, 2022 07:35
|
|
|
Minty Swagger posted:it was far enough that fiber wouldn't work
|
|
#
?
Jan 21, 2022 08:05
|
|
|
Once I remove a hard drive's letter in Windows so I can add it to Stablebit Drivepool, how do I access the drive later on in Windows if I ever want to? Also how do I find out the new "name" of the drive? I've been using Windows since 1994 and not using drive letters is going to take a lot to get used to!
|
|
#
?
Jan 21, 2022 15:20
|
|
|
Crunchy Black posted:I realize 10gbe is wayyyyyy overkill for all home environments but are people really buying into this 2.5gbps bullshit!? Enough to bother spending money on it!?
|
|
#
?
Jan 21, 2022 15:45
|
|
|
Chumbawumba4ever97 posted:Once I remove a hard drive's letter in Windows so I can add it to Stablebit Drivepool, how do I access the drive later on in Windows if I ever want to? Also how do I find out the new "name" of the drive? Open Disk Management (also called 'Create and format hard disk partitions'), right click the volume you want to access, choose 'Change drive Letter and Paths', click 'Add' and then mount the drive in an NTFS folder. For example, all of my Stablebit drives are mounted in C:\DrivePool\Drives
|
|
#
?
Jan 21, 2022 17:09
|
|
|
bobfather posted:Open Disk Management (also called 'Create and format hard disk partitions'), right click the volume you want to access, choose 'Change drive Letter and Paths', click 'Add' and then mount the drive in an NTFS folder. For example, all of my Stablebit drives are mounted in C:\DrivePool\Drives Wow, this is amazing and I never knew it was a thing! Thank you! Is there any way to get the drive to show up in Explorer with all the other drives?
|
|
#
?
Jan 21, 2022 17:23
|
|
|
Chumbawumba4ever97 posted:Wow, this is amazing and I never knew it was a thing! Thank you! I think the individual drives in a pool can be assigned drive letters. I disabled them because you should not really be writing to the individual pool drives themselves - only the amalgamated pool drive that StableBit presents to Windows.
|
|
#
?
Jan 21, 2022 18:03
|
|
|
That Works posted:Would I be dumb to make a router / firewall out of an sp01? Here are some numbers on some different gear. Crunchy Black posted:I realize 10gbe is wayyyyyy overkill for all home environments but are people really buying into this 2.5gbps bullshit!? Enough to bother spending money on it!? Not that I know why one wouldn't have gotten OM2 or OM3 wired if one has done network cable installation in the last 5 years. e.pilot posted:I just aggregated two 1gbit lines between the two switches I have in the house. I pulled a third cable to use later if I need it but I haven’t bottlenecked the dual gig yet. Minty Swagger posted:I bought 2 $25 2.5gbe NICs to link my unraid server and my desktop together with a cat5e cable since it was far enough that fiber wouldn't work and it's lovely. All RJ45 tops out at ~100m. Munkeymon posted:After some digging, I'm wondering if the last time I moved this data, I got "clever" and made a clone instead of a snapshot. After reading the docs I can find, I'm still not sure if making a clone, writing to it and then making a snapshot on the pool would ignore/bypass the clone. Theoretically the way it'd work is that until you write to it, the clone refers back to the same blocks that are in the dataset that was snapshot'd. Also, I forgot to mention this the last time, but you might wanna look at zfs-diff(8) too.
|
|
#
?
Jan 21, 2022 18:27
|
|
|
BlankSystemDaemon posted:All RJ45 tops out at ~100m. * Except for that hateful ethernet/vdsl train wreck I used to have to work on called Cisco LRE
|
|
#
?
Jan 21, 2022 18:41
|
|
|
The big costs for OM2/OM3 in residential aren't the wiring itself (on a per-foot basis fiber is barely more expensive but basically negligible in the grand scheme of a new construction build) but in the media converters for the consumer devices that are still on RJ45 based connectors as well as finding someone that knows wtf to do with the fiber (it's not hard but there's a severe drought in people that know how to install even CAT5e without messing something up like stapling right through the jacket). Even with Xeons you have to get to datacenter-ish product lines to even be offered an SFP+ connection at least which drastically changes connectivity options for anyone outside a server rack deployment. I say all this as someone that's looked carefully at build costs for a custom and semi-custom home where I'd do the wiring and cabling or with a friend that is a professional commercial installer. It kinda pisses me off in a way that we've had super dang fast connectivity for well over a decade but wifi is ultimately king and the market for consumer connectivity. We don't really have any killer app that demands high bandwidth and ultra low latency except maybe AR / VR and there's no such thing as a "must-have AR / VR" app when half our country is on the brink of abject poverty and is content with f2p mobile games.
|
|
#
?
Jan 21, 2022 21:36
|
|
BlankSystemDaemon posted:Well, it depends on your bandwidth and distribution of traffic (ie. how much of it is large packets vs small packets, but a couple cores at +3GHz with FreeBSD is definitely enough to move a good chunk of bits. I have a basic FIOS connection to the house, 1 GB if I recall. I am only running a very limited home network with a few wired devices including my Nas and two omada APs. I have no idea of packet size distribution. My only use is at home are some Usenet downloads, a couple of streaming services, VoIP and some gaming stuff. Pretty lightweight. I only asked about building my own because I have the time the hardware and kind of want to learn more. From reading on the serverbuilds site plenty of people use those machines as pfsense boxes with an Intel quad nic. I was just wondering if they would be enough to run as a firewall and router along with a VPN endpoint.
|
|
|
#
?
Jan 21, 2022 22:08
|
|
|
necrobobsledder posted:The big costs for OM2/OM3 in residential aren't the wiring itself (on a per-foot basis fiber is barely more expensive but basically negligible in the grand scheme of a new construction build) but in the media converters for the consumer devices that are still on RJ45 based connectors as well as finding someone that knows wtf to do with the fiber (it's not hard but there's a severe drought in people that know how to install even CAT5e without messing something up like stapling right through the jacket).
|
|
#
?
Jan 21, 2022 22:25
|
|
|
Economies of scale. Production deployments will use large volumes of SFPs / QSFPs, whereas media converters are typically just a band-aid fix that will get deployed in small numbers only where a better solution isn't possible.
|
|
#
?
Jan 21, 2022 22:38
|
|
|
CopperHound posted:I am so confused. BlankSystemDaemon posted:What do you mean fiber wouldn't work because of distance? Single-mode fiber can carry 10Gbps over 10km with OS1, and multi-mode 10Gbps is 300 meters with OM3. Ha sorry, I said fiber but I meant SFP+ which from what I understood seems to top out at 10 meters for a line (which I needed about 12) and cant be bent as aggressively as a cat5e cable can. Rather than going deluxe with a more powerful solution I just opted to bump up to 2.5 since it was cheap and I could use existing cables. 50 bucks to boost the connection between the two devices I'd actually benefit from going faster on my network made it seem pretty okay in terms of a choice to make.
|
|
#
?
Jan 21, 2022 23:02
|
|
|
Combat Pretzel posted:Interesting that media converters are apparently an expensive item (I have no idea about actual pricing), especially after loosely looking for QSFP transceivers for my 40GBe cards, and being surprised about them only costing like 40 bux a piece. You'd think some RJ45 to fiber transceiver shouldn't cost appreciably more than that. That Works posted:I only asked about building my own because I have the time the hardware and kind of want to learn more. From reading on the serverbuilds site plenty of people use those machines as pfsense boxes with an Intel quad nic. I was just wondering if they would be enough to run as a firewall and router along with a VPN endpoint. You can run pfSense within your own boxed off subnet and go hog wild with VMs and containers and whatnot if you want to learn. For keeping peace in a home with lots of heavily used devices I'm sticking with more stuff someone's paid to maintain.
|
|
#
?
Jan 21, 2022 23:10
|
|
|
Minty Swagger posted:Ha sorry, I said fiber but I meant SFP+ which from what I understood seems to top out at 10 meters for a line (which I needed about 12) and cant be bent as aggressively as a cat5e cable can. Rather than going deluxe with a more powerful solution I just opted to bump up to 2.5 since it was cheap and I could use existing cables. 50 bucks to boost the connection between the two devices I'd actually benefit from going faster on my network made it seem pretty okay in terms of a choice to make. What you're describing is a direct-attached cable which may be fiber or coaxial or whatever the hell else the vendor felt like, hardwired to an SFP module at each end. Those do typically have a fairly short maximum range and are somewhat delicate. They aren't a good choice for cabling installed in a building (instead of, say, between racks of servers in a room, or within a single rack). So, yeah, in your scenario the correct choice would be some form of SFP transceiver at each end (whether to fiber or RJ45) and a matching cable of the appropriate length. Or if you were able to get 2.5G NICs that easily, just what you did.
|
|
#
?
Jan 21, 2022 23:13
|
|
necrobobsledder posted:Depending upon the quality and capabilities of the converter they run anywhere between $50 to $200 each. And using just a single one can wipe out the power savings benefits of using fiber over copper. I do think it would be pretty interesting if consumer and prosumer devices started shipping with QSFP and SFP+ receptacles instead of an RJ45 port with accompanying NIC users might complain about, but it really is a pain for OEMs given essentially most consumer device SOCs are including NICs on them and it would be an extra cost rather than cost savings even if they gave up and increased the power envelope another 7w to accommodate 10GbE. I'm an old man yelling at wifi for 2 decades now but it was mostly because I knew it would likely win and half our drat consumer networking applications and advances would be gated by its progress, not because I'm some fetishist of cabled interconnects. What's your off the shelf solution for a small home network out of curiosity? If I were to go for off the shelf given the 2 omada APs it would probably be a TPlink ER7206 VPN router and the omada controller which would run about $250 for all. I was thinking if I built that HP S-01 and an Intel nic it would come in a bit cheaper. I guess I am surprised that the hardware would be that much quicker. Thank you for your answers on this, this is well outside my current knowledge. I've been spending more of my free time the last few months tinkering around with learning more about home networking as well as working with other open source stuff.
|
|
|
#
?
Jan 22, 2022 00:04
|
|
|
Motronic posted:Cisco LRE necrobobsledder posted:The big costs for OM2/OM3 in residential aren't the wiring itself (on a per-foot basis fiber is barely more expensive but basically negligible in the grand scheme of a new construction build) but in the media converters for the consumer devices that are still on RJ45 based connectors as well as finding someone that knows wtf to do with the fiber (it's not hard but there's a severe drought in people that know how to install even CAT5e without messing something up like stapling right through the jacket). Even with Xeons you have to get to datacenter-ish product lines to even be offered an SFP+ connection at least which drastically changes connectivity options for anyone outside a server rack deployment. I say all this as someone that's looked carefully at build costs for a custom and semi-custom home where I'd do the wiring and cabling or with a friend that is a professional commercial installer. Also, what do you mean finding someone who knows what to do? It's a lucent connector, you don't have to do anything other than pop off the dust covers and insert it - as an example, here's some from fiberstore. If the contractors you're hiring or friends/acquaintances you have staple through cables, then I think maybe there's a large risk of foundational issues with the entire building even before it's done. 802.11 has always, and will always, be about convenience rather than bandwidth, latency, or stability - and those tend to be the thing that're the focus of this thread, so I'm not sure why you mention it. That Works posted:I have a basic FIOS connection to the house, 1 GB if I recall. I am only running a very limited home network with a few wired devices including my Nas and two omada APs. I have no idea of packet size distribution. My only use is at home are some Usenet downloads, a couple of streaming services, VoIP and some gaming stuff. Pretty lightweight. I only asked about building my own because I have the time the hardware and kind of want to learn more. From reading on the serverbuilds site plenty of people use those machines as pfsense boxes with an Intel quad nic. I was just wondering if they would be enough to run as a firewall and router along with a VPN endpoint. If your aim is to do bi-directional 1/1Gbps (ie. 2Gbps total) for that distribution, you're looking at handling around 700kpps and I'm pretty sure ipfw can handle that statefully on the kind of CPU you're looking at. I can't speak to pf since I don't use it, but Kristof Provost (who is FreeBSDs pf maintainer) has been working on making it as fast as ipfw, so I think that might be an option too (at least on FreeBSD, OpenBSDs pf implementation still has a lot pessimizations that impact performance). BlankSystemDaemon fucked around with this message at 00:40 on Jan 22, 2022 |
|
#
?
Jan 22, 2022 00:23
|
|
|
That Works posted:What's your off the shelf solution for a small home network out of curiosity? If I were to go for off the shelf given the 2 omada APs it would probably be a TPlink ER7206 VPN router and the omada controller which would run about $250 for all. The HP S-01 in the above listing is a Celeron G5900 which is straight up worse than my old Haswell i3 in terms of compute. But if dedicated for network switching it'll be perfectly serviceable. People used to run old PII boxes with pfSense pushing maybe 1 GB / s in total packet throughput, so that G5900 would likely push several times higher and all for under 12w with a 4 port NIC, which is really good for a general CPU based solution. The big point is that a dedicated piece of hardware for network will destroy an equivalent price point general purpose device in terms of power and throughput efficiency because the general purpose device is by definition less efficient for specific workload characteristics. But if you want to do things like add PiHole, mess with IDS, layer 7 firewalls, etc. the Celeron will be plenty fine for a master network box and perhaps less annoying to deal with than a Raspberry Pi or similar SBC. I actually bought one of those HP boxes but as a dedicated NAS head unit until I manage to transplant my workstation as the successor. It kinda sucks to lose IPMI but for $110 for it all I kinda can't complain either (and also there's RPi KVM options now that could suffice). E: BlankSystemDaemon posted:X520 NICs can be had for very cheap on ebay, craigslist, various server refurbishing shops, and/or all their local variants - and fiberstore (fs.com) does validated SFP+ modules for cheap (~20 bux) in both the US and Europe. What I mean in terms of contractor reliability is that around high cost of living areas like mine most of those that have even seen a Lucent connector aren't working residential construction anymore such as my friend, and then there's factors like international building code that many neglect or charge extra for such as using improperly rated cables for in-wall and not including a separate line to make a future cable runner's life easier - almost nobody does that in residential. Also, because of how contractors and subcontractors work even with inspectors involved you could have a perfectly structurally excellent home but completely messed up HVAC, plumbing, and electrical. Because I'm an insane person and essentially wanting to run quasi-datacenter spec comm lines in a freakin' house, a professional that does such work (he literally has wired major datacenters for Big Tech and telecoms) is who I'd hopefully hire anyway and my friend (who I'll gladly pay his $150 / hr rate) is a better option in terms of trust for me anyway. While some good folks do decent cabling for better rates, I'm willing to pay some extra to not have more inconveniences lying in my walls for 20+ years when I had a chance to do it right in the beginning. necrobobsledder fucked around with this message at 01:24 on Jan 22, 2022 |
|
#
?
Jan 22, 2022 00:52
|
|
|
|
| # ? May 25, 2024 17:39 |
|
|
BlankSystemDaemon posted:X520 NICs can be had for very cheap Have you used them on TrueNAS/BSD and VMWare ESXi or have any idea if I'd run into issues? I've kinda been thinking about putting to 10GbE between those boxes, but I'm very much on the enterprise (read: I'm not paying for the poo poo I buy at work for a rack in my basement) side of this. Obviously I can do real research on this myself, but heh.....let's throw it here first and see. I do have 2 SFPs on my core switch that I could use but realistically I'd rather just throw a DAC between the two boxes. If I get dual port cards then maybe I'd DAC it to the switch as well. Or buy some reasonable cable management and just use the SFPs and jumpers I already have here "leftover" from work stuff. But I'm pretty sure all I have is single mode which seems......excessive. (I work in telecom, so...yeah)
|
|
#
?
Jan 22, 2022 01:10
|
|