|
WWWWWWWWWWWWWWWWWW posted:The 8TB WD Elements, EasyStore, and MyBooks are all shuckable alike, correct? I just noticed I have three type of drives. E: I guess technically you can mount the encrypted drive with extra steps. https://github.com/themaddoctor/linux-mybook-tools CopperHound fucked around with this message at 19:41 on Jan 4, 2020 |
# ? Jan 4, 2020 19:32 |
|
|
# ? May 25, 2024 15:22 |
|
CopperHound posted:MyBook has a terrible implementation of drive encryption. It is shuckable, but not if you need to retain the data on the drive. Oh wow, what does that mean exactly? As far as I know I don't have any encryption enabled. Is that only something I need to worry about if I use BitLocker or something?
|
# ? Jan 4, 2020 19:40 |
|
WWWWWWWWWWWWWWWWWW posted:Oh wow, what does that mean exactly? As far as I know I don't have any encryption enabled. Is that only something I need to worry about if I use BitLocker or something?
|
# ? Jan 4, 2020 19:51 |
|
If I understand correctly, the enclosure is encrypting data written to the disk. I would assume everyone is buying these and immediately shucking them without even using them in their enclosure so in that case the encryption doesn't even matter...right?
|
# ? Jan 4, 2020 21:24 |
Thermopyle posted:If I understand correctly, the enclosure is encrypting data written to the disk.
|
|
# ? Jan 4, 2020 21:27 |
|
Regarding affordable NAS back-ups: Has anybody else looked into a Google Enterprise account with unlimited Google Drive storage? It's only $12 a month and lots of people over on Reddit have backed up their NASes using rclone. Hell, a lot of people even setup a Plex server with content hosted on Google Drive Enterprise (encrypted, of course). Supposedly Google won't start throttling your usage until you hit 150 TB of space used.
|
# ? Jan 4, 2020 21:37 |
|
Got the FreeNAS box moved onto the 10GB fiber, works great, other than the FreeNAS box started freaking out and rebooting, but I think it was due to a crappy PCIe USB 3.0 card.
|
# ? Jan 4, 2020 21:45 |
|
Gay Retard posted:Regarding affordable NAS back-ups: Has anybody else looked into a Google Enterprise account with unlimited Google Drive storage? It's only $12 a month and lots of people over on Reddit have backed up their NASes using rclone. Hell, a lot of people even setup a Plex server with content hosted on Google Drive Enterprise (encrypted, of course). Supposedly Google won't start throttling your usage until you hit 150 TB of space used. Yes that's what I use for backup. duplicacy with local encryption and then distinct folders per machine. Then for some content a second copy via rclone but they throttle iops there so saving billions of small files won't work very well.
|
# ? Jan 4, 2020 21:51 |
|
D. Ebdrup posted:The operative word, because it's doing a lot of work, in that sentence being 'can' - it only happens under scenarios which require careful preparation, such as ZFS dRAID that I mentioned before where meta-data is stored on non-volatile flash and data is written in 10MB blocks to the disks to make use of sequential I/O. I was referring to the best-case scenario, sequential transfers with large files, because the guy we're advising is using his drives mainly for multimedia storage. He's going to, infrequently, transfer multi-GB files and take advantage of the fastest speeds his drives can offer (while hitting any relevant interface bottlenecks along the way,) but then most of the time not need much bandwidth to access individual files being streamed, hence why I said that any of the bottlenecks/limitations won't really be an issue for his use-case. WWWWWWWWWWWWWWWWWW posted:The 8TB WD Elements, EasyStore, and MyBooks are all shuckable alike, correct? I just noticed I have three type of drives. "Shuckable" means you can open an external enclosure and extract the drive within to repurpose it elsewhere, and most of the time it's possible. The only exceptions I know of are some "portable" HDDs (e.g. specific WD models) that contain a 2.5" drive that has a custom USB connector rather than a standard SATA one. Thermopyle posted:If I understand correctly, the enclosure is encrypting data written to the disk. D. Ebdrup posted:I don't think this applies, but there are schemes like OPAL where the disk itself AES-encrypts stuff written to it. Yes to all of that; there are enclosures whose controllers automatically encrypt, but that generally doesn't matter because the people who just buy them for the internal drives are immediately shucking them.
|
# ? Jan 4, 2020 23:39 |
Atomizer posted:I was referring to the best-case scenario, sequential transfers with large files, because the guy we're advising is using his drives mainly for multimedia storage. He's going to, infrequently, transfer multi-GB files and take advantage of the fastest speeds his drives can offer (while hitting any relevant interface bottlenecks along the way,) but then most of the time not need much bandwidth to access individual files being streamed, hence why I said that any of the bottlenecks/limitations won't really be an issue for his use-case. OPAL is a harddrive-spec, has nothing to do with enclosures. Generally speaking I find that USB to SATA controllers are built down to a price, which precludes doing any hardware encryption even by including ASIC circuitry for the purpose.
|
|
# ? Jan 4, 2020 23:51 |
|
CopperHound posted:My book drives have hardware encryption by default. If you format the drive when you plug it in it will behave just like any other drive. Otherwise you will not be able to directly read the drive without the enclosure unless you do some of the stuff mentioned in the above link. I doubt I formatted it when I first got it. So what do I need to do? Dump all the data off it to another 8TB drive, then shuck the drive, then install it in my desktop and transfer the files back?
|
# ? Jan 5, 2020 04:40 |
|
WWWWWWWWWWWWWWWWWW posted:I doubt I formatted it when I first got it. So what do I need to do? Dump all the data off it to another 8TB drive, then shuck the drive, then install it in my desktop and transfer the files back? That would be the easiest solution, though you'll need to add "reformat" between "install in desktop" and "transfer files back".
|
# ? Jan 5, 2020 04:52 |
|
IOwnCalculus posted:That would be the easiest solution, though you'll need to add "reformat" between "install in desktop" and "transfer files back". Thank you! Just formatting it inside of Windows disk management should be enough?
|
# ? Jan 5, 2020 06:15 |
|
D. Ebdrup posted:I don't trust best-case scenarios. USB to SATA controllers can easily include a hardware encryption block. They're most likely pad limited (*), and even if they aren't, an AES block is approximately no area in sub-90nm nodes. Which these have to be built on, since the SERDES and other IP cores required for USB3 and SATA aren't likely to be available in any node older than 65nm. * - Pad limited refers to a phenomenon where chips with relatively little functionality have the minimum size of the die set by the size and layout of the I/O pad structures rather than the amount of area required for the logic. You can add extra features into a pad limited chip without increasing the marginal cost of production so long as you don't fill up all the unused space.
|
# ? Jan 5, 2020 07:34 |
|
Gay Retard posted:Regarding affordable NAS back-ups: Has anybody else looked into a Google Enterprise account with unlimited Google Drive storage? It's only $12 a month and lots of people over on Reddit have backed up their NASes using rclone. Hell, a lot of people even setup a Plex server with content hosted on Google Drive Enterprise (encrypted, of course). Supposedly Google won't start throttling your usage until you hit 150 TB of space used. I do this. I have it set up to rclone my monthly VM backups, some videos that I may want to rewatch in the future, and I have an encrypted repository for sensitive documents. It took a while to get set up properly, but it's nice to have offsite backups of important stuff.
|
# ? Jan 5, 2020 16:34 |
|
Any opinions on whether a pair of E5-2603v4s are enough CPU for a medium (~20TB) FreeNAS box? I'm going to be building a new ESXi host and converting my old one into a standalone NAS. 12 cores at 1.7GHz has been marginal for virtualization, but I'm hoping it'll be enough grunt to manage storage.
|
# ? Jan 8, 2020 01:42 |
|
If that's not enough CPU, FreeNAS sucks.
|
# ? Jan 8, 2020 01:59 |
|
I mean, it's enough cores but even fileserver stuff responds well to better single-threaded performance. rsync, samba, or tar all have significant single-threaded portions and 1.7 GHz is not super speedy. I was running rsync today moving a bunch of music files off my desktop SSD onto my NAS, and at 1.2 gbit/s I was running 65% utilization on one thread of rsync and 25% utilization on one thread of ssh on my i3-7100 at 3.9 GHz (40% total processor utilization). Scaling against clockrates and factoring a 6% IPC lead, my procesor is 243% as fast yours, so rsync (65% on mine) would require 158% utilization on one thread of your processor, or you would be bottlenecked at something like 760 mbit/s of transfer per thread of rsync. zfs itself is fast, the things you do with it are not necessarily. (you could of course do more things in parallel if you have 10gbit or teamed gigabit, you do have 12 cores there, but you will often be limited by single-thread speed even on fileserver stuff. also it could be worse, these are 20-30 mb flac files, small mp3s or photos would be more cpu-intensive for rsync to manage, and this was moving them onto a virgin zfs dataset so that's as good as it gets as far as rsync overhead or zfs overhead. Samba overhead is even higher.) I am personally not a huge fan of some of the common homelab recommendations. They are loud, power-hungry, and limited in all the wrong ways. You can do very nice builds for home users on Ryzen or Skylake, or if you really want RAM capacity then do X99 with RDIMMs up to 768GB very cheaply. Your home NAS or homelab is not sitting there serving 100 users 24/7, mostly it's sitting there idle for 99% of the day and then for that last 1% you want it to run 1-3 tasks as fast as humanly possible. So a multi-socket, high-core-count, low-clocked, super-high-RAM server is not the thing you need (you don't even have the dual-ported (SAS) drives to do that if you wanted). You want a low-power system with 4-8 cores running at as high a clock as possible when loaded and idling as low as possible when not. And it'll be much quieter to boot. You can get a fair amount of RAM if you need, X99 will do 768GB on a single socket on an ASUS board, but unless you're doing tons of VMs or something the 64GB you get on consumer platforms will probably be OK. If you want to stay on that system I would think strongly about upgrading your CPUs. Paul MaudDib fucked around with this message at 03:54 on Jan 8, 2020 |
# ? Jan 8, 2020 02:15 |
|
fatman1683 posted:Any opinions on whether a pair of E5-2603v4s are enough CPU for a medium (~20TB) FreeNAS box? I'm going to be building a new ESXi host and converting my old one into a standalone NAS. 12 cores at 1.7GHz has been marginal for virtualization, but I'm hoping it'll be enough grunt to manage storage. More than enough. My media esxi server that has a 48tb freenas on it runs off just a e3-1230l v3.
|
# ? Jan 8, 2020 02:56 |
|
Paul MaudDib posted:
Thanks, I'm aware of the single-threaded elements of SMB and other such filestreaming tools, but I don't have any experience working with them on such a significantly CPU-constrained system. Right now, a RAID1 of WD Reds is my main bottleneck so I haven't hit any point at which having more CPU would matter, but when I expand out to a large array I'm anticipating that I'll hit that point rather quickly. I'm probably going to hold off on the upgrade until the next generation of Xeon Scalables drops and hope that the price comes down on v4 E5s. edit: Alternatively, I could try and pick up a used v2 system on the cheaps from one of the refurb houses and try and sell my v4 kit. Odds on getting a decent price for a pair of 2603v4s, an X10 motherboard and 64GB of DDR4? fatman1683 fucked around with this message at 03:03 on Jan 8, 2020 |
# ? Jan 8, 2020 02:59 |
|
fatman1683 posted:Thanks, I'm aware of the single-threaded elements of SMB and other such filestreaming tools, but I don't have any experience working with them on such a significantly CPU-constrained system. Right now, a RAID1 of WD Reds is my main bottleneck so I haven't hit any point at which having more CPU would matter, but when I expand out to a large array I'm anticipating that I'll hit that point rather quickly. I'm probably going to hold off on the upgrade until the next generation of Xeon Scalables drops and hope that the price comes down on v4 E5s. what about swapping your processors out for 2640v3s? they're $75 a pop on ebay and that gets you 2.8 to 3.4 GHz on 8 cores. Or you can get 2637v3 for $65 and that gets you 3.6 to 3.7 (but only 4 cores). Max of 768GB on all these. Broadwell doesn't seem like a super important generation to hold out for to me. Early 14nm was baaaadddd (this is not 14+++++++) and clocks on most of the higher core count parts are abysmal unless you go for the very tippy top SKUs which are probably never going to be cheap. Broadwell is a bit lower power than Haswell, it supports 1.5 TB vs 768GB on Haswell, and up to DDR4-2400 vs 2133 for Haswell, but it's way more expensive and for home users it doesn't seem like a big deal. here's your lists: https://en.wikipedia.org/wiki/List_of_Intel_Haswell-based_Xeon_microprocessors#Xeon_E5-26xx_v3_(dual-processor) https://en.wikipedia.org/wiki/List_of_Intel_Broadwell-based_Xeon_microprocessors#Xeon_E5-26xx_v4_(dual-processor) Paul MaudDib fucked around with this message at 03:24 on Jan 8, 2020 |
# ? Jan 8, 2020 03:13 |
|
Paul MaudDib posted:what about swapping your processors out for 2640v3s? they're $75 a pop on ebay and that gets you 2.8 to 3.4 GHz on 8 cores. Or you can get 2637v3 for $65 and that gets you 3.6 to 3.7 (but only 4 cores). Max of 768GB on all these. It's been awhile since I've bought hardware on ebay, I'm not sure I'd want to roll those dice necessarily, but one of the refurb places might have some v3 CPUs with a warranty of sorts. I'll look into it, thanks. e: Paul MaudDib posted:Broadwell doesn't seem like a super important generation to hold out for to me. Early 14nm was baaaadddd (this is not 14+++++++) and clocks on most of the higher core count parts are abysmal unless you go for the very tippy top SKUs which are probably never going to be cheap. It is a bit lower power than Haswell, it supports 1.5 TB vs 768GB on Haswell, but it's way more expensive and for home users it doesn't seem like a big deal. Yeah, noted. At the time I wanted to get whatever was current in the hopes of keeping it for as long as possible, and prices on Haswell hadn't started to drop yet. It's worked well enough for my purposes so far, but I definitely need to revisit it for this next round of upgrades. fatman1683 fucked around with this message at 03:24 on Jan 8, 2020 |
# ? Jan 8, 2020 03:16 |
|
fatman1683 posted:It's been awhile since I've bought hardware on ebay, I'm not sure I'd want to roll those dice necessarily, but one of the refurb places might have some v3 CPUs with a warranty of sorts. I'll look into it, thanks. Other than issues endemic to the hardware across the entire fleet, I've purchased a LGA1366 NAS setup, a LGA2011 setup because I was chasing the hotness, and then a different LGA2011 board because of the mentioned endemic issues across I believe every single motherboard of that SKU, I've had zero issues whatsoever with any ebay hardware. It's a dice roll yes, but I'd bet it goes bad far less than you think.
|
# ? Jan 8, 2020 03:34 |
|
And if it dies immediately, PayPal buyer protection is always there to gently caress the seller. I bought a server that ended up having 2 dead ram slots, sent photos to seller, they sent a replacement SERVER. No bullshit, no delay, didn't even need to get eBay or PayPal involved.
|
# ? Jan 8, 2020 03:49 |
|
I personally stick to US-based sellers when at all humanly possible, I have less confidence in China to not swap heatspreaders around (on consumer gear) and repolish and relabel engineering samples and other such shenanigans. but I've never had a problem buying parts off ebay. as mentioned the buyer protection can gently caress a seller hard, like you really have up to 6 months if something drastically shits the bed (which it probably won't unless you do something) paradoxically I think server chips may actually be treated better than gamerzzz chips. like yeah sure it's probably run nonstop for 3 years but that's what CPUs are meant to do and there's a virtual certainty that it's never been overvolted/overclocked. I'd take a CPU that's been run hard in a server for 5 years over a CPU that someone has gamed on for 3 years any day. I have an (8-core) e5-1660v3 here ready to go into one of my systems. The fun thing there is they're overclockable and possibly binned better than the gamerzzz cpus. I specifically held out for a very late production batch that are supposed to overclock pretty reliably. right now I have an 8-bay NAS that I'm outgrowing, I'm kinda debating whether I upgrade drives or set up a rack and go for more drives. I got one of those $60 Netapp disk shelves (that guy got hosed so hard on the free shipping lol) or whatever it was from a year ago, I swapped out the IOMs so I could just toss them on a rando LSI SAS card and bring it online. I didn't have a basement when I built my NAS but I bought a house and now I do. In principle I am one SAS card and some cables away from getting that online, once I finish running ethernet. I'm thinking I may look for one of the ASUS X99 boards and plan on swapping up to LRDIMMs someday... edit: nah I guess UDIMMs are down to $165 for 32GB sticks, probably should just suck it up and do TRX40 once IPMI boards come out (really I just need to clean up my plex library, but, lazy) Paul MaudDib fucked around with this message at 06:48 on Jan 8, 2020 |
# ? Jan 8, 2020 04:02 |
|
fatman1683 posted:Any opinions on whether a pair of E5-2603v4s are enough CPU for a medium (~20TB) FreeNAS box? I run a 2620v3 and have given thought to downgrading to a 2603 (which I run on my virtualization box in dual-socket) just for power conservation purposes because I never ever stress the 2620. Paul MaudDib posted:suck it up and do TRX40 once IPMI boards come out yeah this too for me for a variety of reasons. Crunchy Black fucked around with this message at 04:46 on Jan 8, 2020 |
# ? Jan 8, 2020 04:42 |
|
Crunchy Black posted:I run a 2620v3 and have given thought to downgrading to a 2603 (which I run on my virtualization box in dual-socket) just for power conservation purposes because I never ever stress the 2620. Their TDP is the same, though I assume you can hit lower speeds on the 2603. https://ark.intel.com/content/www/us/en/ark/compare.html?productIds=83352,92993 That being said for just file serving basically anything will do. Sure my Synology with its atom cpus pegs out, but I don't really care. It streams movies and files just fine. Most single threaded tasks it just doesn't matter how long it takes, within reason, and you will hit other limits first. Also for computer "refurb" resellers they get everything at basically by-the-ton pricing, I've been the one on the sell side liquidating a thousand+ servers out of a site at once. They have to guarantee they take all of it or they get none of it. Never feel bad asking for a replacement, the worst they can say is "I don't have any more of that exact one, how about one better? Or two? Or a refund? I paid $8 for that $200 server you bought!"
|
# ? Jan 8, 2020 05:01 |
|
So uh, I ordered one of these $50 4xSSD docking stations, it's only JBOD but my wife is making me give up four of my old laptops and I'm not yet ready to part with the SSDs inside of them. It's already arrived but I won't have a chance to play with it until this weekend. It's about the size of a Mayo jar with the fan/lid, maybe a little bigger than the size of my fist with the lid off. The lid seems to just exist for asthetic reasons; technically it has a "cooling fan" but they're ssd so https://www.amazon.com/Sabrent-Drive-Docking-Station-DS-4SSD/dp/B0711L68MS/ Normally I would consider this pretty unremarkable, but the next cheapest device on the market is $75, and after that is $115, and take up considerably more space as they're designed to accept 3.5 traditional as well as 2.5 ssd. This is SSD only. And again, JBOD.
|
# ? Jan 8, 2020 10:32 |
NFS is multithreaded and supported in Windows, macOS, Linux, and basically everywhere else.
|
|
# ? Jan 8, 2020 10:32 |
|
Hadlock posted:So uh, I ordered one of these $50 4xSSD docking stations, it's only JBOD but my wife is making me give up four of my old laptops and I'm not yet ready to part with the SSDs inside of them. It's already arrived but I won't have a chance to play with it until this weekend. It's about the size of a Mayo jar with the fan/lid, maybe a little bigger than the size of my fist with the lid off. The lid seems to just exist for asthetic reasons; technically it has a "cooling fan" but they're ssd so Oh loving hell, it's $118 CAD. That's a $40 CAD premium over the USD price + USD shipping to canada converted to CAD at today's rate (without whatever taxes the govt would put on it). Sigh.`
|
# ? Jan 8, 2020 13:07 |
|
I bought Stablebit DrivePool at the recommendation of this thread. It seems like a fantastic program. Almost too good to be true. I love it! The only thing I am a bit confused about: so lets say my DrivePool drive is F:. I have drives D: and E: making the "F" pool. Shouldn't that get "rid" of the D: and E: drives? Meaning shouldn't it free up drive letters in Windows? Or is really the point of DrivePool to use up all the empty space across a bunch of drives as one drive? I was kind of expecting to get a nice 40TB single drive out of this thing, with that single drive showing up in Windows as a single drive. It's not a big deal if it doesn't do that, I just wanted to make sure I was using the program properly. Would be nice if my TV show collection (which spans two drives) could be all on "one" F:\shows or something.
|
# ? Jan 8, 2020 16:21 |
|
WWWWWWWWWWWWWWWWWW posted:I bought Stablebit DrivePool at the recommendation of this thread. It seems like a fantastic program. Almost too good to be true. I love it! It's been many years since I last used DrivePool, but IIRC it doesn't do anything with your individual drives...they're all still there and usable. I think what it does do is use a special hidden folder at the root of each drive and drops your files into them based upon however you configure it to use free space and whatnot and then groups all of the free space across all of your drives into a virtual drive. When you save to that virtual drive ("F" in your case), it drops files into that hidden folder on the root of your real drives. In other words, your other drives are still usable because DrivePool doesn't take them over it's just saving files to those drives just like you can. Again, this is years-old, barely-remembered knowledge here so take it with a large grain of salt.
|
# ? Jan 8, 2020 16:33 |
|
WWWWWWWWWWWWWWWWWW posted:I bought Stablebit DrivePool at the recommendation of this thread. It seems like a fantastic program. Almost too good to be true. I love it! DrivePool doesn't make any config changes to the drives that make up the pool. Remember, Windows supports mounting drives to folders so if you don't want to see your source drives in the My Computer view you can mount them in a sub-folder somewhere so you only see your DrivePool drive. I mount all of my source drives under C:/PooledDrives/Storage1...StorageN folders.
|
# ? Jan 8, 2020 16:48 |
If one were to draw a Venn-diagram of the people who run SPAN arrays and the people who don't make (programmable and automated) backups with verification, I don't think there'd be much left on either side that wouldn't be overlapped.
|
|
# ? Jan 8, 2020 17:04 |
|
D. Ebdrup posted:If one were to draw a Venn-diagram of the people who run SPAN arrays and the people who don't make (programmable and automated) backups with verification, I don't think there'd be much left on either side that wouldn't be overlapped. Not that you're wrong, but it might not be as bad as you might at first think. I don't really have any experience with any sort of spanning other than WHS and DrivePool and it's been years since then, but it's not exactly what I pre-suppose more conventional span arrays to be like. It operates at a file level instead of something more low level. One of the nice things about DrivePool is that you can select on a per-file, per-folder, or per-regex-matching-against-filepath (or something like that, i don't remember exactly) which files get duplicated across X number of drives. I mean, it's not backup, but it's better in some ways than where I'm at with 80TB of ZFS pools without backups (lol at backing up 80TB).
|
# ? Jan 8, 2020 17:18 |
|
WWWWWWWWWWWWWWWWWW posted:I bought Stablebit DrivePool at the recommendation of this thread. It seems like a fantastic program. Almost too good to be true. I love it! You can turn off the drive letters for the pooled drives, no problem. I do this as well. Just use a drive letter for the Pool itself. Then you have what you wanted.
|
# ? Jan 8, 2020 19:07 |
|
I had drive pool back in 2012/13 and then Microsoft released "storage spaces" with Windows 10 Pro (or was it windows 8?) which is basically the Microsoft supported version of drive pool. It's pretty reliable as it's the basis of Microsoft's Enterprise storage solution(s). As soon as Microsoft released it I switched over and never looked back, and ran it without issue until last fall when I finally became part of the Synology Master Race.
|
# ? Jan 8, 2020 20:17 |
|
Really want to pick up a Powervault M4110 for my M1000e Bladecenter, wonder if itll take SATA.
|
# ? Jan 9, 2020 03:39 |
|
Hadlock posted:I had drive pool back in 2012/13 and then Microsoft released "storage spaces" with Windows 10 Pro (or was it windows 8?) which is basically the Microsoft supported version of drive pool. It's pretty reliable as it's the basis of Microsoft's Enterprise storage solution(s). As soon as Microsoft released it I switched over and never looked back, and ran it without issue until last fall when I finally became part of the Synology Master Race. Storage Spaces is actually really bad though, while DrivePool is really good!
|
# ? Jan 9, 2020 03:46 |
|
|
# ? May 25, 2024 15:22 |
|
Noticed some major latency on I/O operations recently. Looks like it's time for my first hard drive replacement needed. code:
|
# ? Jan 9, 2020 04:36 |