|
So I guess I wasn't clear on Synology's devices...being NAS interface only. I guess I assumed with them having USB ports they could be interfaced through USB. So I'm guessing a standard RAID enclosure is what I need. My needs/concerns are essentially this: Currently my work environment is a single disk that I take the raw data to...parse it...find evidence...make a report. My concern is a failure of this drive while I'm working or before I get around to putting it on a back up would be the the total loss of data as I'm sure you understand. The dream would be to rip straight to the storage device and be able to work on it from there but I'm afraid that USB interface would just be too slow for that? I assume the internal RAID I already have would be faster? (raid controller/bay). What would be the best methodology? Rebuild my RAID as a work environment then move to external RAID as a backup? The features I like on stuff like the QNAP or the Synology is the ability to just hotswap a drive without manually going back and rebuilding. I'd have to do this on a RAID 6 correct? Would the ideal be to setup my work environment on the internal RAID as a RAID 10 and then backup to an external enclosure using multiple drives on RAID 5? Then again the NAS is an attractive option. Device reports are often too large to fit on DVD's for investigators to review or to be dropped into evidence storage. Having a NAS server that people could look at in read only to review their reports would be a really attractive option.
|
#
?
Aug 2, 2016 18:12
|
|
|
|
| # ? May 13, 2024 17:40 |
|
|
Is your computer off the network just because that's how it's set up, or because it's a security requirement? Being able to use NAS or cloud storage does open up a lot of options here. Also, you could get a Synology and just have it connected via Ethernet and it would still work fine without being connected to the Internet or even your facility LAN. Performance over gig-e would probably be adequate to your needs. I'm not expert on external RAID chassis but I can offer up the usual boilerplates. Use RAID6 and not 5 - with the sizes of modern drives, RAID 5 can easily be exposed to data loss. Make sure your backup solution includes protection against ransomware. Just connecting a big storage system and copying files to it can't save you if all your disks get encrypted by the same thing. You want something with snapshots, or something you keep disconnected except during backups, or better yet, both.
|
|
#
?
Aug 2, 2016 18:20
|
|
|
Shachi posted:So I guess I wasn't clear on Synology's devices...being NAS interface only. I guess I assumed with them having USB ports they could be interfaced through USB. So I'm guessing a standard RAID enclosure is what I need. There are a number of 2 disk enclosures you should be able to set up with RAID-1 that will work over USB. This is probably the best for your temp workspace (like this one https://www.amazon.com/MiniPro-FireWire-eSATA-2-Bay-Enclosure/dp/B00P2UAQ3E). This will protect you from single disk failure causing your temp work to be gone. For your archive, you want something like RAID-6 or maybe a more nearline storage, since it is so infrequently accessed, like Unraid or SnapRaid. Both of those allow easily adding new disks, they even allow different sized disks. SnapRaid supports multiple disk parity as well, (RAID-6 is 2 disk parity), so you could really beef up on redundancy if you want. For super long term storage, something like some Bluray discs in a safe is fine, just make sure to burn multiples to help recovery if any of them get damaged. Skandranon fucked around with this message at 18:38 on Aug 2, 2016 |
|
#
?
Aug 2, 2016 18:35
|
|
|
If I connect to the Synology with just ethernet, I'd just do thay through a switch yeah? Also the nature of how Synology software manages the device and encrypts it makes it resistant to the risk of ransom ware? I've got a 5 Bay RAID array in my tower already. I'm thinking of setting it up with 4 drives on a RAID 10 as a work space? That kind of dorks me on potential space being only about 4TB with the drives I have on hand...but then again I'm only working off a 3TB drive at the moment. RAID 10 gives the striping advantage for speed like 0 plus redundancy of 1 correct? I think getting the Synology would give me options for the future of having a cloud server people could access their data on, and it would be separate from my rig so more resistant to a ransome ware attack Shachi fucked around with this message at 18:56 on Aug 2, 2016 |
|
#
?
Aug 2, 2016 18:51
|
|
|
Shachi posted:If I connect to the Synology with just ethernet, I'd just do thay through a switch yeah? RAID-10 is probably overkill for swap space, unless you have multiple people connecting, it is mainly a setup meant for high IOPS situations, like databases. If it's just swap/temp space, a mirror is perfect. RAID-1 also gives a significant speed boost as data can be read from both drives at once, like RAID-0.
|
|
#
?
Aug 2, 2016 18:59
|
|
|
Shachi posted:If I connect to the Synology with just ethernet, I'd just do thay through a switch yeah? In the old days you could use a crossover cable and skip the switch, but I think modern NICs (I haven't had to do this in a loooooong time) might even be able to just go port-to-port on a straight through cable now.
|
|
#
?
Aug 2, 2016 19:10
|
|
|
Modern Ethernet cards (since like... 2005) have auto-sensed Tx/Rx pins so it doesn't matter if you have a straight-through or cross-over cable anymore. In fact, this is how I'm planning to setup a 10GbE "network" at home for connecting some servers to my NAS / SAN in the future via FCoE without needing to pay $800+ for a 10GbE switch. I don't care about going over 1 Gb and < 500 us latency for most of my network connections, so isolating to just centralized storage is optimal anyway (and what you should be doing in the first place with storage topologies - separating storage traffic from network where possible).
|
|
#
?
Aug 2, 2016 19:30
|
|
|
Since I just moved (and found out that 2 of my case fans had seized A+++++), I'm slowly getting my home IT garbage up to snuff. I have a NAS4Free setup with a ZFS RAID-Z2 pool. I want to move to something a bit more flexible and not a fork of a fork. Is FreeNAS and mounting the existing pool seem like the route to go?
|
|
#
?
Aug 2, 2016 20:13
|
|
|
Yeah, I switched from N4F to FreeNAS. I'm not even going crazy with jails and poo poo, I just like the UI better (and the upcoming FreeNAS 10 UI looks even nicer) and it seems to be getting updates much more reliably. It's also got support for mirroring the boot device on two USB sticks, which I'm going to take advantage of once I get a couple of fresh sticks to use. I also switched my compute box from ESXi to Docker on Ubuntu 16.04, and once FreeNAS fully supports Docker I might actually be able to get back to a single system.
|
|
#
?
Aug 2, 2016 20:50
|
|
|
PerrineClostermann posted:So I'm planning on upgrading from my 2600k to something modern sometime in the near future, and I'm hoping to use it to replace my NAS, which is currently running on an e6750 with 6GB of RAM. How easy is it to move my FreeNAS setup to a new machine? Thinking of doing the same with my 2500k. Are you going to reconfigure anything to run at lower power consumption? (Turn off overclocking etc?) Also any issues with the lack of ECC ram?
|
|
#
?
Aug 2, 2016 20:55
|
|
|
Shachi posted:Also the nature of how Synology software manages the device and encrypts it makes it resistant to the risk of ransom ware? I wouldn't say so. The encryption helps most against someone grabbing the disks and stealing the data. It may also prevent someone grabbing the whole Synology device, if the encryption requires inputing a password for the data to become accessible. But when the Synology is up and running then any computer that has write access to the NAS can encrypt the data by some ransomware. But if most of the client computers only have read access to the NAS, then it is immune againts those computers. But backups are the only protection against computers that can write to the NAS, and those computers must not be able to access the backups at the same time.
|
|
#
?
Aug 2, 2016 22:55
|
|
|
There is a snapshot replication plugin for the latest Synology DSM, and it can be configured for scheduled replication. That can buy you a lot of protection from ransomware - even if the malware crawls through all your files, local and NAS, and encrypts them all, you just restore from a snapshot. It doesn't help you if you lose the Synology itself, though, so having a single Synology be your only backup for legally consequential data would scare me. For really long term storage, if you can use the Internet, I'd start to look at something like Amazon Glacier. It's tailor-made for situations where you want to get data off prem and safe but don't need to access it often or fast. Presumably it's OK if someone needs a phone image from four years ago and you tell them it will take time to retrieve it. https://aws.amazon.com/glacier/
|
|
#
?
Aug 2, 2016 23:05
|
|
|
necrobobsledder posted:Modern Ethernet cards (since like... 2005) have auto-sensed Tx/Rx pins so it doesn't matter if you have a straight-through or cross-over cable anymore. In fact, this is how I'm planning to setup a 10GbE "network" at home for connecting some servers to my NAS / SAN in the future via FCoE without needing to pay $800+ for a 10GbE switch. I don't care about going over 1 Gb and < 500 us latency for most of my network connections, so isolating to just centralized storage is optimal anyway (and what you should be doing in the first place with storage topologies - separating storage traffic from network where possible). What cards are you using? I've certainly blown out my 1GbE network now that cheap SSDs are a thing. My other option is running a intel pro dual 1GbE card (forty dollars!) and running it in teaming mode to get 2GbE at the cost of running two cables between the file server and desktop.
|
|
#
?
Aug 2, 2016 23:55
|
|
|
Things may be better now but last time I looked at teaming/bonding/whatever, each individual TCP connection or UDP flow could only use one datalink. You need to have multiple connections to use multiple links. That's not a major drawback if you're configuring a server with many clients, but a big downer for point to point networks.
|
|
#
?
Aug 3, 2016 00:32
|
|
|
Megaman posted:If I have a running Freenas volume that I've created and filled with data that is unencrypted, and I want to now encrypt it, is that impossible? Do I have to recreate everything? Or can I just encrypt it on the fly and preserve all the data? Bump for answers
|
|
#
?
Aug 3, 2016 02:38
|
|
|
Hadlock posted:What cards are you using? I've certainly blown out my 1GbE network now that cheap SSDs are a thing.
|
|
#
?
Aug 3, 2016 03:15
|
|
|
people posted:
Hadlock posted:What cards are you using? I've certainly blown out my 1GbE network now that cheap SSDs are a thing. Megaman posted:Bump for answers necrobobsledder posted:I'm planning on ditching my existing servers for a Xeon-D build and a lot of those motherboards have 10 GbE NICs supported by Linux and FreeBSD. NIC teaming or Etherchannel or other 2-NICs-One-IP type solution really doesn't help for a single data link typically. It'll significantly help, however, if you have multiple clients that are trying to access the resource and you have some I/O capacity leftover on the server. This is fairly common for enterprise environments and is not the same thing as redundant NICs for failover. Speaking of "direct-attached storage" and ideal setups, my dream setup is a FreeBSD machine with encrypted zfs on root and beadm with iSCSI sharing of a zvol on a pool with both L2ARC and (mirrored) SLOG, and a 10G SFP+ NIC in both my server and my workstation, the latter of which will use the iSCSI target boot feature. That way I can get SSD-level preformance for the OS and multi-TB redundant storage for my workstation.
|
|
#
?
Aug 3, 2016 22:13
|
|
about enclosures to attach RAID'd disks via USB
|
priznat posted:Thinking of doing the same with my 2500k. Are you going to reconfigure anything to run at lower power consumption? (Turn off overclocking etc?) I'm not sure. I'd like to try running plex on it, where the cpu power would help. Lack of ecc isn't a huge deal. All operations are vulnerable to flips at all times if you're not using ecc across the board, including anything you do on your home computers. I don't worry about it.
|
|
#
?
Aug 3, 2016 23:28
|
|
|
D. Ebdrup posted:You need a 10Gbps switch to make use of those 10GBaseT NICs (which, on some Xeon-D platforms is a non-SOC Intel X550 in addition to the 1GBase-T SoC NIC) My dream setup is one I don't have to give a drat about because I'll pay some joker $500k / year to make sure my household never has a storage problem and that my wife's Bravo reality shows are at a lower storage tier than my VMs I use professionally. But that just means "Pay for AWS or GCE" at this point.
|
|
#
?
Aug 4, 2016 04:11
|
|
|
necrobobsledder posted:Where'd you read this? I've been reading that all direct cable connections on even 10GbE should work at least from a physical signalling level. I know that there's SFP and SFP+ connectors but that's just connectors that's only part of the equation. The the reason I wanna use SFP+ instead of RJ45 for my storage is lower latency that it offers (about an order of magnitude lower) that it offers, which is important for IOPs (and like I said before, the pool having mirrored SLOG devices for syncronous writes made by iSCSI). BlankSystemDaemon fucked around with this message at 09:19 on Aug 4, 2016 |
|
#
?
Aug 4, 2016 09:02
|
|
|
Do I want WD Reds or HGST Deskstar NAS drives? Biggest difference I see is 5400 vs 7200 rpm but I'm not really sure how much of a difference this makes for streaming media from and overnight scheduled rsyncs for backup. Backblaze likes the HGST drives but they're also using the enterprise drives in their reports so I don't think it's comparable like that. Both are roughly the same price with the WD Reds being a bit cheaper. If 1080p video can be streamed over my LAN either way then I'm leaning towards the WD Red for the less noise and slightly lower energy consumption.
|
|
#
?
Aug 4, 2016 10:27
|
|
|
The difference between 5400 and 7200 rpm drives don't matter if all you're looking to do is satuate 1Gbps. The primary difference between the WD Red with NASware3.0 (essentially v3 of the firmware on WD Reds, but older drives cannot use it) and the HGST Deskstar NAS is that the latter has double the load-unload cycle count, has double the cache, uses slightly more power, supports In-Order Data Delivery and Sense Data Reporting (though both are disabled, you only want to enable it if you know what they do and know that you need them), and has streaming enabled by default. So, if all you care about is streaming 1080p video (the biggest of which take 50Mbps out of a potential 1Gbps), buy the WD Reds and make sure you enable streaming mode as this optimizes the drive for streaming media.
|
|
#
?
Aug 4, 2016 10:51
|
|
|
Of course you want the HGST drives. They are badass. Low RMA rates
|
|
#
?
Aug 4, 2016 14:39
|
|
|
re: the last three posts ^ would one of these https://www.startech.com/HDD/Enclosures/NAS-RAID-Enclosure~S352BMU3N (Dual Bay Gigabit NAS RAID Enclosure) and two of these https://amzn.com/B00HHAJRU0 (HGST Deskstar NAS 3.5-Inch 4TB) plugged into one of these https://amzn.com/B00DES2FQW (ASUS RT-AC56U) Work reliably as a RAID-1 network attached drive where I could point a couple of macs for Time Machine purposes? Could it at the same time work as a simple shared drive for all of the computers on the network?
|
|
#
?
Aug 4, 2016 21:18
|
|
|
duodenum posted:re: the last three posts ^ I think yes to everything, though I am somewhat skeptical of StarTech stuff in general.
|
|
#
?
Aug 5, 2016 02:34
|
|
|
Yeah I wouldn't trust my data in one of those. Shell out for a synology or qnap. https://www.amazon.com/gp/aw/d/B01B...tl9L&ref=plSrch
|
|
#
?
Aug 5, 2016 03:24
|
|
|
Thank you, gents. Synology it is.
|
|
#
?
Aug 5, 2016 04:00
|
|
|
Boris Galerkin posted:Backblaze likes the HGST drives but they're also using the enterprise drives in their reports so I don't think it's comparable like that. FYI: I don't think this is right. For example, that HGST that Fry's keeps putting on sale for $89-$99 shipped that I posted last page is one of the drives that Backblaze uses - HGST HMS5C4040ALE640 Seems to do really well from a reliability standpoint per their stats, as well
|
|
#
?
Aug 5, 2016 04:30
|
|
|
So would a Synology 416 of some variety work for me if I'm looking for RAID-5, Sonarr, Sabnzbd, plex, and possibly a local crashplan backup? I only really need a couple tb generally speaking, so I'm contemplating the slim version cause I think it looks better, is there a preferred drive for those since they can only take 12.5mm height drives?
|
|
#
?
Aug 5, 2016 04:59
|
|
|
Fancy_Lad posted:FYI: I don't think this is right. For example, that HGST that Fry's keeps putting on sale for $89-$99 shipped that I posted last page is one of the drives that Backblaze uses - HGST HMS5C4040ALE640 Wait, what? Fry's sells them for 100 bucks?
|
|
#
?
Aug 5, 2016 06:05
|
|
|
Azhais posted:So would a Synology 416 of some variety work for me if I'm looking for RAID-5, Sonarr, Sabnzbd, plex, and possibly a local crashplan backup? Yeah, but remember those NAS units don't have a ton of CPU power for your Plex transcoding. Also the looks pretty awesome, but remember 2.5" drives are going to be more expensive for the space.
|
|
#
?
Aug 5, 2016 17:20
|
|
|
Synology's web interface is very nice but it is painfully slow with only 256 MB of RAM, so I would highly recommend getting at least 512 if not 1G.
|
|
#
?
Aug 5, 2016 21:05
|
|
|
I hope none of you used raid5 or raid6 with btrfs, because it has been described as fatally flawed after it was found out that the parity caculation is broken beyond easy repair earlier this July.
|
|
#
?
Aug 6, 2016 13:06
|
|
|
D. Ebdrup posted:I hope none of you used raid5 or raid6 with btrfs, because it has been described as fatally flawed after it was found out that the parity caculation is broken beyond easy repair earlier this July. Oops. Not me, but anyone who does should be making GBS threads themselves to transfer. That's ugly.
|
|
#
?
Aug 6, 2016 14:13
|
|
|
Isn't Synology's SHR and SHR2 based on Raid 5/6? If so, does that mean my new 1515+ with btrfs is screwed?
|
|
#
?
Aug 6, 2016 22:03
|
|
|
No, it's not because SHR uses mdadm with ext3 on top, whereas btrfs is more like zfs, ie. a combined logical volume manager and software raid. I've actually been looking into the history of btrfs a bit, and it seems that people took an announcement of the disk format in btrfs being stable to mean that the entire thing was production ready (which it doesn't mean, it means that if there is a new change, the old format will still be mountable on all newer kernels) - because I can't actually find any official word that the parity calculation has been declared production ready. BlankSystemDaemon fucked around with this message at 11:07 on Aug 7, 2016 |
|
#
?
Aug 7, 2016 10:57
|
|
|
EDIT: So apparently I completely missed the fact that setting file integrity on a directory (note the flag in powershell is -File) will basically force every file to have an integrity stream and is recursive, which renders about half of below moot. The rest of the findings remain unaltered. About once every couple of months I've asked this thread about Windows Storage Spaces and Integrity Streams, which is ReFS' way to prevent bit rot. Noone ever had a good answer, so I've spent several hours with a VM and a hex editor to simulate bit rot, I'm thoroughly disappointed with Microsoft's implementation. tl;dr version: * Apparently enabling or disabling integrity streams must be done per-file. I've researched every where and couldn't find a way to create files with this flag on by default. The cmdlet that controls this is Set-FileIntegrity * Even with a storage space that support integrity streams, the scheduled task that actually does the scanning is disabled unless you explicitly enable it. The scheduled task that is on by default only scans ReFS' internal data structures for corruption. It's the other job that's disabled by default that does full scans. * Once you get past that big gotcha, the job does work. It detected the error and fixed the corrupt block. For some reason these are just "warnings" in the Event Viewer and not "errors" -- which seems like a bizarre decision considering how much trivial stuff is at an error level in Windows and too many of these warnings means your disk is dying. * Even with three-way data duplication, Set-FileIntegrity must be explicitly set on each file for windows to scan it. This was the shocker to me, because the docs basically state this shouldn't be necessary since with 3-way duplication if there is corruption detected, windows can use voting to determine the good copy. I'm still not convinced this is not a bug, and might open a bug report and see what happens. * The md5 hash of the corrupt and uncorrupt file were the same. This isn't that surprising since there was only a 33% chance of picking the disk with the corruption. I didn't want to spend the time to figure out what happens if it picks the disk with the corruption because at this point we're already past the point where I am writing off ReFS + Storage Spaces where your primary concern is integrity. Avoid Storage Spaces as an alternative to ZFS for integrity. I'm really disappointed by this because I just wanted to throw up a 3-disk 2-way mirror to hold me over until I build my monster NAS. Apparently that's not an option. All Microsoft really needs to do to fix this is to have an option to create files with Integrity Streams on by default. If such an option exists, I've scoured the powershell cmdlets and the internet and haven't found it. EDIT2: After playing around with it some more post-discovering the recursive Set-FileIntegrity directory option (important note: only applies to *new* files and directories created), I think I can live with it until I get my big NAS set up. There are so many little gotchas though that I strongly encourage anyone else in the same position to spend some time playing around with it in a VM, as well as read the huge technet article on Storage Spaces and ReFS. Chuu fucked around with this message at 05:26 on Aug 8, 2016 |
|
#
?
Aug 8, 2016 03:36
|
|
|
Krailor posted:If you're moving over to windows than instead of storage spaces I'd suggest taking a look at DrivePool. It allows you to set file duplication at the folder level and supports drives of different sizes so you won't lose any space like you would in RAID5 if you're using mismatched drives. It's been a couple months, and my setup has been working mostly good, but with one big (well, "first-world big") problem. To recap: I have 5x drives in this enclosure, and I have the enclosure hooked up to my PC using this eSata to USB3.0 adapter. I then have all 5 drives mounted in Windows 10, and I have them all pooled together as one big drive using DriveBender. This works almost flawlessly. The big problem is that at least 50% of the time, when I try to access the pooled drive, it will take like a full minute to load the drive. Often times it fails when trying to access the drive, and I have to try to access it again before it actually loads. I know that this is because one or more of the individual drives are becoming unavailable, but I don't know why this is happening. I don't know if they are going into power-save mode, or sleep mode, or if the enclosure doesn't have enough power to keep them all powered up at once or if there's an issue with the eSata to USB adapter or whatever else it could be. I know that I've gone through every place that I know of to disable any kind of sleep mode or power save mode and it still happens. I even downloaded KeepAwakeHD, a program that writes a small text file to the root of your connected drives at 5 minute intervals meant to make sure the drives stay awake. And still this happens. It doesn't seem to be linked to any specific amount of time passing. Sometimes I'll be browsing my drive, and then go to check something literally 5 minutes later and it will be asleep. Sometimes I'll come back after hours of not using my computer, and the drive will open up right away. It's incredibly frustrating, and I'd like to figure out what the issue is. The individual drives don't become unmounted, and I can't tell if they're actually spinning down or not because they're in an enclosure. Any ideas?
|
|
#
?
Aug 8, 2016 03:48
|
|
|
Seagate showed off a 60TB SSD today http://www.pcmag.com/news/346904/seagate-unveils-60tb-solid-state-drive
|
|
#
?
Aug 10, 2016 20:12
|
|
|
|
| # ? May 13, 2024 17:40 |
|
|
I'll finally be able to download most of my steam library!
|
|
#
?
Aug 10, 2016 21:04
|
|